Abstract

In this paper, we propose a multi-manifold deep met-ric learning (MMDML) method for image set classification,which aims to recognize an object of interest from a set ofimage instances captured from varying viewpoints or undervarying illuminations. Motivated by the fact that manifoldcan be effectively used to model the nonlinearity of sam-ples in each image set and deep learning has demonstratedsuperb capability to model the nonlinearity of samples, wepropose a MMDML method to learn multiple sets of nonlin-ear transformations, one set for each object class, to non-linearly map multiple sets of image instances into a sharedfeature subspace, under which the manifold margin of dif-ferent class is maximized, so that both discriminative andclass-specific information can be exploited, simultaneous-ly. Our method achieves the state-of-the-art performanceon five widely used datasets.