Abstract

We address the problem of comparing sets of images for ob- ject recognition, where the sets may represent arbitrary variations in an ob ject’s appearance due to changing camera pose and lighting condi- tions. The concept of Canonical Correlations (also known as principal angles) can be viewed as the angles between two subspaces. As a way of comparing sets of vectors or images, canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the classical parametric distribution-based and non-parametric sample-based meth- ods. Here, this is demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effiectiveness, a novel discriminative learning over sets is proposed for ob ject recognition. Specifically, inspired by classical Linear Discriminant Analysis (LDA), we develop a linear discriminant func- tion that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. The proposed method significantly outperforms the state-of-the-art methods on two different ob ject recognition problems using face image sets with arbi- trary motion captured under different illuminations and image sets of five hundred general ob ject categories taken at different views.