Abstract

This paper addresses the problem of clustering images of ob- jects seen from different viewpoints. That is, given an unlabelled set of images of n ob jects, we seek an unsupervised algorithm that can group the images into n disjoint subsets such that each subset only contains images of a single ob ject. We formulate this clustering problem under a very broad geometric framework. The theme is the interplay between the geometry of appearance manifolds and the symmetry of the 2D affine group. Specifically, we identify three important notions for image cluster- ing: the L2 distance metric of the image space, the local linear structure of the appearance manifolds, and the action of the 2D affine group in the image space. Based on these notions, we propose a new image clus- tering algorithm. In a broad outline, the algorithm uses the metric to determine a neighborhood structure in the image space for each input image. Using local linear structure, comparisons (affinities) between im- ages are computed only among the neighbors. These local comparisons are agglomerated into an afinity matrix, and a spectral clustering algo- rithm is used to yield the final clustering result. The technical part of the algorithm is to make all of these compatible with the action of the 2D affine group. Using human face images and images from the COIL database, we demonstrate experimentally that our algorithm is effective in clustering images (according to o jbect identity) where there is a large range of pose variation.