Abstract

We are interested in large-scale image classi fication and especially in the setting where images corresponding to new or existing classes are con- tinuously added to the training set. Our goal is to devise classi fiers which can incorporate such images and classes on-the-fly at (near) zero cost. We cast this problem into one of learning a metric which is shared across all classes and ex- plore k-nearest neighbor (k-NN) and nearest class mean (NCM) classi fiers. We learn metrics on the ImageNet 2010 challenge data set, which contains more than 1.2M training images of 1K classes. Surprisingly, the NCM classi fier compares favorably to the more flexible k-NN classi fier, and has comparable performance to linear SVMs. We also study the generalization performance, among others by using the learned metric on the ImageNet-10K dataset, and we obtain competi- tive performance. Finally, we explore zero-shot classi fication, and show how the zero-shot model can be combined very effectively with small training datasets.