Abstract

Learning the distance metric between pairs of examplesis of great importance for learning and visual recognition.With the remarkable success from the state of the art convo-lutional neural networks, recent works [1, 31] have shownpromising results on discriminatively training the networksto learn semantic feature embeddings where similar exam-ples are mapped close to each other and dissimilar exam-ples are mapped farther apart. In this paper, we describe analgorithm for taking full advantage of the training batchesin the neural network training by lifting the vector of pair-wise distances within the batch to the matrix of pairwisedistances. This step enables the algorithm to learn the stateof the art feature embedding by optimizing a novel structured prediction objective on the lifted problem. Additionally, we collected Stanford Online Products dataset: 120k images of 23k classes of online products for metric learn-ing. Our experiments on the CUB-200-2011 [37], CARS196[19], and Stanford Online Products datasets demonstrate significant improvement over existing deep feature embedding methods on all experimented embedding sizes with the GoogLeNet [33] network. The source code and the dataset are available at: https://github.com/rksltnl/ Deep-Metric-Learning-CVPR16.