Abstract

Many visual search and matching systems represent images using sparse sets of “visual words”: descriptors that have been quantized by assignment to the best-matching symbol in a discrete vocabulary. Er- rors in this quantization procedure propagate throughout the rest of the system, either harming performance or requiring correction using addi- tional storage or processing. This paper aims to reduce these quantization errors at source, by learning a pro jection from descriptor space to a new Euclidean space in which standard clustering techniques are more likely to assign matching descriptors to the same cluster, and non-matching descriptors to different clusters. To achieve this, we learn a non-linear transformation model by mini- mizing a novel margin-based cost function, which aims to separate match- ing descriptors from two classes of non-matching descriptors. Training data is generated automatically by leveraging geometric consistency. Scalable, stochastic gradient methods are used for the optimization. For the case of particular ob ject retrieval, we demonstrate impressive gains in performance on a ground truth dataset: our learnt 32-D de- scriptor without spatial re-ranking outperforms a baseline method using 128-D SIFT descriptors with spatial re-ranking.