Abstract

Many new computer vision applications are utilizing large-scale data- sets of places derived from the many billions of photos on the Web. Such applica- tions often require knowledge of the visual connectivity structure of these image collections—describing which images overlap or are otherwise related—and an important step in understanding this structure is to identify connected components of this underlying image graph. As the structure of this graph is often initially un- known, this problem can be posed as one of exploring the connectivity between images as quickly as possible, by intelligently selecting a subset of image pairs for feature matching and geometric veri fication, without having to test all O(n2 ) possible pairs. We propose a novel, scalable algorithm called MatchMiner that efficiently explores visual relations between images, incorporating ideas from rel- evance feedback to improve decision making over time, as well as a simple yet effective rank distance measure for detecting outlier images. Using these ideas, our algorithm automatically prioritizes image pairs that can potentially connect or contribute to large connected components, using an information-theoretic al- gorithm to decide which image pairs to test next. Our experimental results show that MatchMiner can efficiently find connected components in large image col- lections, signi ficantly outperforming state-of-the-art image matching methods.