Abstract

We propose a novel method for template matching in unconstrained environments. Its essence is the Best-BuddiesSimilarity (BBS), a useful, robust, and parameter-free simi-larity measure between two sets of points. BBS is based oncounting the number of Best-Buddies Pairs (BBPs)—pairsof points in source and target sets, where each point is thenearest neighbor of the other. BBS has several key featuresthat make it robust against complex geometric deformationsand high levels of outliers, such as those arising from background clutter and occlusions. We study these properties, provide a statistical analysis that justifies them, and demonstrate the consistent success of BBS on a challenging realworld dataset.