Abstract

An error occurs in graph-based keypoint matching when key- points in two different images are matched by an algorithm but do not correspond to the same physical point. Most previous methods acquire keypoints in a black-box manner, and focus on developing better algo- rithms to match the provided points. However to study the complete performance of a matching system one has to study errors through the whole matching pipeline, from keypoint detection, candidate selection to graph optimisation. We show that in the full pipeline there are six differ- ent types of errors that cause mismatches. We then present a matching framework designed to reduce these errors. We achieve this by adapt- ing keypoint detectors to better suit the needs of graph-based matching, and achieve better graph constraints by exploiting more information from their keypoints. Our framework is applicable in general images and can handle clutter and motion discontinuities. We also propose a method to identify many mismatches a posteriori based on Left-Right Consis- tency inspired by stereo matching due to the asymmetric way we detect keypoints and define the graph.