Abstract. Many recent energy-based methods for optical flow estimation rely on a good initialization that is typically provided by some kind
of feature matching. So far, however, these initial matching approaches
are rather general: They do not incorporate any additional information
that could help to improve the accuracy or the robustness of the estimation. In particular, they do not exploit potential cues on the camera
poses and the thereby induced rigid motion of the scene. In the present
paper, we tackle this problem. To this end, we propose a novel structurefrom-motion-aware PatchMatch approach that, in contrast to existing
matching techniques, combines two hierarchical feature matching methods: a recent two-frame PatchMatch approach for optical flow estimation
(general motion) and a specifically tailored three-frame PatchMatch approach for rigid scene reconstruction (SfM). While the motion PatchMatch serves as baseline with good accuracy, the SfM counterpart takes
over at occlusions and other regions with insufficient information. Experiments with our novel SfM-aware PatchMatch approach demonstrate
its usefulness. They not only show excellent results for all major benchmarks (KITTI 2012/2015, MPI Sintel), but also improvements up to 50%
compared to a PatchMatch approach without structure information