Abstract
This paper proposes a data-driven approach for image
alignment. Our main contribution is a novel network architecture that combines the strengths of convolutional neural
networks (CNNs) and the Lucas-Kanade algorithm. The
main component of this architecture is a Lucas-Kanade
layer that performs the inverse compositional algorithm on
convolutional feature maps. To train our network, we develop a cascaded feature learning method that incorporates
the coarse-to-fine strategy into the training process. This
method learns a pyramid representation of convolutional
features in a cascaded manner and yields a cascaded network that performs coarse-to-fine alignment on the feature
pyramids. We apply our model to the task of homography
estimation, and perform training and evaluation on a large
labeled dataset generated from the MS-COCO dataset. Experimental results show that the proposed approach signifi-
cantly outperforms the other methods