Abstract. We propose a new method for the absolute camera pose problem (PnP) which handles Rolling Shutter (RS) effects. Unlike all existing
methods which perform 3D-2D registration after augmenting the Global
Shutter (GS) projection model with the velocity parameters under various kinematic models, we propose to use local differential constraints.
These are established by drawing an analogy with Shape-from-Template
(SfT). The main idea consists in considering that RS distortions due to
camera ego-motion during image acquisition can be interpreted as virtual
deformations of a template captured by a GS camera. Once the virtual
deformations have been recovered using SfT, the camera pose and egomotion are computed by registering the deformed scene on the original
template. This 3D-3D registration involves a 3D cost function based on
the Euclidean point distance, more physically meaningful than the reprojection error or the algebraic distance based cost functions used in
previous work. Results on both synthetic and real data show that the
proposed method outperforms existing RS pose estimation techniques in
terms of accuracy and stability of performance in various configurations.