Abstract.
An original concept for computing instantaneous 3D pose and 3D velocity of fast moving ob jects using a single view is proposed, implemented and validated. It takes advantage of the image deforma- tions induced by rolling shutter in CMOS image sensors. First of all, after analysing the rolling shutter phenomenon, we introduce an origi- nal model of the image formation when using such a camera, based on a general model of moving rigid sets of 3D points. Using 2D-3D point correspondences, we derive two complementary methods, compensating for the rolling shutter deformations to deliver an accurate 3D pose and exploiting them to also estimate the full 3D velocity. The first solution is a general one based on non-linear optimization and bundle adjust- ment, usable for any ob ject, while the second one is a closed-form linear solution valid for planar ob jects. The resulting algorithms enable us to transform a CMOS low cost and low power camera into an innovative and powerful velocity sensor. Finally, experimental results with real data confirm the relevance and accuracy of the approach.