Abstract

When a purely rotating camera observes a general scene, overlapping views are related by a parallax-free warp which can be esti- mated by direct image alignment methods that iterate to optimise photo- consistency. However, building globally consistent mosaics from video has usually been tackled as an off-line task, while sequential methods suitable for real-time implementation have often suffered from long-term drift. In this paper we present a high performance real-time video mosaicing algo- rithm based on parallel image alignment via ESM (Efficient Second-order Minimisation) and global optimisation of a map of keyframes over the whole viewsphere. We present real-time results for drift-free camera rota- tion tracking and globally consistent spherical mosaicing from a variety of cameras in real scenes, demonstrating high global accuracy and the ability to track very rapid rotation while maintaining solid 30Hz opera- tion. We also show that automatic camera calibration refinement can be straightforwardly built into our framework.