Abstract

The paper presents a method for multi-dimensional registra- tion of two video streams. The sequences are captured by two hand-held cameras moving independently with respect to each other, both observ- ing one ob ject rigidly moving apart from the background. The method is based on uncalibrated Structure-from-Motion (SfM) to extract 3D mod- els for the foreground ob ject and the background, as well as for their relative motion. It fixes the relative scales between the scene parts within and between the videos. It also provides the registration between all par- tial 3D models, and the temporal synchronization between the videos. The crux is that not a single point on the foreground or background needs to be in common between both video streams. Extensions to more than two cameras and multiple foreground ob jects are possible.