Abstract

Multiple-view geometry and structure-from-motion are well established techniques to compute the structure of a moving rigid ob ject. These techniques are all based on strong algebraic constraints imposed by the rigidity of the ob ject. Unfortunately, many scenes of in- terest, e.g. faces or cloths, are dynamic and the rigidity constraint no longer holds. Hence, there is a need for non-rigid structure-from-motion (NRSfM) methods which can deal with dynamic scenes. A prominent framework to model deforming and moving non-rigid ob jects is the fac- torization technique where the measurements are assumed to lie in a low-dimensional subspace. Many different formulations and variations for factorization-based NRSfM have been proposed in recent years. However, due to the complex interactions between several subspaces, the distin- guishing properties between two seemingly related approaches are often unclear. For example, do two approaches just vary in the optimization method used or is really a different model beneath? In this paper, we show that these NRSfM factorization approaches are most naturally modeled with tensor algebra. This results in a clear pre- sentation which subsumes many previous techniques. In this regard, this paper brings several strings of research together and provides a unified point of view. Moreover, the tensor formulation can be extended to the case of a camera network where multiple static affine cameras observe the same deforming and moving non-rigid ob ject. Thanks to the insights gained through this tensor notation, a closed-form and an efficient iter- ative algorithm can be derived which provide a reconstruction even if there are no feature point correspondences at all between different cam- eras. An evaluation of the theory and algorithms on motion capture data show promising results.