Abstract

We present an algorithm for marker-less performance cap- ture of interacting humans using only three hand-held Kinect cameras. Our method reconstructs human skeletal poses, deforming surface geom- etry and camera poses for every time step of the depth video. Skeletal configurations and camera poses are found by solving a joint energy min- imization problem which optimizes the alignment of RGBZ data from all cameras, as well as the alignment of human shape templates to the Kinect data. The energy function is based on a combination of geometric correspondence finding, implicit scene segmentation, and correspondence finding using image features. Only the combination of geometric and pho- tometric correspondences and the integration of human pose and cam- era pose estimation enables reliable performance capture with only three sensors. As opposed to previous performance capture methods, our al- gorithm succeeds on general uncontrolled indoor scenes with potentially dynamic background, and it succeeds even if the cameras are moving.