HybridFusion: Real-Time Performance Capture
Using a Single Depth Sensor and Sparse IMUs
Abstract. We propose a light-weight yet highly robust method for realtime human performance capture based on a single depth camera and
sparse inertial measurement units (IMUs). Our method combines nonrigid surface tracking and volumetric fusion to simultaneously reconstruct challenging motions, detailed geometries and the inner human
body of a clothed subject. The proposed hybrid motion tracking algorithm and efficient per-frame sensor calibration technique enable nonrigid surface reconstruction for fast motions and challenging poses with
severe occlusions. Significant fusion artifacts are reduced using a new
confidence measurement for our adaptive TSDF-based fusion. The above
contributions are mutually beneficial in our reconstruction system, which
enable practical human performance capture that is real-time, robust,
low-cost and easy to deploy. Experiments show that extremely challenging performances and loop closure problems can be handled successfully