Abstract

We propose a new 3D model of the human body that isboth realistic and part-based. The body is represented by agraphical model in which nodes of the graph correspond tobody parts that can independently translate and rotate in 3Dand deform to represent different body shapes and to cap-ture pose-dependent shape variations. Pairwise potentialsdefine a “stitching cost” for pulling the limbs apart, giv-ing rise to the stitched puppet (SP) model. Unlike existingrealistic 3D body models, the distributed representation fa-cilitates inference by allowing the model to more effectively explore the space of poses, much like existing 2D pictorial structures models. We infer pose and body shape using a form of particle-based max-product belief propagation. This gives SP the realism of recent 3D body models with the computational advantages of part-based models. We apply SP to two challenging problems involving estimating human shape and pose from 3D data. The first is the FAUST mesh alignment challenge, where ours is the first method to suc-cessfully align all 3D meshes with no pose prior. The secondinvolves estimating pose and shape from crude visual hull representations of complex body movements.