Abstract

This paper proposes a method for estimating the 3D bodyshape of a person with robustness to clothing. We formulatethe problem as optimization over the manifold of valid depthmaps of body shapes learned from synthetic training data.The manifold itself is represented using a novel data struc-ture, a Multi-Resolution Manifold Forest (MRMF), whichcontains vertical edges between tree nodes as well as hori-zontal edges between nodes across trees that correspond tooverlapping partitions. We show that this data structure allows both efficient localization and navigation on the manifold for on-the-fly building of local linear models (manifold charting). We demonstrate shape estimation of clothed users, showing significant improvement in accuracy over global shape models and models using pre-computed clusters. We further compare the MRMF with alternative manifold charting methods on a public dataset for estimating 3D motion from noisy 2D marker observations, obtainingstate-of-the-art results.