Weakly-Supervised Discovery of Geometry-Aware Representationfor 3D Human Pose Estimation
Abstract
Recent studies have shown remarkable advances in 3D
human pose estimation from monocular images, with the
help of large-scale in-door 3D datasets and sophisticated
network architectures. However, the generalizability to different environments remains an elusive goal.
In this work, we propose a geometry-aware 3D representation for the human pose to address this limitation by
using multiple views in a simple auto-encoder model at the
training stage and only 2D keypoint information as supervision. A view synthesis framework is proposed to learn the
shared 3D representation between viewpoints with synthesizing the human pose from one viewpoint to the other one.
Instead of performing a direct transfer in the raw imagelevel, we propose a skeleton-based encoder-decoder mechanism to distil only pose-related representation in the latent
space. A learning-based representation consistency constraint is further introduced to facilitate the robustness of
latent 3D representation. Since the learnt representation
encodes 3D geometry information, mapping it to 3D pose
will be much easier than conventional frameworks that use
an image or 2D coordinates as the input of 3D pose estimator. We demonstrate our approach on the task of 3D human pose estimation. Comprehensive experiments on three
popular benchmarks show that our model can significantly
improve the performance of state-of-the-art methods with
simply injecting the representation as a robust 3D prior.