Abstract
We consider the problem of depth-based robust 3D facial
pose tracking under unconstrained scenarios with heavy occlusions and arbitrary facial expression variations. Unlike the previous depth-based discriminative or data-driven
methods that require sophisticated training or manual intervention, we propose a generative framework that unifies
pose tracking and face model adaptation on-the-fly. Particularly, we propose a statistical 3D face model that owns
the flexibility to generate and predict the distribution and
uncertainty underlying the face model. Moreover, unlike
prior arts employing the ICP-based facial pose estimation,
we propose a ray visibility constraint that regularizes the
pose based on the face model’s visibility against the input
point cloud, which augments the robustness against the occlusions. The experimental results on Biwi and ICT-3DHP
datasets reveal that the proposed framework is effective and
outperforms the state-of-the-art depth-based methods