Abstract
In this paper, we address the problem of 3D object mesh
reconstruction from RGB videos. Our approach combines
the best of multi-view geometric and data-driven methods
for 3D reconstruction by optimizing object meshes for multiview photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise
image alignment problem for each mesh face projection. Our
approach allows us to update shape parameters from the
photometric error without any depth or mask information.
Moreover, we show how to avoid a degeneracy of zero photometric gradients via rasterizing from a virtual viewpoint.
We demonstrate 3D object mesh reconstruction results from
both synthetic and real-world videos with our photometric
mesh optimization, which is unachievable with either na¨?ve
mesh generation networks or traditional pipelines of surface
reconstruction without heavy manual post-processing.