Abstract
We introduce a data-driven approach to complete partial
3D shapes through a combination of volumetric deep neural
networks and 3D shape synthesis. From a partially-scanned
input shape, our method first infers a low-resolution – but
complete – output. To this end, we introduce a 3D-EncoderPredictor Network (3D-EPN) which is composed of 3D convolutional layers. The network is trained to predict and fill
in missing data, and operates on an implicit surface representation that encodes both known and unknown space.
This allows us to predict global structure in unknown areas at high accuracy. We then correlate these intermediary results with 3D geometry from a shape database at test
time. In a final pass, we propose a patch-based 3D shape
synthesis method that imposes the 3D geometry from these
retrieved shapes as constraints on the coarsely-completed
mesh. This synthesis process enables us to reconstruct finescale detail and generate high-resolution output while respecting the global mesh structure obtained by the 3D-EPN.
Although our 3D-EPN outperforms state-of-the-art completion method, the main contribution in our work lies in the
combination of a data-driven shape predictor and analytic
3D shape synthesis. In our results, we show extensive evaluations on a newly-introduced shape completion benchmark
for both real-world and synthetic data.