ScanComplete: Large-Scale Scene Completion and
Semantic Segmentation for 3D Scans
Abstract
We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input
and predicting a complete 3D model along with per-voxel
semantic labels. The key contribution of our method is its
ability to handle large scenes with varying spatial extent,
managing the cubic growth in data size as scene size increases. To this end, we devise a fully-convolutional generative 3D CNN model whose filter kernels are invariant to
the overall scene size. The model can be trained on scene
subvolumes but deployed on arbitrarily large scenes at test
time. In addition, we propose a coarse-to-fine inference
strategy in order to produce high-resolution output while
also leveraging large input context sizes. In an extensive
series of experiments, we carefully evaluate different model
design choices, considering both deterministic and probabilistic models for completion and semantic inference. Our
results show that we outperform other methods not only in
the size of the environments handled and processing effi-
ciency, but also with regard to completion quality and semantic segmentation performance by a significant margin