Abstract

In this paper, we propose a non-local structured priorfor volumetric multi-view 3D reconstruction. Towards thisgoal, we present a novel Markov random field model basedon ray potentials in which assumptions about large 3D sur-face patches such as planarity or Manhattan world con-straints can be efficiently encoded as probabilistic priors. We further derive an inference algorithm that reasonsjointly about voxels, pixels and image segments, and es-timates marginal distributions of appearance, occupancy,depth, normals and planarity. Key to tractable inferenceis a novel hybrid representation that spans both voxel andpixel space and that integrates non-local information from2D image segmentations in a principled way. We compareour non-local prior to commonly employed local smooth-ness assumptions and a variety of state-of-the-art volumetric reconstruction baselines on challenging outdoor scenes with textureless and reflective surfaces. Our experiments indicate that regularizing over larger distances has the potential to resolve ambiguities where local regularizers fail.