Abstract
We present an approach to accelerate multi-view stereo
(MVS) by prioritizing computation on image patches that
are likely to produce accurate 3D surface reconstructions.
Our key insight is that the accuracy of the surface reconstruction from a given image patch can be predicted significantly faster than performing the actual stereo matching.
The intuition is that non-specular, fronto-parallel, in-focus
patches are more likely to produce accurate surface reconstructions than highly specular, slanted, blurry patches —
and that these properties can be reliably predicted from the
image itself. By prioritizing stereo matching on a subset of
patches that are highly reconstructable and also cover the
3D surface, we are able to accelerate MVS with minimal
reduction in accuracy and completeness. To predict the reconstructability score of an image patch from a single view,
we train an image-to-reconstructability neural network: the
I2RNet. This reconstructability score enables us to effi-
ciently identify image patches that are likely to provide the
most accurate surface estimates before performing stereo
matching. We demonstrate that the I2RNet, when trained
on the ScanNet dataset, generalizes to the DTU and Tanks
& Temples MVS datasets. By using our I2RNet with an existing MVS implementation, we show that our method can
achieve more than a 30? speed-up over the baseline with
only an minimal loss in completeness.