Fast Light Field Reconstruction With Deep
Coarse-To-Fine Modeling of Spatial-Angular Clues
Abstract. Densely-sampled light fields (LFs) are beneficial to many applications
such as depth inference and post-capture refocusing. However, it is costly and
challenging to capture them. In this paper, we propose a learning based algorithm
to reconstruct a densely-sampled LF fast and accurately from a sparsely-sampled
LF in one forward pass. Our method uses computationally efficient convolutions
to deeply characterize the high dimensional spatial-angular clues in a coarse-to-
fine manner. Specifically, our end-to-end model first synthesizes a set of intermediate novel sub-aperture images (SAIs) by exploring the coarse characteristics
of the sparsely-sampled LF input with spatial-angular alternating convolutions.
Then, the synthesized intermediate novel SAIs are efficiently refined by further
recovering the fine relations from all SAIs via guided residual learning and stride-
2 4-D convolutions. Experimental results on extensive real-world and synthetic
LF images show that our model can provide more than 3 dB advantage in reconstruction quality in average than the state-of-the-art methods while being computationally faster by a factor of 30. Besides, more accurate depth can be inferred
from the reconstructed densely-sampled LFs by our method