Abstract

The goal of most image based rendering systems can be stated as follows: given a set of pictures taken from various vantage points, synthesize the image that would be obtained from a novel view- point. In this paper we present a novel approach to view synthesis which hinges on the observation that human viewers tend to be quite sensitive to the motion of features in the image corresponding to intensity discon- tinuities or edges. Our system focuses its efiorts on recovering the 3D position of these features so that their motions can be synthesized cor- rectly. In the current implementation these feature points are recovered from image sequences by employing the epipolar plane image (EPI) anal- ysis techniques proposed by Bolles, Baker, and Marimont. The output of this procedure resembles the output of an edge extraction system where the edgels are augmented with accurate depth information. This method has the advantage of producing accurate depth estimates for most of the salient features in the scene including those corresponding to occluding contours. We will demonstrate that it is possible to produce compelling novel views based on this information. The paper will also describe a principled approach to reasoning about the 3D structure of the scene based on the quasi-sparse features returned by the EPI analysis. This analysis allows us to correctly reproduce oc- clusion and disocclusion efiects in the synthetic views without requiring dense correspondences. Importantly, the technique could also be used to analyze and refine the 3-D results returned by range finders, stereo sys- tems or structure from motion algorithms. Results obtained by applying the proposed techniques to actual image data sets are presented. Keywords: Structure From Motion, Surface Geometry, Image Based Rendering