Abstract

In this work, we propose a multi-camera system wherewe combine a main high-quality camera with two low-resauxiliary cameras. The auxiliary cameras are well calibrated and act as a passive depth sensor by generating dis-parity maps. The main camera has an interchangeable lensand can produce good quality images at high resolution.Our goal is, given the low-res depth map from the auxiliary cameras, generate a depth map from the viewpoint of the main camera. The advantage of our system, compared toother systems such as light-field cameras or RGBD sensors, is the ability to generate a high-resolution color image witha complete depth map, without sacrificing resolution and with minimal auxiliary hardware. Since the main camera has an interchangeable lens, it cannot be calibrated beforehand, and directly applying stereo matching on it and either of the auxiliary cameras often leads to unsatisfactory results. Utilizing both the calibrated cameras at once, we propose a novel approach to better estimate the disparity map of the main camera. Thenby combining the defocus cue of the main camera, the disparity map can be further improved. We demonstrate the performance of our algorithm on various scenes.