Abstract

In this paper, we consider scenes that are immersed in trans- parent refractive media with a dynamic surface. We take the first steps to reconstruct both the 3D fluid surface shape and the 3D structure of immersed scene simultaneously by utilizing distortion and defocus clues. We demonstrate that the images captured through a refractive dynamic fluid surface are the distorted and blurred versions of all-in-focused (AIF) images captured through a flat fluid surface. The amounts of distortion and refractive blur are formulated by the shape of fluid surface, scene depth and camera parameters, based on our refractive geometry model of a finite aperture imaging system. An iterative optimization algorithm is proposed to reconstruct the distortion and immersed scene depth, which are then used to infer the 3D fluid surface. We validate and demonstrate the effectiveness of our approach on a variety of synthetic and real scenes under different fluid surfaces.