Abstract
3D reconstruction of dynamic fluid surfaces is an open
and challenging problem in computer vision. Unlike previous approaches that reconstruct each surface point independently and often return noisy depth maps, we propose
a novel global optimization-based approach that recovers
both depths and normals of all 3D points simultaneously.
Using the traditional refraction stereo setup, we capture the
wavy appearance of a pre-generated random pattern, and
then estimate the correspondences between the captured images and the known background by tracking the pattern.
Assuming that the light is refracted only once through the
fluid interface, we minimize an objective function that incorporates both the cross-view normal consistency constraint
and the single-view normal consistency constraints. The key
idea is that the normals required for light refraction based
on Snell’s law from one view should agree with not only
the ones from the second view, but also the ones estimated
from local 3D geometry. Moreover, an effective reconstruction error metric is designed for estimating the refractive
index of the fluid. We report experimental results on both
synthetic and real data demonstrating that the proposed approach is accurate and shows superiority over the conventional stereo-based method