Abstract

Specular reflections present difficulties for many areas of computer vision such as stereo and segmentation. To separate specu- lar and diffiuse reflection components, previous approaches generally re- quire accurate segmentation, regionally uniform reflectance or structured lighting. To overcome these limiting assumptions, we propose a method based on color analysis and multibaseline stereo that simultaneously es- timates the separation and the true depth of specular refiections. First, pixels with a specular component are detected by a novel form of color histogram difierencing that utilizes the epipolar constraint. This process uses relevant data from all the stereo images for robustness, and ad- dresses the problem of color occlusions. Based on the Lambertian model of diffiuse refiectance, stereo correspondence is then employed to compute for specular pixels their corresponding diffiuse components in other views. The results of color-based detection aid the stereo correspondence, which determines both separation and true depth of specular pixels. Our ap- proach integrates color analysis and multibaseline stereo in a synergistic manner to yield accurate separation and depth, as demonstrated by our results on synthetic and real image sequences.