Abstract

We present a photometric stereo method for non-rigid ob- jects of unknown and spatially varying materials. The prior art uses time- multiplexed illumination but assumes constant surface normals across several frames, fundamentally limiting the accuracy of the estimated normals. We explicitly account for time-varying surface orientations, and show that for unknown Lambertian materials, five images are sufficient to recover surface orientation in one frame. Our optimized system imple- mentation exploits the physical properties of typical cameras and LEDs to reduce the required number of images to just three, and also facilitates frame-to-frame image alignment using standard optical flow methods, de- spite varying illumination. We demonstrate the system’s performance by computing surface orientations for several different moving, deforming ob jects.