Abstract

Inverse graphics attempts to take sensor data and infer 3D geometry, illumination, materials, and motions such that a graphics ren- derer could realistically reproduce the observed scene. Renderers, how- ever, are designed to solve the forward process of image synthesis. To go in the other direction, we propose an approximate differentiable ren- derer (DR) that explicitly models the relationship between changes in model parameters and image observations. We describe a publicly avail- able OpenDR framework that makes it easy to express a forward graph- ics model and then automatically obtain derivatives with respect to the model parameters and to optimize over them. Built on a new auto- differentiation package and OpenGL, OpenDR provides a local optimiza- tion method that can be incorporated into probabilistic programming frameworks. We demonstrate the power and simplicity of programming with OpenDR by using it to solve the problem of estimating human body shape from Kinect depth and RGB data.