Abstract

As ob jects are inherently 3-dimensional, they have been mod- eled in 3D in the early days of computer vision. Due to the ambiguities arising from mapping 2D features to 3D models, 2D feature-based models are the predominant paradigm in ob ject recognition today. While such models have shown competitive bounding box (BB) detection perfor- mance, they are clearly limited in their capability of fine-grained reason- ing in 3D or continuous viewpoint estimation as required for advanced tasks such as 3D scene understanding. This work extends the deformable part model [1] to a 3D ob ject model. It consists of multiple parts mod- eled in 3D and a continuous appearance model. As a result, the model generalizes beyond BB oriented ob ject detection and can be jointly op- timized in a discriminative fashion for ob ject detection and viewpoint estimation. Our 3D Deformable Part Model (3D2PM) leverages on CAD data of the ob ject class, as a 3D geometry proxy.