Abstract

In this work, we focus on the problem of tracking ob jects un- der significant viewpoint variations, which poses a big challenge to tradi- tional ob ject tracking methods. We propose a novel method to track an ob ject and estimate its continuous pose and part locations under severe viewpoint change. In order to handle the change in topological appear- ance introduced by viewpoint transformations, we represent ob jects with 3D aspect parts and model the relationship between viewpoint and 3D aspect parts in a part-based particle filtering framework. Moreover, we show that instance-level online-learned part appearance can be incorpo- rated into our model, which makes it more robust in difficult scenarios with occlusions. Experiments are conducted on a new dataset of chal- lenging YouTube videos and a subset of the KITTI dataset [14] that include significant viewpoint variations, as well as a standard sequence for car tracking. We demonstrate that our method is able to track the 3D aspect parts and the viewpoint of ob jects accurately despite significant changes in viewpoint.