Abstract

Hidden State Shape Models (HSSMs) were previously pro- posed to represent and detect ob jects in images that exhibit not just deformation of their shape but also variation in their structure. In this paper, we introduce Dynamic Hidden-State Shape Models (DHSSMs) to track and recognize the non-rigid motion of such ob jects, for exam- ple, human hands. Our recursive Bayesian filtering method, called DP- Tracking, combines an exhaustive local search for a match between image features and model states with a dynamic programming approach to find a global registration between the model and the ob ject in the image. Our contribution is a technique to exploit the hierarchical struc- ture of the dynamic programming approach that on average considerably speeds up the search for matches. We also propose to embed an online learning approach into the tracking mechanism that updates the DHSSM dynamically. The learning approach ensures that the DHSSM accurately represents the tracked ob ject and distinguishes any clutter potentially present in the image. Our experiments show that our method can recog- nize the digits of a hand while the fingers are being moved and curled to various degrees. The method is robust to various illumination conditions, the presence of clutter, occlusions, and some types of self-occlusions. The experiments demonstrate a significant improvement in both effi- ciency and accuracy of recognition compared to the non-recursive way of frame-by-frame detection.