Abstract

Part-based visual tracking is advantageous due to its ro-bustness against partial occlusion. However, how to effec-tively exploit the confidence scores of individual parts toconstruct a robust tracker is still a challenging problem. In this paper, we address this problem by simultaneouslymatching parts in each of multiple frames, which is realizedby a locality-constrained low-rank sparse learning methodthat establishes multi-frame part correspondences throughoptimization of partial permutation matrices. The proposedpart matching tracker (PMT) has a number of attractiveproperties. (1) It exploits the spatial-temporal locality-constrained property for robust part matching. (2) It match-es local parts from multiple frames jointly by consideringtheir low-rank and sparse structure information, which caneffectively handle part appearance variations due to occlu-sion or noise. (3) The proposed PMT model has the inbuiltmechanism of leveraging multi-mode target templates, sothat the dilemma of template updating when encounteringocclusion in tracking can be better handled. This contrastswith existing methods that only do part matching between apair of frames. We evaluate PMT and compare with 10 popular state-of-the-art methods on challenging benchmarks. Experimental results show that PMT consistently outperform these existing trackers.