A Causal And-Or Graph Model for Visibility Fluent Reasoning in
Tracking Interacting Objects
Abstract
Tracking humans that are interacting with the other subjects or environment remains unsolved in visual tracking,
because the visibility of the human of interests in videos is
unknown and might vary over time. In particular, it is still
difficult for state-of-the-art human trackers to recover complete human trajectories in crowded scenes with frequent
human interactions. In this work, we consider the visibility status of a subject as a fluent variable, whose change is
mostly attributed to the subject’s interaction with the surrounding, e.g., crossing behind another object, entering a
building, or getting into a vehicle, etc. We introduce a
Causal And-Or Graph (C-AOG) to represent the causaleffect relations between an object’s visibility fluent and its
activities, and develop a probabilistic graph model to jointly
reason the visibility fluent change (e.g., from visible to invisible) and track humans in videos. We formulate this joint
task as an iterative search of a feasible causal graph structure that enables fast search algorithm, e.g., dynamic programming method. We apply the proposed method on challenging video sequences to evaluate its capabilities of estimating visibility fluent changes of subjects and tracking
subjects of interests over time. Results with comparisons
demonstrate that our method outperforms the alternative
trackers and can recover complete trajectories of humans
in complicated scenarios with frequent human interactions