Abstract

Beyond recognizing the actions of individuals, activity group localization aims to determine “who participates in each group” and “what activity the group performs”. In this paper, we propose a latent graphical model to group participants while inferring each group’s activ- ity by exploring the relations among them, thus simultaneously address- ing the problems of group localization and activity recognition. Our key insight is to exploit the relational graph among the participants. Specif- ically, each group is represented as a tree with an activity label while relations among groups are modeled as a fully connected graph. Infer- ence of such a graph is reduced into an extended minimum spanning forest problem, which is casted into a max-margin framework. It there- fore avoids the limitation of high-ordered hierarchical model and can be solved efficiently. Our model is able to provide strong and discriminative contextual cues for activity recognition and to better interpret scene in- formation for localization. Experiments on three datasets demonstrate that our model achieves significant improvements in activity group. lo- calization and state-of-the-arts performance on activity recognition.