Abstract

Space-time detection of human activities in videos can signi ficantly enhance visual search. To handle such tasks, while solely using low-level fea- tures has been found somewhat insufficient for complex datasets; mid-level fea- tures (like body parts) that are normally considered, are not robustly accounted for their inaccuracy. Moreover, the activity detection mechanisms do not con- structively utilize the importance and trustworthiness of the features. This paper addresses these problems and introduces a uni fied formulation for robustly detecting activities in videos. Our first contribution is the formulation of the detection task as an undirected node- and edge-weighted graphical struc- ture called Part Bricolage (PB), where the node weights represent the type of features along with their importance, and edge weights incorporate the probabil- ity of the features belonging to a known activity class, while also accounting for the trustworthiness of the features connecting the edge. Prize-Collecting-Steiner- Tree (PCST) problem [19] is solved for such a graph that gives the best connected subgraph comprising the activity of interest. Our second contribution is a novel technique for robust body part estimation, which uses two types of state-of-the-art pose detectors, and resolves the plausible detection ambiguities with pre-trained classi fiers that predict the trustworthiness of the pose detectors. Our third con- tribution is the proposal of fusing the low-level descriptors with the mid-level ones, while maintaining the spatial structure between the features. For a quantitative evaluation of the detection power of PB, we run PB on Hollywood and MSR-Actions datasets and outperform the state-of-the-art by a signi ficant margin for various detection paradigms.