Abstract

An action is typically composed of different parts of the object moving in particular sequences. The presence of different motions (represented as a 1D histogram) has beenused in the traditional bag-of-words (BoW) approach forrecognizing actions. However the interactions among themotions also form a crucial part of an action. Different object-parts have varying degrees of interactions with the other parts during an action cycle. It is these interactions we want to quantify in order to bring in additional information about the actions. In this paper we propose a causalitybased approach for quantifying the interactions to aid ac-tion classification. Granger causality is used to computethe cause and effect relationships for pairs of motion trajectories of a video. A 2D histogram descriptor for thevideo is constructed using these pairwise measures. Ourproposed method of obtaining pairwise measures for videos is also applicable for large datasets. We have conducted ex-periments on challenging action recognition databases suchas HMDB51 and UCF50 and shown that our causality de-scriptor helps in encoding additional information regarding the actions and performs on par with the state-of-the art approaches. Due to the complementary nature, a further increase in performance can be observed by combining our approach with state-of-the-art approaches.