Abstract

Early prediction of ongoing activity has been more and more valuable in a large variety of time-critical applications. To build an ef- fective representation for prediction, human activities can be character- ized by a complex temporal composition of constituent simple actions. Different from early recognition on short-duration simple activities, we propose a novel framework for long -duration complex activity prediction by discovering the causal relationships between constituent actions and the predictable characteristics of activities. The ma jor contributions of our work include: (1) we propose a novel activity decomposition method by monitoring motion velocity which encodes a temporal decomposition of long activities into a sequence of meaningful action units; (2) Proba- bilistic Suffix Tree (PST) is introduced to represent both large and small order Markov dependencies between action units; (3) we present a Pre- dictive Accumulative Function (PAF) to depict the predictability of each kind of activity. The effectiveness of the proposed method is evaluated on two experimental scenarios: activities with middle-level complexity and activities with high-level complexity. Our method achieves promis- ing results and can predict global activity classes and local action units.