Abstract

In this paper, we focus on complex event detection in internet videos while also providing the key evidences of the detection results. Convolutional Neural Networks (CNNs) have achieved promising performance in image classifification and action recognition tasks. However, it remains an open problem how to use CNNs for video event detection and recounting, mainly due to the complexity and diversity of video events. In this work, we propose a flflexible deep CNN infrastructure, namely Deep Event Network (DevNet), that simultaneously detects pre-defifined events and provides key spatial-temporal evidences. Taking key frames of videos as input, we fifirst detect the event of interest at the video level by aggregating the CNN features of the key frames. The pieces of evidences which recount the detection results, are also automatically localized, both temporally and spatially. The challenge is that we only have video level labels, while the key evidences usually take place at the frame levels. Based on the intrinsic property of CNNs, we fifirst generate a spatial-temporal saliency map by back passing through DevNet, which then can be used to fifind the key frames which are most indicative to the event, as well as to localize the specifific spatial position, usually an object, in the frame of the highly indicative area. Experiments on the large scale TRECVID 2014 MEDTest dataset demonstrate the promising performance of our method, both for event detection and evidence recounting