Abstract

This paper tackles the problem of spatio-temporal ac-tion localization in a video, without assuming the availabil-ity of multiple videos or any prior annotations. Action islocalized by employing images downloaded from internetusing action name. Given web images, we first dampen im-age noise using random walk and evade distracting backgrounds within images using image action proposals. Then, given a video, we generate multiple spatio-temporal actionproposals. We suppress camera and background generatedproposals by exploiting optical flow gradients within pro-posals. To obtain the most action representative proposals,we propose to reconstruct action proposals in the video byleveraging the action proposals in images. Moreover, we preserve the temporal smoothness of the video and reconstruct all proposal bounding boxes jointly using the con-straints that push the coefficients for each bounding box to-ward a common consensus, thus enforcing the coefficient similarity across multiple frames. We solve this optimization problem using variant of two-metric projection algorithm. Finally, the video proposal that has the lowest reconstruction cost and is motion salient is used to localize the action. Our method is not only applicable to the trimmed videos, but it can also be used for action localization in untrimmed videos, which is a very challenging problem. We present extensive experiments on trimmed aswell as untrimmed datasets to validate the effectiveness ofthe proposed approach.