Abstract

The recognition of human actions and the determinationof human attributes are two tasks that call for fine-grainedclassification. Indeed, often rather small and inconspicuous objects and features have to be detected to tell their classesapart. In order to deal with this challenge, we proposea novel convolutional neural network that mines mid-levelimage patches that are sufficiently dedicated to resolve thecorresponding subtleties. In particular, we train a newly designed CNN (DeepPattern) that learns discriminative patchgroups. There are two innovative aspects to this. On the onehand we pay attention to contextual information in an origi-nal fashion. On the other hand, we let an iteration of feature learning and patch clustering purify the set of dedicated patches that we use. We validate our method for action clas-sification on two challenging datasets: PASCAL VOC 2012 Action and Stanford 40 Actions, and for attribute recogni-tion we use the Berkeley Attributes of People dataset. Our discriminative mid-level mining CNN obtains state-of-theart results on these datasets, without a need for annotations about parts and poses.