Abstract

Recognizing fifine-grained categories (e.g., bird species) is diffificult due to the challenges of discriminative region localization and fifine-grained feature learning. Existing approaches predominantly solve these challenges independently, while neglecting the fact that region detection and fifine-grained feature learning are mutually correlated and thus can reinforce each other. In this paper, we propose a novel recurrent attention convolutional neural network (RA-CNN) which recursively learns discriminative region attention and region-based feature representation at multiple scales in a mutually reinforced way. The learning at each scale consists of a classifification sub-network and an attention proposal sub-network (APN). The APN starts from full images, and iteratively generates region attention from coarse to fifine by taking previous predictions as a reference, while a fifiner scale network takes as input an amplifified attended region from previous scales in a recurrent way. The proposed RA-CNN is optimized by an intra-scale classifification loss and an inter-scale ranking loss, to mutually learn accurate region attention and fifine-grained representation. RA-CNN does not need bounding box/part annotations and can be trained end-to-end. We conduct comprehensive experiments and show that RA-CNN achieves the best performance in three fifine-grained tasks, with relative accuracy gains of 3.3%, 3.7%, 3.8%, on CUB Birds, Stanford Dogs and Stanford Cars, respectively.