Abstract

Deep convolutional neural networks (CNN) have seen tremendous success in large-scale generic object recognition. In comparison with generic object recognition, fifinegrained image classifification (FGIC) is much more challenging because (i) fifine-grained labeled data is much more expensive to acquire (usually requiring domain expertise); (ii) there exists large intra-class and small inter-class variance. Most recent work exploiting deep CNN for image recognition with small training data adopts a simple strategy: pre-train a deep CNN on a large-scale external dataset (e.g., ImageNet) and fifine-tune on the small-scale target data to fifit the specifific classifification task. In this paper, beyond the fifine-tuning strategy, we propose a systematic framework of learning a deep CNN that addresses the challenges from two new perspectives: (i) identifying easily annotated hyper-classes inherent in the fifine-grained data and acquiring a large number of hyper-class-labeled images from readily available external sources (e.g., image search engines), and formulating the problem into multitask learning; (ii) a novel learning model by exploiting a regularization between the fifine-grained recognition model and the hyper-class recognition model. We demonstrate the success of the proposed framework on two small-scale fifinegrained datasets (Stanford Dogs and Stanford Cars) and on a large-scale car dataset that we collected.