Abstract In this paper, we choose to learn useful cues from object recognition mechanisms of the human visual cortex, and propose a DCNN performance improvement method without the need for increasing the network complexity. Inspired by the categoryselective property of the neuron population in the IT layer of the human visual cortex, we enforce the neuron responses at the top DCNN layer to be category selective. To achieve this, we propose the Sparse Category-Selective Objective Function (SCSOF) to modulate the neuron outputs of the top DCNN layer. The proposed method is generic and can be applied to any DCNN models. As experimental results show, when applying the proposed method to the “Quick” model and NIN models, image classification performances are remarkably improved on four widely used benchmark datasets: CIFAR-10, CIFAR-100, MNIST and SVHN, which demonstrate the effectiveness of the presented method.