Abstract Deep mutual learning jointly trains multiple essential networks having similar properties to improve semisupervised classifification. However, the commonly used consistency regularization between the outputs of the networks may not fully leverage the difference between them. In this paper, we explore how to capture the complementary information to enhance mutual learning. For this purpose, we propose a complementary correction network (CCN), built on top of the essential networks, to learn the mapping from the output of one essential network to the ground truth label, conditioned on the features learnt by another. To make the second essential network increasingly complementary to the fifirst one, this network is supervised by the corrected predictions. As a result, minimizing the prediction divergence between the two complementary networks can lead to signifificant performance gains in semisupervised learning. Our experimental results demonstrate that the proposed approach clearly improves mutual learning between essential networks, and achieves state-of-theart results on multiple semi-supervised classifification benchmarks. In particular, the test error rates are reduced from previous 21.23% and 14.65% to 12.05% and 10.37% on CIFAR-10 with 1000 and 2000 labels, respectively.