Abstract

In this paper, we propose a recurrent framework for joint unsupervised learning of deep representations and image clusters. In our framework, successive operations in a clustering algorithm are expressed as steps in a recurrent process, stacked on top of representations output by a Convolutional Neural Network (CNN). During training, image clusters and representations are updated jointly: image clustering is conducted in the forward pass, while representation learning in the backward pass. Our key idea behind this framework is that good representations are benefificial to image clustering and clustering results provide supervisory signals to representation learning. By integrating two processes into a single model with a uni- fified weighted triplet loss function and optimizing it endto-end, we can obtain not only more powerful representations, but also more precise image clusters. Extensive experiments show that our method outperforms the stateof-the-art on image clustering across a variety of image datasets. Moreover, the learned representations generalize well when transferred to other tasks. The source code can be downloaded from https://github.com/ jwyang/joint-unsupervised-learning