Disentangling Factors of Variation with
Cycle-Consistent Variational Auto-Encoders
Abstract. Generative models that learn disentangled representations
for different factors of variation in an image can be very useful for targeted data augmentation. By sampling from the disentangled latent subspace of interest, we can efficiently generate new data necessary for a
particular task. Learning disentangled representations is a challenging
problem, especially when certain factors of variation are difficult to label. In this paper, we introduce a novel architecture that disentangles the
latent space into two complementary subspaces by using only weak supervision in form of pairwise similarity labels. Inspired by the recent success
of cycle-consistent adversarial architectures, we use cycle-consistency in
a variational auto-encoder framework. Our non-adversarial approach is
in contrast with the recent works that combine adversarial training with
auto-encoders to disentangle representations. We show compelling results of disentangled latent subspaces on three datasets and compare
with recent works that leverage adversarial training.