Abstract
Compositionality of semantic concepts in image
synthesis and analysis is appealing as it can help in decomposing known and generatively recomposing unknown
data. For instance, we may learn concepts of changing
illumination, geometry or albedo of a scene, and try to
recombine them to generate physically meaningful, but
unseen data for training and testing. In practice however
we often do not have samples from the joint concept space
available: We may have data on illumination change in one
data set and on geometric change in another one without
complete overlap. We pose the following question: How
can we learn two or more concepts jointly from different
data sets with mutual consistency where we do not have
samples from the full joint space? We present a novel
answer in this paper based on cyclic consistency over
multiple concepts, represented individually by generative
adversarial networks (GANs). Our method, ConceptGAN,
can be understood as a drop in for data augmentation to
improve resilience for real world applications. Qualitative
and quantitative evaluations demonstrate its efficacy in
generating semantically meaningful images, as well as one
shot face verification as an example application