Abstract. Systems that perform image manipulation using deep convolutional networks have achieved remarkable realism. Perceptual losses
and losses based on adversarial discriminators are the two main classes
of learning objectives behind these advances. In this work, we show how
these two ideas can be combined in a principled and non-additive manner for unaligned image translation tasks. This is accomplished through a
special architecture of the discriminator network inside generative adversarial learning framework. The new architecture, that we call a perceptual
discriminator, embeds the convolutional parts of a pre-trained deep classification network inside the discriminator network. The resulting architecture can be trained on unaligned image datasets, while benefiting from
the robustness and efficiency of perceptual losses. We demonstrate the
merits of the new architecture in a series of qualitative and quantitative
comparisons with baseline approaches and state-of-the-art frameworks
for unaligned image translation