Scribble-to-Painting Transformation with
Multi-Task Generative Adversarial Networks
Abstract
We propose the Dual Scribble-to-Painting Network
(DSP-Net), which is able to produce artistic paintings based on user-generated scribbles. In scribbleto-painting transformation, a neural net has to infer additional details of the image, given relatively
sparse information contained in the outlines of the
scribble. Therefore, it is more challenging than
classical image style transfer, in which the information content is reduced from photos to paintings.
Inspired by the human cognitive process, we propose a multi-task generative adversarial network,
which consists of two jointly trained neural nets –
one for generating artistic images and the other one
for semantic segmentation. We demonstrate that
joint training on these two tasks brings in additional
benefit. Experimental result shows that DSP-Net
outperforms state-of-the-art models both visually
and quantitatively. In addition, we publish a large
dataset for scribble-to-painting transformation