Abstract
Several recent works have used deep convolutional networks to generate realistic imagery. These methods sidestep
the traditional computer graphics rendering pipeline and
instead generate imagery at the pixel level by learning from
large collections of photos (e.g. faces or bedrooms). However, these methods are of limited utility because it is dif-
ficult for a user to control what the network produces. In
this paper, we propose a deep adversarial image synthesis
architecture that is conditioned on sketched boundaries and
sparse color strokes to generate realistic cars, bedrooms,
or faces. We demonstrate a sketch based image synthesis
system which allows users to scribble over the sketch to indicate preferred color for objects. Our network can then
generate convincing images that satisfy both the color and
the sketch constraints of user. The network is feed-forward
which allows users to see the effect of their edits in real
time. We compare to recent work on sketch to image synthesis and show that our approach generates more realistic,
diverse, and controllable outputs. The architecture is also
effective at user-guided colorization of grayscale images.