Abstract. We contribute the first large-scale dataset of scene sketches,
SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created
through a novel and carefully designed crowdsourcing pipeline, enabling
users to efficiently generate large quantities of realistic and diverse scene
sketches. SketchyScene contains more than 29,000 scene-level sketches,
7,000+ pairs of scene templates and photos, and 11,000+ object sketches.
All objects in the scene sketches have ground-truth semantic and instance
masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate
the potential impact of SketchyScene by training new computational
models for semantic segmentation of scene sketches and showing how
the new dataset enables several applications including image retrieval,
sketch colorization, editing, and captioning, etc. The dataset and code
can be found at https://github.com/SketchyScene/SketchyScene