Stacked Conditional Generative Adversarial Networks for Jointly Learning
Shadow Detection and Shadow Removal
Abstract
Understanding shadows from a single image consists of
two types of task in previous studies, containing shadow detection and shadow removal. In this paper, we present a
multi-task perspective, which is not embraced by any existing work, to jointly learn both detection and removal in an
end-to-end fashion that aims at enjoying the mutually improved benefits from each other. Our framework is based on
a novel STacked Conditional Generative Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each with a generator and a discriminator. Specifically,
a shadow image is fed into the first generator which produces a shadow detection mask. That shadow image, concatenated with its predicted mask, goes through the second
generator in order to recover its shadow-free image consequently. In addition, the two corresponding discriminators
are very likely to model higher level relationships and global scene characteristics for the detected shadow region and
reconstruction via removing shadows, respectively. More
importantly, for multi-task learning, our design of stacked
paradigm provides a novel view which is notably different
from the commonly used one as the multi-branch version.
To fully evaluate the performance of our proposed framework, we construct the first large-scale benchmark with
1870 image triplets (shadow image, shadow mask image,
and shadow-free image) under 135 scenes. Extensive experimental results consistently show the advantages of STCGAN over several representative state-of-the-art methods
on two large-scale publicly available datasets and our newly released one