Abstract
In this paper, we address a rain removal problem from
a single image, even in the presence of heavy rain and rain
streak accumulation. Our core ideas lie in our new rain image model and new deep learning architecture. We add a
binary map that provides rain streak locations to an existing model, which comprises a rain streak layer and a background layer. We create a model consisting of a component representing rain streak accumulation (where individual
streaks cannot be seen, and thus visually similar to mist or
fog), and another component representing various shapes
and directions of overlapping rain streaks, which usually
happen in heavy rain. Based on the model, we develop a
multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the
clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss
function can provide additional strong information to the
network. To handle rain streak accumulation (again, a phenomenon visually similar to mist or fog) and various shapes
and directions of overlapping rain streaks, we propose a recurrent rain detection and removal network that removes
rain streaks and clears up the rain accumulation iteratively and progressively. In each recurrence of our method, a
new contextualized dilated network is developed to exploit
regional contextual information and to produce better representations for rain detection. The evaluation on real images, particularly on heavy rain, shows the effectiveness of
our models and architecture