Heavy Rain Image Restoration: Integrating Physics Model and ConditionalAdversarial Learning?
Abstract
Most deraining works focus on rain streaks removal but
they cannot deal adequately with heavy rain images. In
heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the
image, further scenes are relatively more blurry, etc. In this
paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first
stage estimates the rain streaks, the transmission, and the
atmospheric light governed by the underlying physics. To
tease out these components more reliably, a guided filtering framework is used to decompose the image into its lowand high-frequency components. This filtering is guided by
a rain-free residue image — its content is used to set the
passbands for the two channels in a spatially-variant manner so that the background details do not get mixed up with
the rain-streaks. For the second stage, the refinement stage,
we put forth a depth-guided GAN to recover the background
details failed to be retrieved by the first stage, as well as correcting artefacts introduced by that stage. We have evaluated our method against the state of the art methods. Extensive experiments show that our method outperforms them on
real rain image data, recovering visually clean images with
good details.