Coarse-to-Fine Image Inpainting via Region-wise Convolutions
and Non-Local Correlation
Abstract
Recently deep neural networks have achieved
promising performance for filling large missing
regions in image inpainting tasks. They usually adopted the standard convolutional architecture
over the corrupted image, where the same convolution filters try to restore the diverse information on
both existing and missing regions, and meanwhile
ignore the long-distance correlation among the regions. Only relying on the surrounding areas inevitably leads to meaningless contents and artifacts, such as color discrepancy and blur. To address
these problems, we first propose region-wise convolutions to locally deal with the different types of
regions, which can help exactly reconstruct existing
regions and roughly infer the missing ones from existing regions at the same time. Then, a non-local
operation is introduced to globally model the correlation among different regions, promising visual
consistency between missing and existing regions. Finally, we integrate the region-wise convolutions and non-local correlation in a coarse-to-fine
framework to restore semantically reasonable and
visually realistic images. Extensive experiments
on three widely-used datasets for image inpainting
tasks have been conducted, and both qualitative and
quantitative experimental results demonstrate that
the proposed model significantly outperforms the
state-of-the-art approaches, especially for the large
irregular missing regions