Abstract
Recent deep learning based approaches have shown
promising results for the challenging task of inpainting
large missing regions in an image. These methods can
generate visually plausible image structures and textures,
but often create distorted structures or blurry textures inconsistent with surrounding areas. This is mainly due to
ineffectiveness of convolutional neural networks in explicitly borrowing or copying information from distant spatial locations. On the other hand, traditional texture and
patch synthesis approaches are particularly suitable when
it needs to borrow textures from the surrounding regions.
Motivated by these observations, we propose a new deep
generative model-based approach which can not only synthesize novel image structures but also explicitly utilize
surrounding image features as references during network
training to make better predictions. The model is a feedforward, fully convolutional neural network which can process images with multiple holes at arbitrary locations and
with variable sizes during the test time. Experiments on
multiple datasets including faces (CelebA, CelebA-HQ),
textures (DTD) and natural images (ImageNet, Places2)
demonstrate that our proposed approach generates higherquality inpainting results than existing ones. Code, demo
and models are available at: https://github.com/
JiahuiYu/generative_inpainting.