Abstract. This paper studies the problem of blind face restoration from
an unconstrained blurry, noisy, low-resolution, or compressed image (i.e.,
degraded observation). For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a
high-quality guided image of the same identity as input to our guided
face restoration network (GFRNet). However, the degraded observation
and guided image generally are different in pose, illumination and expression, thereby making plain CNNs (e.g., U-Net) fail to recover fine
and identity-aware facial details. To tackle this issue, our GFRNet model includes both a warping subnetwork (WarpNet) and a reconstruction
subnetwork (RecNet). The WarpNet is introduced to predict flow field
for warping the guided image to correct pose and expression (i.e., warped
guidance), while the RecNet takes the degraded observation and warped
guidance as input to produce the restoration result. Due to that the
ground-truth flow field is unavailable, landmark loss together with total
variation regularization are incorporated to guide the learning of WarpNet. Furthermore, to make the model applicable to blind restoration,
our GFRNet is trained on the synthetic data with versatile settings on
blur kernel, noise level, downsampling scale factor, and JPEG quality
factor. Experiments show that our GFRNet not only performs favorably
against the state-of-the-art image and face restoration methods, but also
generates visually photo-realistic results on real degraded facial images