Super-FAN: Integrated facial landmark localization and super-resolution of
real-world low resolution faces in arbitrary poses with GANs
Abstract
This paper addresses 2 challenging tasks: improving the
quality of low resolution facial images and accurately locating the facial landmarks on such poor resolution images. To
this end, we make the following 5 contributions: (a) we propose Super-FAN: the very first end-to-end system that addresses both tasks simultaneously, i.e. both improves face
resolution and detects the facial landmarks. The novelty
or Super-FAN lies in incorporating structural information
in a GAN-based super-resolution algorithm via integrating
a sub-network for face alignment through heatmap regression and optimizing a novel heatmap loss. (b) We illustrate
the benefit of training the two networks jointly by reporting good results not only on frontal images (as in prior
work) but on the whole spectrum of facial poses, and not
only on synthetic low resolution images (as in prior work)
but also on real-world images. (c) We improve upon the
state-of-the-art in face super-resolution by proposing a new
residual-based architecture. (d) Quantitatively, we show
large improvement over the state-of-the-art for both face
super-resolution and alignment. (e) Qualitatively, we show
for the first time good results on real-world low resolution
images like the ones of Fig