Abstract
Given a tiny face image, existing face hallucination
methods aim at super-resolving its high-resolution (HR)
counterpart by learning a mapping from an exemplar
dataset. Since a low-resolution (LR) input patch may correspond to many HR candidate patches, this ambiguity may
lead to distorted HR facial details and wrong attributes such
as gender reversal. An LR input contains low-frequency facial components of its HR version while its residual face image, defined as the difference between the HR ground-truth
and interpolated LR images, contains the missing highfrequency facial details. We demonstrate that supplementing residual images or feature maps with additional facial
attribute information can significantly reduce the ambiguity
in face super-resolution. To explore this idea, we develop an
attribute-embedded upsampling network, which consists of
an upsampling network and a discriminative network. The
upsampling network is composed of an autoencoder with
skip-connections, which incorporates facial attribute vectors into the residual features of LR inputs at the bottleneck of the autoencoder and deconvolutional layers used
for upsampling. The discriminative network is designed to
examine whether super-resolved faces contain the desired
attributes or not and then its loss is used for updating the
upsampling network. In this manner, we can super-resolve
tiny (16×16 pixels) unaligned face images with a large upscaling factor of 8× while reducing the uncertainty of oneto-many mappings remarkably. By conducting extensive
evaluations on a large-scale dataset, we demonstrate that
our method achieves superior face hallucination results and
outperforms the state-of-the-art.