Abstract. State-of-the-art face super-resolution methods leverage deep
convolutional neural networks to learn a mapping between low-resolution
(LR) facial patterns and their corresponding high-resolution (HR) counterparts by exploring local appearance information. However, most of
these methods do not account for facial structure and suffer from degradations due to large pose variations and misalignments. In this paper,
we propose a method that explicitly incorporates structural information
of faces into the face super-resolution process by using a multi-task convolutional neural network (CNN). Our CNN has two branches: one for
super-resolving face images and the other branch for predicting salient regions of a face coined facial component heatmaps. These heatmaps
encourage the upsampling stream to generate super-resolved faces with
higher-quality details. Our method not only uses low-level information
(i.e., intensity similarity), but also middle-level information (i.e., face
structure) to further explore spatial constraints of facial components
from LR inputs images. Therefore, we are able to super-resolve very
small unaligned face images (16×16 pixels) with a large upscaling factor
of 8×, while preserving face structure. Extensive experiments demonstrate that our network achieves superior face hallucination results and
outperforms the state-of-the-art