Abstract
Face recognition achieves exceptional success thanks to
the emergence of deep learning. However, many contemporary face recognition models still perform relatively poor in
processing profile faces compared to frontal faces. A key
reason is that the number of frontal and profile training
faces are highly imbalanced - there are extensively more
frontal training samples compared to profile ones. In addition, it is intrinsically hard to learn a deep representation that is geometrically invariant to large pose variations. In this study, we hypothesize that there is an inherent mapping between frontal and profile faces, and consequently, their discrepancy in the deep representation space
can be bridged by an equivariant mapping. To exploit this
mapping, we formulate a novel Deep Residual EquivAriant
Mapping (DREAM) block, which is capable of adaptively
adding residuals to the input deep representation to transform a profile face representation to a canonical pose that
simplifies recognition. The DREAM block consistently enhances the performance of profile face recognition for many
strong deep networks, including ResNet models, without deliberately augmenting training data of profile faces. The
block is easy to use, light-weight, and can be implemented
with a negligible computational overhead