Abstract
Facial expression recognition (FER) is a challenging
task due to different expressions under arbitrary poses.
Most conventional approaches either perform face frontalization on a non-frontal facial image or learn separate classifiers for each pose. Different from existing methods, in
this paper, we propose an end-to-end deep learning model
by exploiting different poses and expressions jointly for simultaneous facial image synthesis and pose-invariant facial
expression recognition. The proposed model is based on
generative adversarial network (GAN) and enjoys several
merits. First, the encoder-decoder structure of the generator can learn a generative and discriminative identity representation for face images. Second, the identity representation is explicitly disentangled from both expression and pose
variations through the expression and pose codes. Third,
our model can automatically generate face images with different expressions under arbitrary poses to enlarge and
enrich the training set for FER. Quantitative and qualitative evaluations on both controlled and in-the-wild datasets
demonstrate that the proposed algorithm performs favorably against state-of-the-art methods