Deep Adaptive Attention for Joint Facial Action
Unit Detection and Face Alignment
Abstract. Facial action unit (AU) detection and face alignment are two highly correlated tasks since facial landmarks can provide precise AU
locations to facilitate the extraction of meaningful local features for AU
detection. Most existing AU detection works often treat face alignment
as a preprocessing and handle the two tasks independently. In this paper, we propose a novel end-to-end deep learning framework for joint
AU detection and face alignment, which has not been explored before.
In particular, multi-scale shared features are learned firstly, and highlevel features of face alignment are fed into AU detection. Moreover, to
extract precise local features, we propose an adaptive attention learning module to refine the attention map of each AU adaptively. Finally,
the assembled local features are integrated with face alignment features
and global features for AU detection. Experiments on BP4D and DISFA
benchmarks demonstrate that our framework significantly outperforms
the state-of-the-art methods for AU detection