Bilateral Ordinal Relevance Multi-instance Regression
for Facial Action Unit Intensity Estimation
Abstract
Automatic intensity estimation of facial action units
(AUs) is challenging in two aspects. First, capturing subtle changes of facial appearance is quite difficult. Second,
the annotation of AU intensity is scarce and expensive. Intensity annotation requires strong domain knowledge thus
only experts are qualified. The majority of methods directly
apply supervised learning techniques to AU intensity estimation while few methods exploit unlabeled samples to improve the performance. In this paper, we propose a novel
weakly supervised regression model-Bilateral Ordinal Relevance Multi-instance Regression (BORMIR), which learns
a frame-level intensity estimator with weakly labeled sequences. From a new perspective, we introduce relevance to
model sequential data and consider two bag labels for each
bag. The AU intensity estimation is formulated as a joint regressor and relevance learning problem. Temporal dynamics of both relevance and AU intensity are leveraged to build
connections among labeled and unlabeled image frames to
provide weak supervision. We also develop an efficient algorithm for optimization based on the alternating minimization framework. Evaluations on three expression databases
demonstrate the effectiveness of the proposed method