Abstract. In this paper, we propose a dual-agent deep reinforcement
learning (DADRL) method for deformable face tracking, which generates bounding boxes and detects facial landmarks interactively from face
videos. Most existing deformable face tracking methods learn models for
these two tasks individually, and perform these two procedures subsequently during the testing phase, which ignore the intrinsic connections
of these two tasks. Motivated by the fact that the performance of facial
landmark detection depends heavily on the accuracy of the generated
bounding boxes, we exploit the interactions of these two tasks in probabilistic manner by following a Bayesian model and propose a unified
framework for simultaneous bounding box tracking and landmark detection. By formulating it as a Markov decision process, we define two
agents to exploit the relationships and pass messages via an adaptive
sequence of actions under a deep reinforcement learning framework to
iteratively adjust the positions of the bounding boxes and facial landmarks. Our proposed DADRL achieves performance improvements over
the state-of-the-art deformable face tracking methods on the most challenging category of the 300-VW dataset