DeepPhys: Video-Based Physiological
Measurement Using Convolutional
Attention Networks
Abstract. Non-contact video-based physiological measurement has many
applications in health care and human-computer interaction. Practical
applications require measurements to be accurate even in the presence of
large head rotations. We propose the first end-to-end system for videobased measurement of heart and breathing rate using a deep convolutional network. The system features a new motion representation based
on a skin reflection model and a new attention mechanism using appearance information to guide motion estimation, both of which enable robust
measurement under heterogeneous lighting and major motions. Our approach significantly outperforms all current state-of-the-art methods on
both RGB and infrared video datasets. Furthermore, it allows spatialtemporal distributions of physiological signals to be visualized via the
attention mechanism