Abstract. In human-object interactions (HOI) recognition, conventional methods consider the human body as a whole and pay a uniform
attention to the entire body region. They ignore the fact that normally, human interacts with an object by using some parts of the body.
In this paper, we argue that different body parts should be paid with
different attention in HOI recognition, and the correlations between different body parts should be further considered. This is because our body
parts always work collaboratively. We propose a new pairwise body-part
attention model which can learn to focus on crucial parts, and their correlations for HOI recognition. A novel attention based feature selection
method and a feature representation scheme that can capture pairwise
correlations between body parts are introduced in the model. Our proposed approach achieved 10% relative improvement (36.1 mAP? 39.9
mAP) over the state-of-the-art results in HOI recognition on the HICO
dataset. We will make our model and source codes publicly available