Abstract

We propose a notion of affordance that takes into ac-count physical quantities generated when the human bodyinteracts with real-world objects, and introduce a learningframework that incorporates the concept of human utilities,which in our opinion provides a deeper and finer-grainedaccount not only of object affordance but also of people’s interaction with objects. Rather than defining affordance in terms of the geometric compatibility between body poses and 3D objects, we devise algorithms that employ physicsbased simulation to infer the relevant forces/pressures act-ing on body parts. By observing the choices people make invideos (particularly in selecting a chair in which to sit) oursystem learns the comfort intervals of the forces exerted on body parts (while sitting). We account for people’s preferences in terms of human utilities, which transcend comfort intervals to account also for meaningful tasks within scenes and spatiotemporal constraints in motion planning, such as for the purposes of robot task planning.