Abstract. Scientific fields that are interested in faces have developed
their own sets of concepts and procedures for understanding how a target model system (be it a person or algorithm) perceives a face under
varying conditions. In computer vision, this has largely been in the form
of dataset evaluation for recognition tasks where summary statistics are
used to measure progress. While aggregate performance has continued
to improve, understanding individual causes of failure has been difficult,
as it is not always clear why a particular face fails to be recognized, or
why an impostor is recognized by an algorithm. Importantly, other fields
studying vision have addressed this via the use of visual psychophysics:
the controlled manipulation of stimuli and careful study of the responses
they evoke in a model system. In this paper, we suggest that visual
psychophysics is a viable methodology for making face recognition algorithms more explainable. A comprehensive set of procedures is developed
for assessing face recognition algorithm behavior, which is then deployed
over state-of-the-art convolutional neural networks and more basic, yet
still widely used, shallow and handcrafted feature-based approaches