Abstract

Eliciting and representing experts’ remarkable perceptual capability of locating, identifying and categorizing objects in images specifific to their domains of expertise will benefifit image understanding in terms of transferring human domain knowledge and perceptual expertise into image-based computational procedures. In this paper, we present a hierarchical probabilistic framework to summarize the stereotypical and idiosyncratic eye movement patterns shared within 11 board-certifified dermatologists while they are examining and diagnosing medical images. Each inferred eye movement pattern characterizes the similar temporal and spatial properties of its corresponding segments of the experts’ eye movement sequences. We further discover a subset of distinctive eye movement patterns which are commonly exhibited across multiple images. Based on the combinations of the exhibitions of these eye movement patterns, we are able to categorize the images from the perspective of experts’ viewing strategies. In each category, images share similar lesion distributions and con- fifigurations. The performance of our approach shows that modeling physicians’ diagnostic viewing behaviors informs about medical images’ understanding to correct diagnosis