Abstract

Large-scale image retrieval benchmarks invariably consist of images from the Web. Many of these benchmarks are derived from on- line photo sharing networks, like Flickr, which in addition to hosting images also provide a highly interactive social community. Such com- munities generate rich metadata that can naturally be harnessed for image classification and retrieval. Here we study four popular bench- mark datasets, extending them with social-network metadata, such as the groups to which each image belongs, the comment thread associated with the image, who uploaded it, their location, and their network of friends. Since these types of data are inherently relational, we propose a model that explicitly accounts for the interdependencies between im- ages sharing common properties. We model the task as a binary labeling problem on a network, and use structured learning techniques to learn model parameters. We find that social-network metadata are useful in a variety of classification tasks, in many cases outperforming methods based on image content.