Abstract

There is a huge diversity of definitions of “visually meaning- ful” image segments, ranging from simple uniformly colored segments, textured segments, through symmetric patterns, and up to complex se- mantically meaningful ob jects. This diversity has led to a wide range of different approaches for image segmentation. In this paper we present a single unified framework for addressing this problem – “Segmentation by Composition”. We define a good image segment as one which can be easily composed using its own pieces, but is difficult to compose using pieces from other parts of the image. This non-parametric approach cap- tures a large diversity of segment types, yet requires no pre-definition or modelling of segment types, nor prior training. Based on this definition, we develop a segment extraction algorithm – i.e., given a single point-of- interest, provide the “best” image segment containing that point. This induces a figure-ground image segmentation, which applies to a range of different segmentation tasks: single image segmentation, simultaneous co-segmentation of several images, and class-based segmentations.