Abstract

Spectral embedding provides a framework for solving perceptual organization problems, including image segmentation and fifigure/ground organization. From an affifinity matrix describing pairwise relationships between pixels, it clusters pixels into regions, and, using a complex-valued extension, orders pixels according to layer. We train a convolutional neural network (CNN) to directly predict the pairwise relationships that defifine this affifinity matrix. Spectral embedding then resolves these predictions into a globallyconsistent segmentation and fifigure/ground organization of the scene. Experiments demonstrate signifificant benefifit to this direct coupling compared to prior works which use explicit intermediate stages, such as edge detection, on the pathway from image to affifinities. Our results suggest spectral embedding as a powerful alternative to the conditional random fifield (CRF)-based globalization schemes typically coupled to deep neural networks.