Abstract

Scene labeling task is to segment the image into mean-ingful regions and categorize them into classes of objectswhich comprised the image. Commonly used methods typi-cally find the local features for each segment and label themusing classifiers. Afterwards, labeling is smoothed in orderto make sure that neighboring regions receive similar la-bels. However, these methods ignore expressive connections between labels and non-local dependencies among regions. In this paper, we propose to use a sparse estimation of pre-cision matrix (also called concentration matrix), which isthe inverse of covariance matrix of data obtained by graph-ical lasso to find interaction between labels and regions. Todo this, we formulate the problem as an energy minimiza-tion over a graph, whose structure is captured by applying sparse constraint on the elements of the precision matrix. This graph encodes (or represents) only significant interac-tions and avoids a fully connected graph, which is typically used to reflect the long distance associations. We use local and global information to achieve better labeling. We assess our approach on three datasets and obtained promising results.