Abstract

In this paper we introduce the concept of local label de- scriptor, which is a concatenation of label histograms for each cell in a patch. Local label descriptors alleviate the label patch misalignment issue in combining structured label predictions for semantic image label- ing. Given an input image, we solve for a label map whose local label descriptors can be approximated as a sparse convex combination of ex- emplar label descriptors in the training data, where the sparsity is regu- larized by the similarity measure between the local feature descriptor of the input image and that of the exemplars in the training data set. Low- level image over-segmentation can be incorporated into our formulation to improve efficiency. Our formulation and algorithm compare favorably with the baseline method on the CamVid and Barcelona datasets.