Abstract

This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching. We focus on rare object classes, which play an important role in achieving richer semantic understanding of visual scenes, compared to common background classes. Towards this end, we make two novel contributions: rare class expansion and semantic context description. First, considering the long-tailed nature of the label distribution, we expand the retrieval set by rare class exemplars and thus achieve more balanced superpixel classifification results. Second, we incorporate both global and local semantic context information through a feedback based mechanism to refifine image retrieval and superpixel matching. Results on the SIFTflflow and LMSun datasets show the superior performance of our algorithm, especially on the rare classes, without sacrifificing overall labeling accuracy