Abstract

In this paper we aim for ob ject classification and segmenta- tion by attributes. Where existing work considers attributes either for the global image or for the parts of the ob ject, we propose, as our first novelty, to learn and extract attributes on segments containing the entire ob ject. Ob ject-level attributes suffer less from accidental content around the ob ject and accidental image conditions such as partial occlusions, scale changes and viewpoint changes. As our second novelty, we propose joint learning for simultaneous ob ject classification and segment proposal ranking, solely on the basis of attributes. This naturally brings us to our third novelty: ob ject-level attributes for zero-shot, where we use attribute descriptions of unseen classes for localizing their instances in new images and classifying them accordingly. Results on the Caltech UCSD Birds, Leeds Butterflies, and an a-Pascal subset demonstrate that i) extracting attributes on oracle ob ject-level brings substantial benefits ii) our joint learning model leads to accurate attribute-based classification and seg- mentation, approaching the oracle results and iii) ob ject-level attributes also allow for zero-shot classification and segmentation. We conclude that attributes make sense on segmented ob jects.