Abstract

Class hierarchies are commonly used to reduce the complex- ity of the classification problem. This is crucial when dealing with a large number of categories. In this work, we evaluate class hierarchies cur- rently constructed for visual recognition. We show that top-down as well as bottom-up approaches, which are commonly used to automatically construct hierarchies, incorporate assumptions about the separability of classes. Those assumptions do not hold for visual recognition of a large number of ob ject categories. We therefore propose a modification which is appropriate for most top-down approaches. It allows to construct class hierarchies that postpone decisions in the presence of uncertainty and thus provide higher recognition accuracy. We also compare our method to a one-against-all approach and show how to control the speed-for- accuracy trade-off with our method. For the experimental evaluation, we use the Caltech-256 visual ob ject classes dataset and compare to state- of-the-art methods.