Abstract

For learning problems where human supervision is expens- ive, active query selection methods are often exploited to maximise the return of each supervision. Two problems where this has been success- fully applied are active discovery – where the aim is to discover at least one instance of each rare class with few supervisions; and active learn- ing – where the aim is to maximise a classifier’s performance with least supervision. Recently, there has been interest in optimising these tasks jointly, i.e., active learning with undiscovered classes, to support efficient interactive modelling of new domains. Mixtures of active discovery and learning and other schemes have been exploited, but perform poorly due to heuristic ob jectives. In this study, we show with systematic theoretical analysis how the previously disparate tasks of active discovery and learn- ing can be cleanly unified into a single problem, and hence are able for the first time to develop a unified query algorithm to directly optimise this problem. The result is a model which consistently outperforms pre- vious attempts at active learning in the presence of undiscovered classes, with no need to tune parameters for different datasets.