Abstract. This paper addresses the semantic instance segmentation
task in the open-set conditions, where input images can contain known
and unknown object classes. The training process of existing semantic
instance segmentation methods requires annotation masks for all object
instances, which is expensive to acquire or even infeasible in some realistic
scenarios, where the number of categories may increase boundlessly. In
this paper, we present a novel open-set semantic instance segmentation
approach capable of segmenting all known and unknown object classes
in images, based on the output of an object detector trained on known
object classes. We formulate the problem using a Bayesian framework,
where the posterior distribution is approximated with a simulated annealing optimization equipped with an efficient image partition sampler. We
show empirically that our method is competitive with state-of-the-art supervised methods on known classes, but also performs well on unknown
classes when compared with unsupervised methods.