Abstract

Foreground object segmentation is a critical step formany image analysis tasks. While automated methods canproduce high-quality results, their failures disappoint usersin need of practical solutions. We propose a resource al-location framework for predicting how best to allocate afixed budget of human annotation effort in order to collecthigher quality segmentations for a given batch of imagesand automated methods. The framework is based on a pro-posed prediction module that estimates the quality of given algorithm-drawn segmentations. We demonstrate the value of the framework for two novel tasks related to “pulling the plug” on computer and human annotators. Specifically, we implement two systems that automatically decide, for a batch of images, when to replace 1) humans with computers to create coarse segmentations required to initialize segmentation tools and 2) computers with humans to createfinal, fine-grained segmentations. Experiments demonstratethe advantage of relying on a mix of human and computer efforts over relying on either resource alone for segmenting objects in three diverse datasets representing visible, phase contrast microscopy, and fluorescence microscopy images.