Abstract
Video object segmentation is a fundamental yet challenging task in computer vision community. In this paper,
we formulate this problem as a Markov Decision Process,
where agents are learned to segment object regions under a
deep reinforcement learning framework. Essentially, learning agents for segmentation is nontrivial as segmentation
is a nearly continuous decision-making process, where the
number of the involved agents (pixels or superpixels) and
action steps from the seed (super)pixels to the whole object mask might be incredibly huge. To overcome this difficulty, this paper simplifies the learning of segmentation
agents to the learning of a cutting-agent, which only has
a limited number of action units and can converge in just
a few action steps. The basic assumption is that object
segmentation mainly relies on the interaction between object regions and their context. Thus, with an optimal object (box) region and context (box) region, we can obtain
the desirable segmentation mask through further inference.
Based on this assumption, we establish a novel reinforcement cutting-agent learning framework, where the cuttingagent consists of a cutting-policy network and a cuttingexecution network. The former learns policies for deciding
optimal object-context box pair, while the latter executes
the cutting function based on the inferred object-context box
pair. With the collaborative interaction between the two networks, our method can achieve the outperforming VOS performance on two public benchmarks, which demonstrates
the rationality of our assumption as well as the effectiveness of the proposed learning framework