Abstract

Video object segmentation is challenging due to fastmoving objects, deforming shapes, and cluttered back-grounds. Optical flow can be used to propagate an ob-ject segmentation over time but, unfortunately, flow is ofteninaccurate, particularly around object boundaries. Suchboundaries are precisely where we want our segmentation to be accurate. To obtain accurate segmentation acrosstime, we propose an efficient algorithm that considers videosegmentation and optical flow estimation simultaneously.For video segmentation, we formulate a principled, multi-scale, spatio-temporal objective function that uses opticalflow to propagate information between frames. For opti-cal flow estimation, particularly at object boundaries, wecompute the flow independently in the segmented regions and recompose the results. We call the process object flow and demonstrate the effectiveness of jointly optimizing optical flow and video segmentation using an iterative scheme. Experiments on the SegTrack v2 and YoutubeObjects datasets show that the proposed algorithm performs favorably against the other state-of-the-art methods.