Boundary Flow: A Siamese Network that Predicts Boundary Motion without
Training on Motion
Abstract
Using deep learning, this paper addresses the problem
of joint object boundary detection and boundary motion
estimation in videos, which we named boundary flow estimation. Boundary flow is an important mid-level visual cue
as boundaries characterize objects’ spatial extents, and the
flow indicates objects’ motions and interactions. Yet, most
prior work on motion estimation has focused on dense object motion or feature points that may not necessarily reside
on boundaries. For boundary flow estimation, we specify a
new fully convolutional Siamese network (FCSN) that jointly
estimates object-level boundaries in two consecutive frames.
Boundary correspondences in the two frames are predicted
by the same FCSN with a new, unconventional deconvolution
approach. Finally, the boundary flow estimate is improved
with an edgelet-based filtering. Evaluation is conducted on
three tasks: boundary detection in videos, boundary flow
estimation, and optical flow estimation. On boundary detection, we achieve the state-of-the-art performance on the
benchmark VSB100 dataset. On boundary flow estimation,
we present the first results on the Sintel training dataset. For
optical flow estimation, we run the recent approach CPMFlow but on the augmented input with our boundary-flow
matches, and achieve significant performance improvement
on the Sintel benchmark