Abstract

Moving ob ject segmentation from an image sequence is es- sential for a robot to interact with its environment. Traditional vision approaches appeal to pure motion analysis on videos without exploit- ing the source of the background motion. We observe, however, that the background motion (from the robot’s egocentric view) has stronger correlation to the robot’s motor signals than the foreground motion. We propose a novel approach to detecting moving ob jects by clustering features into background and foreground according to their motion con- sistency with motor signals. Specifically, our approach learns homogra- phy and fundamental matrices as functions of motor signals, and predict sparse feature locations from the learned matrices. The errors between the predictions and their actual tracked locations are used to label them into background and foreground. The labels of the sparse features are then propagated to all pixels. Our approach does not require building a dense mosaic background or searching for affine, homography, or fun- damental matrix parameters for foreground separation. In addition, it does not need to explicitly model the intrinsic and extrinsic calibration parameters hence requires much less prior geometry knowledge. It works completely in 2D image space, and does not involve any complex analysis or computation in 3D space.