Abstract

We propose an online background subtraction algorithm with superpixel-based density estimation for videos captured by moving cam- era. Our algorithm maintains appearance and motion models of fore- ground and background for each superpixel, computes foreground and background likelihoods for each pixel based on the models, and deter- mines pixelwise labels using binary belief propagation. The estimated la- bels trigger the update of appearance and motion models, and the above steps are performed iteratively in each frame. After convergence, ap- pearance models are propagated through a sequential Bayesian filtering, where predictions rely on motion fields of both labels whose computation exploits the segmentation mask. Superpixel-based modeling and label in- tegrated motion estimation make propagated appearance models more accurate compared to existing methods since the models are constructed on visually coherent regions and the quality of estimated motion is im- proved by avoiding motion smoothing across regions with different labels. We evaluate our algorithm with challenging video sequences and present significant performance improvement over the state-of-the-art techniques quantitatively and qualitatively.