Abstract

We present a physically interpretable, continuous three-dimensional (3D) model for handling occlusions with appli-cations to road scene understanding. We probabilisticallyassign each point in space to an object with a theoreticalmodeling of the reflection and transmission probabilities forthe corresponding camera ray. Our modeling is unified inhandling occlusions across a variety of scenarios, such asassociating structure from motion (SFM) point tracks withpotentially occluding objects or modeling object detectionscores in applications such as 3D localization. For pointtrack association, our model uniformly handles static anddynamic objects, which is an advantage over motion seg-mentation approaches traditionally used in multibody SFM.Detailed experiments on the KITTI raw dataset show thesuperiority of the proposed method over both state-of-the-artmotion segmentation and a baseline that heuristically uses detection bounding boxes for resolving occlusions. We also demonstrate how our continuous occlusion model may be applied to the task of 3D localization in road scenes.