Abstract
Scene understanding requires reasoning about both what we can see and what is occluded. We offer a simple and general approach to infer labels of occluded background regions. Our approach incorpo- rates estimates of visible surrounding background, detected ob jects, and shape priors from transferred training regions. We demonstrate the abil- ity to infer the labels of occluded background regions in both the out- door StreetScenes dataset and an indoor scene dataset using the same approach. Our experiments show that our method outperforms competent baselines.