Abstract
Objects often occlude each other in scenes; Inferring
their appearance beyond their visible parts plays an important role in scene understanding, depth estimation, object
interaction and manipulation. In this paper, we study the
challenging problem of completing the appearance of occluded objects. Doing so requires knowing which pixels to
paint (segmenting the invisible parts of objects) and what
color to paint them (generating the invisible parts). Our
proposed novel solution, SeGAN, jointly optimizes for both
segmentation and generation of the invisible parts of objects. Our experimental results show that: (a) SeGAN can
learn to generate the appearance of the occluded parts of
objects; (b) SeGAN outperforms state-of-the-art segmentation baselines for the invisible parts of objects; (c) trained
on synthetic photo realistic images, SeGAN can reliably
segment natural images; (d) by reasoning about occluderoccludee relations, our method can infer depth layering