Abstract. Disparity estimation for binocular stereo images finds a wide
range of applications. Traditional algorithms may fail on featureless regions, which could be handled by high-level clues such as semantic segments. In this paper, we suggest that appropriate incorporation of semantic cues can greatly rectify prediction in commonly-used disparity
estimation frameworks. Our method conducts semantic feature embedding and regularizes semantic cues as the loss term to improve learning
disparity. Our unified model SegStereo employs semantic features from
segmentation and introduces semantic softmax loss, which helps improve
the prediction accuracy of disparity maps. The semantic cues work well
in both unsupervised and supervised manners. SegStereo achieves stateof-the-art results on KITTI Stereo benchmark and produces decent prediction on both CityScapes and FlyingThings3D datasets