Abstract
Despite the recent success of stereo matching with convolutional neural networks (CNNs), it remains arduous to
generalize a pre-trained deep stereo model to a novel domain. A major difficulty is to collect accurate groundtruth disparities for stereo pairs in the target domain. In
this work, we propose a self-adaptation approach for CNN
training, utilizing both synthetic training data (with groundtruth disparities) and stereo pairs in the new domain (without ground-truths). Our method is driven by two empirical
observations. By feeding real stereo pairs of different domains to stereo models pre-trained with synthetic data, we
see that: i) a pre-trained model does not generalize well to
the new domain, producing artifacts at boundaries and illposed regions; however, ii) feeding an up-sampled stereo
pair leads to a disparity map with extra details. To avoid
i) while exploiting ii), we formulate an iterative optimization problem with graph Laplacian regularization. At each
iteration, the CNN adapts itself better to the new domain:
we let the CNN learn its own higher-resolution output; at
the meanwhile, a graph Laplacian regularization is imposed
to discriminatively keep the desired edges while smoothing
out the artifacts. We demonstrate the effectiveness of our
method in two domains: daily scenes collected by smartphone cameras, and street views captured in a driving car