Abstract. Classical computation of optical flow involves generic priors (regularizers) that capture rudimentary statistics of images, but not
long-range correlations or semantics. On the other hand, fully supervised
methods learn the regularity in the annotated data, without explicit regularization and with the risk of overfitting. We seek to learn richer priors
on the set of possible flows that are statistically compatible with an
image. Once the prior is learned in a supervised fashion, one can easily learn the full map to infer optical flow directly from two or more
images, without any need for (additional) supervision. We introduce a
novel architecture, called Conditional Prior Network (CPN), and show
how to train it to yield a conditional prior. When used in conjunction
with a simple optical flow architecture, the CPN beats all variational
methods and all unsupervised learning-based ones using the same data
term. It performs comparably to fully supervised ones, that however are
fine-tuned to a particular dataset. Our method, on the other hand, performs well even when transferred between datasets. Code is available at:
https://github.com/YanchaoYang/Conditional-Prior-Networks