Abstract

Models defined using higher-order potentials are becoming increasingly popular in computer vision. However, the exact representa- tion of a general higher-order potential defined over many variables is computationally unfeasible. This has led prior works to adopt paramet- ric potentials that can be compactly represented. This paper proposes a non-parametric higher-order model for image labeling problems that uses a patch-based representation of its potentials. We use the transformation scheme of [11, 25] to convert the higher-order potentials to a pair-wise form that can be handled using traditional inference algorithms. This representation is able to capture structure, geometrical and topological information of labels from training data and to provide more precise seg- mentations. Other tasks such as image denoising and reconstruction are also possible. We evaluate our method on denoising and segmentation problems with synthetic and real images.