Abstract

In recent years, sensors capable of measuring both colorand depth information have become increasingly popular.Despite the abundance of colored point set data, stateof-the-art probabilistic registration techniques ignore theavailable color information. In this paper, we propose aprobabilistic point set registration framework that exploits available color information associated with the points. Ourmethod is based on a model of the joint distribution of3D-point observations and their color information. Theproposed model captures discriminative color information,while being computationally efficient. We derive an EM al-gorithm for jointly estimating the model parameters and therelative transformations. Comprehensive experiments are performed on the Stanford Lounge dataset, captured by an RGB-D camera, and two point sets captured by a Lidar sensor. Our results demonstrate a significant gain in robustness and accuracy when incorporating color information. On the Stanford Lounge dataset, our approach achieves a relative reductionof the failure rate by 78% compared to the baseline. Fur-thermore, our proposed model outperforms standard strategies for combining color and 3D-point information, leading to state-of-the-art results.