Abstract
Single-image piece-wise planar 3D reconstruction aims
to simultaneously segment plane instances and recover
3D plane parameters from an image. Most recent approaches leverage convolutional neural networks (CNNs)
and achieve promising results. However, these methods are
limited to detecting a fixed number of planes with certain
learned order. To tackle this problem, we propose a novel
two-stage method based on associative embedding, inspired
by its recent success in instance segmentation. In the first
stage, we train a CNN to map each pixel to an embedding
space where pixels from the same plane instance have similar embeddings. Then, the plane instances are obtained
by grouping the embedding vectors in planar regions via
an efficient mean shift clustering algorithm. In the second stage, we estimate the parameter for each plane instance by considering both pixel-level and instance-level
consistencies. With the proposed method, we are able to
detect an arbitrary number of planes. Extensive experiments on public datasets validate the effectiveness and effi-
ciency of our method. Furthermore, our method runs at 30
fps at the testing time, thus could facilitate many real-time
applications such as visual SLAM and human-robot interaction. Code is available at https://github.com/
svip-lab/PlanarReconstruction.