Abstract. This paper proposes a novel deep neural architecture that automatically reconstructs a floorplan by walking through a house with a smartphone, an ultimate goal of indoor mapping research. The challenge lies in the processing of RGBD streams spanning a large 3D space. The proposed neural architecture, dubbed FloorNet, effectively processes the data through three neural network branches: 1) PointNet with 3D points, exploiting 3D information; 2) CNN with a 2D point density image in a top-down view, enhancing local spatial reasoning; and 3) CNN with RGB images, utilizing full image information. FloorNet exchanges intermediate features across the branches to exploit all the architectures. We have created a benchmark for floorplan reconstruction by acquiring RGBD video streams for 155 residential houses or apartments with Google Tango phones and annotating complete floorplan information. Our qualitative and quantitative evaluations demonstrate that the fusion of three branches effectively improves the reconstruction quality. We hope that the paper together with the benchmark will be an important step towards solving a challenging vector-graphics floorplan reconstruction problem