Abstract

The goal of this paper is to enable a 3D “virtual-tour” of an apartment given a small set of monocular images of different rooms, as well as a 2D flfloor plan. We frame the problem as inference in a Markov Random Field which reasons about the layout of each room and its relative pose (3D rotation and translation) within the full apartment. This gives us accurate camera pose in the apartment for each image. What sets us apart from past work in layout estimation is the use of flfloor plans as a source of prior knowledge, as well as localization of each image within a bigger space (apartment). In particular, we exploit the flfloor plan to impose aspect ratio constraints across the layouts of different rooms, as well as to extract semantic information, e.g., the location of windows which are marked in flfloor plans. We show that this information can signifificantly help in resolving the challenging room-apartment alignment problem. We also derive an effificient exact inference algorithm which takes only a few ms per apartment. This is due to the fact that we exploit integral geometry as well as our new bounds on the aspect ratio of rooms which allow us to carve the space, signifificantly reducing the number of physically possible confifigurations. We demonstrate the effectiveness of our approach on a new dataset which contains over 200 apartments