Abstract

This paper presents an approach to address the problem of image fa¸cade labelling. In the architectural literature, domain knowledge is usually expressed geometrically in the final design, so fa¸cade labelling should on the one hand conform to visual evidence, and on the other hand to the architectural principles – how individual assets (e.g. doors, windows) interact with each other to form a fa¸cade as a whole. To this end, we first propose a recursive splitting method to segment fa¸cades into a bunch of tiles for semantic recognition. The segmentation improves the processing speed, guides visual recognition on suitable scales and renders the extraction of architectural principles easy. Given a set of segmented training fa¸cades with their label maps, we then identify a set of meta-features to capture both the visual evidence and the architectural principles. The features are used to train our fa¸cade labelling model. In the test stage, the features are extracted from segmented fa¸cades and the inferred label maps. The following three steps are iterated until the optimal labelling is reached: 1) proposing modifications to the current labelling; 2) extracting new features for the proposed labelling; 3) feeding the new features to the labelling model to decide whether to accept the modifications. In experiments, we evaluated our method on the ECP fa¸cade dataset and achieved higher precision than the state-of-the-art at both the pixel level and the structural level.