Abstract

As part of an architectural modeling pro ject, this paper in- vestigates the problem of understanding and manipulating images of buildings. Our primary motivation is to automatically detect and seam- lessly remove unwanted foreground elements from urban scenes. With- out explicit handling, these ob jects will appear pasted as artifacts on the model. Recovering the building facade in a video sequence is relatively simple because parallax induces foreground/background depth layers, but here we consider static images only. We develop a series of methods that enable foreground removal from images of buildings or brick walls. The key insight is to use a priori knowledge about grid patterns on build- ing facades that can be modeled as Near Regular Textures (NRT). We describe a Markov Random Field (MRF) model for such textures and in- troduce a Markov Chain Monte Carlo (MCMC) optimization procedure for discovering them. This simple spatial rule is then used as a start- ing point for inference of missing windows, facade segmentation, outlier identification, and foreground removal.