Abstract

Geographical Information System (GIS) databases contain information about many ob jects, such as traffic signals, road signs, fire hydrants, etc. in urban areas. This wealth of information can be utilized for assisting various computer vision tasks. In this paper, we propose a method for improving ob ject detection using a set of priors acquired from GIS databases. Given a database of ob ject locations from GIS and a query image with metadata, we compute the expected spatial location of the visible ob jects in the image. We also perform ob ject detection in the query image (e.g., using DPM) and obtain a set of candidate bounding boxes for the ob jects. Then, we fuse the GIS priors with the potential detections to find the final ob ject bounding boxes. To cope with various inaccuracies and practical complications, such as noisy metadata, occlu- sion, inaccuracies in GIS, and poor candidate detections, we formulate our fusion as a higher-order graph matching problem which we robustly solve using RANSAC. We demonstrate that this approach outperforms well established ob ject detectors, such as DPM, with a large margin. Furthermore, we propose that the GIS ob jects can be used as cues for discovering the location where an image was taken. Our hypothesis is based on the idea that the ob jects visible in one image, along with their relative spatial location, provide distinctive cues for the geo-location. In order to estimate the geo-location based on the generic ob jects, we perform a search on a dense grid of locations over the covered area. We assign a score to each location based on the similarity of its GIS ob jects and the imperfect ob ject detections in the image. We demonstrate that over a broad urban area of >10 square kilometers, this semantic approach can significantly narrow down the localization search space, and occasionally, even find the correct location.