Abstract

We present an approach for compact video summaries that allows fast and direct access to video data. The video is segmented into shots and, in appropriate video genres, into scenes, using previously pro- posed methods. A new concept that supports the hierarchical represen- tation of video is presented, and is based on physical setting and camera locations. We use mosaics to represent and cluster shots, and detect ap- propriate mosaics to represent scenes. In contrast to approaches to video indexing which are based on key-frames, our eficient mosaic-based scene representation allows fast clustering of scenes into physical settings, as well as further comparison of physical settings across videos. This enables us to detect plots of difierent episodes in situation comedies and serves as a basis for indexing whole video sequences. In sports videos where settings are not as well defined, our approach allows classifying shots for characteristic event detection. We use a novel method for mosaic compar- ison and create a highly compact non-temporal representation of video. This representation allows accurate comparison of scenes across difierent videos and serves as a basis for indexing video libraries.