ClusterNet: Detecting Small Objects in Large Scenes
by Exploiting Spatio-Temporal Information
Abstract
Object detection in wide area motion imagery (WAMI)
has drawn the attention of the computer vision research
community for a number of years. WAMI proposes a number of unique challenges including extremely small object sizes, both sparse and densely-packed objects, and extremely large search spaces (large video frames). Nearly
all state-of-the-art methods in WAMI object detection report that appearance-based classifiers fail in this challenging data and instead rely almost entirely on motion information in the form of background subtraction or framedifferencing. In this work, we experimentally verify the
failure of appearance-based classifiers in WAMI, such as
Faster R-CNN and a heatmap-based fully convolutional
neural network (CNN), and propose a novel two-stage
spatio-temporal CNN which effectively and efficiently combines both appearance and motion information to signifi-
cantly surpass the state-of-the-art in WAMI object detection. To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the
convolutional architecture, and proposes regions of objects
of interest (ROOBI). These ROOBI can contain from one to
clusters of several hundred objects due to the large video
frame size and varying object density in WAMI. The second
stage (FoveaNet) then estimates the centroid location of all
objects in that given ROOBI simultaneously via heatmap
estimation. The proposed method exceeds state-of-the-art
results on the WPAFB 2009 dataset by 5-16% for moving
objects and nearly 50% for stopped objects, as well as being the first proposed method in wide area motion imagery
to detect completely stationary objects