Vehicle Detection with RetinaNet

Vehicle and pedestrian detection and tracking play a vital role in autonomous driving. In previous project, I implemented a vehicle detection and tracking pipeline based on traditional computer vision techniques. This project is to explore application of RetinaNet on the vehicle detection taask.

Dataset

The training and evaluation of this project is based on the Udacity annotated driving dataset. It includes driving in Mountain View California and neighboring cities during daylight conditions. I combined the two datasets and only retained bounding box annotations for car, truck, and pedestrian. The combined dataset

Here's an overview of the dataset

Model evalulation

In this project, I'm interested in the detection accuracy of the models as well as their inference speed. The goal is to find a model that can detect vehicles with good accuracy in real time.

The accuracy of models is primarily evaluated by mean Average Precision (mAP) and mean Average Recall (mAR) at IOU of 0.5.

The models being benchmarked are

1. sliding window method based on HOG feature and linear classifier

2. RetinaNet with ResNet50 backbone, pre-trained on COCO

3. RetinaNet with ResNet18 backbone, trained on driving dataset

4. RetinaNet with MobileNet backbone, trained on driving dataset

Main results

Benchmark

Example detection result

Vehicle tracking on movie

Here's the result of running RetinaNet-ResNet50-COCO on a dash camera video

Here's the result of running RetinaNet-MobileNet-0.25 on a dash camera video

Appendix

The following graph shows the structure of feature pyramid net (FPN) built on top of ResNet backbone.

The following graph showes the structure of regress and classification subnet.