ESPNet: Efficient Spatial Pyramid of Dilated
Convolutions for Semantic Segmentation
Abstract. We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints. ESPNet is based on a new convolutional module, efficient spatial pyramid (ESP), which is efficient in terms of computation, memory, and power. ESPNet is 22 times faster (on a standard GPU) and 180 times smaller than the
state-of-the-art semantic segmentation network PSPNet, while its category-wise
accuracy is only 8% less. We evaluated ESPNet on a variety of semantic segmentation datasets including Cityscapes, PASCAL VOC, and a breast biopsy whole
slide image dataset. Under the same constraints on memory and computation,
ESPNet outperforms all the current efficient CNN networks such as MobileNet,
ShuffleNet, and ENet on both standard metrics and our newly introduced performance metrics that measure efficiency on edge devices. Our network can process
high resolution images at a rate of 112 and 9 frames per second on a standard
GPU and edge device, respectively. Our code is open-source and available at
https://sacmehta.github.io/ESPNet/