Abstract
Structured pruning of filters or neurons has received increased focus for compressing convolutional neural networks. Most existing methods rely on multi-stage optimizations in a layer-wise manner for iteratively pruning and retraining which may not be optimal and may be computation
intensive. Besides, these methods are designed for pruning
a specific structure, such as filter or block structures without jointly pruning heterogeneous structures. In this paper,
we propose an effective structured pruning approach that
jointly prunes filters as well as other structures in an endto-end manner. To accomplish this, we first introduce a soft
mask to scale the output of these structures by defining a
new objective function with sparsity regularization to align
the output of baseline and network with this mask. We then
effectively solve the optimization problem by generative adversarial learning (GAL), which learns a sparse soft mask
in a label-free and an end-to-end manner. By forcing more
scaling factors in the soft mask to zero, the fast iterative
shrinkage-thresholding algorithm (FISTA) can be leveraged
to fast and reliably remove the corresponding structures.
Extensive experiments demonstrate the effectiveness of GAL
on different datasets, including MNIST, CIFAR-10 and ImageNet ILSVRC 2012. For example, on ImageNet ILSVRC
2012, the pruned ResNet-50 achieves 10.88% Top-5 error
and results in a factor of 3.7× speedup. This significantly
outperforms state-of-the-art methods.