Abstract
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively
shrinks and expands a network, shrinking via a resourceweighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers.
In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per
inference), and capable of increasing the network’s performance. When applied to standard network architectures on
a wide variety of datasets, our approach discovers novel
structures in each domain, obtaining higher performance
while respecting the resource constraint