Abstract
Inner product-based convolution has been a central component of convolutional neural networks (CNNs) and the
key to learning visual representations. Inspired by the
observation that CNN-learned features are naturally decoupled with the norm of features corresponding to the
intra-class variation and the angle corresponding to the semantic difference, we propose a generic decoupled learning framework which models the intra-class variation and
semantic difference independently. Specifically, we first
reparametrize the inner product to a decoupled form and
then generalize it to the decoupled convolution operator
which serves as the building block of our decoupled networks. We present several effective instances of the decoupled convolution operator. Each decoupled operator is
well motivated and has an intuitive geometric interpretation. Based on these decoupled operators, we further propose to directly learn the operator from data. Extensive
experiments show that such decoupled reparameterization
renders significant performance gain with easier convergence and stronger robustness.