Abstract
We show that gradient descent on full width linear convolutional networks of depth L converges to a linear predictor related to the bridge penalty in the frequency domain. This is in contrast to fully connected linear networks, where regardless of depth, gradient descent converges to the maximum margin solution.