资源分类
》 资源 》论文》Path-SGD: Path-Normalized Optimization in Deep Neural Networks

Path-SGD: Path-Normalized Optimization in Deep Neural Networks

2020-02-05 | |  64 |   40 |   0

Abstract

 We revisit the choice of SGD for training deep neural networks by reconsidering the appropriate geometry in which to optimize the weights. We argue for a geometry invariant to rescaling of weights that does not affect the output of the network, and suggest Path-SGD, which is an approximate steepest descent method with respect to a path-wise regularizer related to max-norm regularization. Path-SGD is easy and efficient to implement and leads to empirical gains over SGD and AdaGrad.

上一篇：Tensorizing Neural Networks

下一篇：Adaptive Primal-Dual Splitting Methods for Statistical Learning and Image Processing

用户评价
全部评价

热门资源

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...

  • Rating-Boosted La...

    The performance of a recommendation system reli...