资源分类
》 资源 》论文》The Physical Systems Behind Optimization Algorithms

The Physical Systems Behind Optimization Algorithms

2020-02-13 | |  59 |   49 |   0

Abstract 

We use differential equations based approaches to provide some physics insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient descent, coordinate gradient descent, proximal coordinate gradient, and Newton’s methods as well as their Nesterov’s accelerated variants in a unified framework motivated by a natural connection of optimization algorithms to physical systems. Our analysis is applicable to more general algorithms and optimization problems beyond convexity and strong convexity, e.g. Polyak-?ojasiewicz and error bound conditions (possibly nonconvex).

上一篇：Learning To Learn Around A Common Mean

下一篇：Connectionist Temporal Classification with Maximum Entropy Regularization

用户评价
全部评价

热门资源

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • Learning to learn...

    The move from hand-designed features to learned...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...