资源分类
》 资源 》论文》Stochastic Modified Equations and Adaptive Stochastic Gradient Algorithms

Stochastic Modified Equations and Adaptive Stochastic Gradient Algorithms

2020-03-10 | |  70 |   49 |   0

Abstract

We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations. We exploit the continuous formulation together with optimal control theory to derive novel adaptive hyper-parameter adjustment policies. Our algorithms have competitive performance with the added benefit of being robust to varying models and datasets. This provides a general methodology for the analysis and design of stochastic gradient algorithms.

上一篇：Axiomatic Attribution for Deep Networks

下一篇：Learning in POMDPs with Monte Carlo Tree Search

用户评价
全部评价

热门资源

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to learn...

    The move from hand-designed features to learned...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...