资源分类
》 资源 》论文》A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights

A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights

2020-01-19 | |  60 |   37 |   0

Abstract

We derive a second-order ordinary differential equation (ODE), which is the limit of Nesterov’s accelerated gradient method. This ODE exhibits approximate equivalence to Nesterov’s scheme and thus can serve as a tool for analysis. We show that the continuous time ODE allows for a better understanding of Nesterov’s scheme. As a byproduct, we obtain a family of schemes with similar convergence rates. The ODE interpretation also suggests restarting Nesterov’s scheme leading to an algorithm, which can be rigorously proven to converge at a linear rate whenever the objective is strongly convex.

上一篇：Structure Regularization for Structured Prediction Xu Sun

下一篇：Near–Optimal Density Estimation in Near–Linear Time Using Variable–Width Histograms

用户评价
全部评价

热门资源

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...

  • Rating-Boosted La...

    The performance of a recommendation system reli...