Gradient-based Hyperparameter Optimization through Reversible Learning
| 
60 | 
41 | 
0
Abstract
Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients allow us to optimize thousands of hyperparameters, including step-size and momentum schedules, weight initialization distributions, richly rameterized regularization schemes, and neural network architectures. We compute hyperparameter gradients by exactly reversing the dynamics of stochastic gradient descent with momentum.
还没有评论，说两句吧！
热门资源
Learning to Predi...
Much of model-based reinforcement learning invo...
Stratified Strate...
In this paper we introduce Stratified Strategy ...
The Variational S...
Unlike traditional images which do not offer in...
A Mathematical Mo...
Direct democracy, where each voter casts one vo...
Rating-Boosted La...
The performance of a recommendation system reli...
智能在线
400-630-6780
聆听.建议反馈
E-mail: support@tusaishared.com
资源分类
