资源分类
》 资源 》算法》AdaBound-Tensorflow

AdaBound-Tensorflow

2020-01-06 | |  56 |   0 |   0

Hyperparameter

  • learning_rate = 0.01

  • final_lr = 0.1

  • beta1 = 0.9

  • beta2 = 0.999

Usage

  from AdaBound import AdaBoundOptimizer
  
  train_op = AdaBoundOptimizer(learning_rate=0.01, final_lr=0.1, beta1=0.9, beta2=0.999, amsbound=False).minimize(loss)

Network Architecture

  x = fully_connected(inputs=images, units=100)
  x = relu(x)
  logits = fully_connected(inputs=x, units=10)

Fashion-mnist Result

batch_size=32, lr=0.01, final_lr=0.1, beta1=0.9, beta2=0.99

   

OptimizerBest Test Acc
SGD86.33%
Adam85.81%
AMSGrad87.28%
AdaBound87.68%
AMSBound87.76%

batch_size=32, lr=0.01, final_lr=0.1, beta1=0.9, beta2=0.999

 

OptimizerBest Test Acc
SGD86.33%
Adam86.14%
AMSGrad86.63%
AdaBound86.88%
AMSBound87.25%

Related works

Author

Junho Kim


上一篇：keras-adabound

下一篇：advertorch_sample

用户评价
全部评价

热门资源

  • seetafaceJNI

    项目介绍 基于中科院seetaface2进行封装的JAVA...

  • spark-corenlp

    This package wraps Stanford CoreNLP annotators ...

  • Keras-ResNeXt

    Keras ResNeXt Implementation of ResNeXt models...

  • capsnet-with-caps...

    CapsNet with capsule-wise convolution Project ...

  • inferno-boilerplate

    This is a very basic boilerplate example for pe...