资源分类
》 资源 》论文》Essentially No Barriers in Neural Network Energy Landscape

Essentially No Barriers in Neural Network Energy Landscape

2020-03-16 | |  55 |   46 |   0

Abstract

Training neural networks involves finding minima of a high-dimensional non-convex loss function. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, the paths are essentially flat in both the training and test landscap This implies that minima are perhaps best seen as points on a single connected manifold of low loss, rather than as the bottoms of distinct valleys.

上一篇：Decoupling Gradient-Like Learning Rules from Representations

下一篇：Progress & Compress: A scalable framework for continual learning

用户评价
全部评价

热门资源

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to learn...

    The move from hand-designed features to learned...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...