资源分类
》 资源 》论文》Information-theoretic analysis of generalization capability of learning algorithms

Information-theoretic analysis of generalization capability of learning algorithms

2020-02-10 | |  95 |   59 |   0

Abstract 

We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The bounds provide an information-theoretic understanding of generalization in learning problems, and give theoretical guidelines for striking the right balance between data fit and generalization by controlling the input-output mutual information. We propose a number of methods for this purpose, among which are algorithms that regularize the ERM algorithm with relative entropy or with random noise. Our work extends and leads to nontrivial improvements on the recent results of Russo and Zou.

上一篇：Generalized Linear Model Regression under Distance-to-set Penalties

下一篇：Thy Friend is My Friend: Iterative Collaborative Filtering for Sparse Matrix Estimation

用户评价
全部评价

热门资源

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...

  • Rating-Boosted La...

    The performance of a recommendation system reli...