Abstract

In meta-learning an agent extracts knowledge from observed tasks, aiming to facilitate learning of novel future tasks. Under the assumption that future tasks are ‘related’ to previous tasks, accumulated knowledge should be learned in a way which captures the common structure across learned tasks, while allowing the learner sufficie flexibility to adapt to novel aspects of new tasks We present a framework for meta-learning that is based on generalization error bounds, allowing us to extend various PAC-Bayes bounds to metalearning. Learning takes place through the construction of a distribution over hypotheses based on the observed tasks, and its utilization for lea ing a new task. Thus, prior knowledge is incorporated through setting an experience-dependent prior for novel tasks. We develop a gradient-based algorithm which minimizes an objective function derived from the bounds and demonstrate its effectiveness numerically with deep neural net works. In addition to establishing the improved performance available through meta-learning, we demonstrate the intuitive way by which prior information is manifested at different levels of the network.