Learning Shared Knowledge for Deep Lifelong Learning
Using Deconvolutional Networks
Abstract
Current mechanisms for knowledge transfer in deep
networks tend to either share the lower layers between tasks, or build upon representations trained
on other tasks. However, existing work in non-deep
multi-task and lifelong learning has shown success
with using factorized representations of the model
parameter space for transfer, permitting more flexible construction of task models. Inspired by this
idea, we introduce a novel architecture for sharing
latent factorized representations in convolutional
neural networks (CNNs). The proposed approach,
called a deconvolutional factorized CNN, uses a
combination of deconvolutional factorization and
tensor contraction to perform flexible transfer between tasks. Experiments on two computer vision
data sets show that the DF-CNN achieves superior
performance in challenging lifelong learning settings, resists catastrophic forgetting, and exhibits
reverse transfer to improve previously learned tasks
from subsequent experience without retraining