Abstract

We propose a Bayesian evidence framework to facilitatetransfer learning from pre-trained deep convolutional neu-ral networks (CNNs). Our framework is formulated on topof a least squares SVM (LS-SVM) classifier, which is simpleand fast in both training and testing, and achieves compet-itive performance in practice. The regularization param-eters in LS-SVM is estimated automatically without gridsearch and cross-validation by maximizing evidence, whichis a useful measure to select the best performing CNN outof multiple candidates for transfer learning; the evidenceis optimized efficiently by employing Aitken’s delta-squaredprocess, which accelerates convergence of fixed point up-date. The proposed Bayesian evidence framework also pro-vides a good solution to identify the best ensemble of heterogeneous CNNs through a greedy algorithm. Our Bayesian evidence framework for transfer learning is tested on 12 visual recognition datasets and illustrates the state-of-the-art performance consistently in terms of prediction accuracyand modeling efficiency.