Abstract

Factorized information criterion (FIC) is a recently developed approximation technique for the marginal log-likelihood, which provides an automatic model selection framework for a few latent variable models (LVMs) with tractable inference algorithms. This paper reconsiders FIC and fills theoretical gaps of previous FIC studies. First, we reveal the core idea of FIC that allows generalization for a broader class of LVMs. Second, we investigate the model selection mechanism of the generalized FIC, which we provide a formal justification of FIC as a model selection criterion for LVMs and also a systematic procedure for pruning redundant latent variables. Third, we uncover a few previously-unknown relationships between FIC and the variational free energy. A demonstrative study on Bayesian principal component analysis is provided and numerical experiments support our theoretical results.