Abstract

Sparse matrix factorization is a popular tool to obtain interpretable data decompositions, which are also effective to perform data completion or denoising. Its applicability to large datasets has been addressed with online and randomized methods, that reduce the complexity in one of the matrix dimension, but not in both of them. In this paper, we tackle very large matrices in both dimensions. We propose a new factorization method that scales gracefully to terabyte-scale datasets. Those could not be processed by previous algorithms in a reasonable amount of time. We demonstrate the efficiency of our approach on massive functional Magnetic Resonance Imaging (fMRI) data, and on matrix completion problems for recommender systems, where we obtain significant speed-ups compared to state-of-the art coordinate descent methods. Matrix factorization is a flexible tool for uncovering lafactors in low-rank or sparse models. For instance, building on low-rank structure, it has proven very powerful fomatrix completion, e.g. in recommender systems (Srebro et al., 2004; Cand? & Recht, 2009). In signal processing and computer vision, matrix factorization with a sparregularization is often called dictionary learning and haproven very effective for denoising and visual feature encoding (see Mairal, 2014, for a review). It is also flexienough to accommodate a large set of constraints and regularizations, and has gained significant attention in sciedomains where interpretability is a key aspect