Abstract 

We consider the problemP of finding the minimizer of a convex function F : of the form where a low-rank factorization of is readily available. We consider the regime where We propose randomized Newton-type algorithms that exploit non-uniform sub-sampling of as well as inexact updates, as means to reduce the computational complexity, and are applicable to a wide range of problems in machine learning. Two non-uniform sampling distributions based on block norm squares and block partial leverage scores are considered. Under certain assumptions, we show that our algorithms inherit a linear-quadratic convergence rate in w and achieve a lower computational complexity compared to similar existing methods. In addition, we show that our algorithms exhibit more robustness and better dependence on problem specific quantities, such as the condition number. We empirically demonstrate that our methods are at least twice as fast as Newton’s methods on several real datasets.