Abstract Pose variation is one of the challenging factors for face recognition. In this paper, we propose a novel cross-pose face recognition method named as Regularized Latent Least Square Regression (RLLSR). The basic assumption is that the images captured under different poses of one person can be viewed as pose-specific transforms of a single ideal object. We treat the observed images as regressor, the ideal object as response, and then formulate this assumption in the least square regression framework, so as to learn the multiple pose-specific transforms. Specifically, we incorporate some prior knowledge as two regularization terms into the least square approach: 1) the smoothness regularization, as the transforms for nearby poses should not differ too much; 2) the local consistency constraint, as the distribution of the latent ideal objects should preserve the geometric structure of the observed image space. We develop an alternating algorithm to simultaneously solve for the ideal objects of the training individuals and a set of pose-specific transforms. The experimental results on the Multi-PIE dataset demonstrate the effectiveness of the proposed method and superiority over the previous methods.