Abstract In this paper, we propose a novel linear subspace learning algorithm called Flexible Orthogonal Neighborhood Preserving Embedding (FONPE), which is a linear approximation of Locally Linear Embedding (LLE) algorithm. Our novel objective function integrates two terms related to manifold smoothness and a flexible penalty defined on the projection fitness. Different from Neighborhood Preserving Embedding (NPE), we relax the hard constraint PT X = Y by modeling the mismatch between PT X and Y, which makes it better cope with the data sampled from a non-linear manifold. Besides, instead of enforcing an orthogonality between the projected points, i.e. (PT X)(PT X)T = I, we enforce the mapping to be orthogonal, i.e. PT P = I. By using this method, FONPE tends to preserve distances so that the overall geometry can be preserved. Unlike LLE, as FONPE has an explicit linear mapping between the input and the reduced spaces, it can handle novel testing data straightforwardly. Moreover, when P becomes an identity matrix, our model can be transformed into denoising LLE (DLLE). Compared with the standard LLE, we demonstrate that DLLE can handle data with noise better. Comprehensive experiments on several benchmark databases demonstrate the effectiveness of our algorithm.