Abstract In computer vision applications, features often lie on Riemannian manifolds with known geometry. Popular learning algorithms such as discriminant analysis, partial least squares, support vector machines, etc., are not directly applicable to such features due to the non-Euclidean nature of the underlying spaces. Hence, classifification is often performed in an extrinsic manner by mapping the manifolds to Euclidean spaces using kernels. However, for kernel based approaches, poor choice of kernel often results in reduced performance. In this paper, we address the issue of kernelselection for the classifification of features that lie on Riemannian manifolds using the kernel learning approach. We propose two criteria for jointly learning the kernel and the classififier using a single optimization problem. Specififically, for the SVM classififier, we formulate the problem of learning a good kernel-classififier combination as a convex optimization problem and solve it effificiently following the multiple kernel learning approach. Experimental results on image set-based classifification and activity recognition clearly demonstrate the superiority of the proposed approach over existing methods for classifification of manifold features.