Abstract We propose to tackle the multiview 2D/3D rigid registration problem via a Point-Of-Interest Network for Tracking and Triangulation (POINT2 ). POINT2 learns to establish 2D point-to-point correspondences between the preand intra-intervention images by tracking a set of pointof-interests (POIs). The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In POINT2 , the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches, POINT2 only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant, POINT2 is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam computed tomography dataset show that the proposed POINT2 method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore, when used as an initial pose estimator, our method also improves the robustness and speed of the state-of-the-art optimizationbased approaches by ten folds.