Abstract

It is well known that the rolling shutter effect in imagescaptured with a moving rolling shutter camera causes inac-curacies to 3D reconstructions. The problem is further ag-gravated with weak visual connectivity from wide baselineimages captured with a fast moving camera. In this paper,we propose and implement a pipeline for sparse to dense3D construction with wide baseline images captured from afast moving rolling shutter camera. Specifically, we proposea cost function for Bundle Adjustment (BA) that models therolling shutter effect, incorporates GPS/INS readings, andenforces pairwise smoothness between neighboring poses.We optimize over the 3D structures, camera poses and ve-locities. We also introduce a novel interpolation scheme forthe rolling shutter plane sweep stereo algorithm that allowsus to achieve a 7× speed up in the depth map computationsfor dense reconstruction without losing accuracy. We eval-uate our proposed pipeline over a 2.6km image sequence captured with a rolling shutter camera mounted on a moving car.