Abstract

We introduce the problem of joint ob ject class sequencing and tra jectory triangulation (JOST), which is defined as the reconstruc- tion of the motion path of a class of dynamic ob jects through a scene from an unordered set of images. We leverage standard ob ject detection tech- inques to identify ob ject instances within a set of registered images. Each of these ob ject detections defines a single 2D point with a corresponding viewing ray. The set of viewing rays attained from the aggregation of all detections belonging to a common ob ject class is then used to estimate a motion path denoted as the ob ject class tra jectory. Our method jointly determines the topology of the ob jects motion path and reconstructs the 3D ob ject points corresponding to our ob ject detections. We pose the problem as an optimization over both the unknown 3D points and the topology of the path, which is approximated by a Generalized Min- imum Spanning Tree (GMST) on a multipartite graph and then refined through a continuous optimization over the 3D ob ject points. Experi- ments on synthetic and real datasets demonstrate the effectiveness of our method and the feasibility to solve a previously intractable problem.