Abstract. Humans recognize object structure from both their appearance and motion; often, motion helps to resolve ambiguities in object
structure that arise when we observe object appearance only. There
are particular scenarios, however, where neither appearance nor spatialtemporal motion signals are informative: occluding twigs may look connected and have almost identical movements, though they belong to
different, possibly disconnected branches. We propose to tackle this problem through spectrum analysis of motion signals, because vibrations of
disconnected branches, though visually similar, often have distinctive
natural frequencies. We propose a novel formulation of tree structure
based on a physics-based link model, and validate its effectiveness by theoretical analysis, numerical simulation, and empirical experiments. With
this formulation, we use nonparametric Bayesian inference to reconstruct
tree structure from both spectral vibration signals and appearance cues.
Our model performs well in recognizing hierarchical tree structure from
real-world videos of trees and vessels