Abstract
We study data-driven representations for threedimensional triangle meshes, which are one of the
prevalent objects used to represent 3D geometry. Recent
works have developed models that exploit the intrinsic geometry of manifolds and graphs, namely the Graph Neural
Networks (GNNs) and its spectral variants, which learn
from the local metric tensor via the Laplacian operator.
Despite offering excellent sample complexity and builtin invariances, intrinsic geometry alone is invariant to isometric deformations, making it unsuitable for many applications. To overcome this limitation, we propose several
upgrades to GNNs to leverage extrinsic differential geometry properties of three-dimensional surfaces, increasing its
modeling power. In particular, we propose to exploit the
Dirac operator, whose spectrum detects principal curvature directions — this is in stark contrast with the classical
Laplace operator, which directly measures mean curvature.
We coin the resulting models Surface Networks (SN).
We prove that these models define shape representations that are stable to deformation and to discretization,
and we demonstrate the efficiency and versatility of SNs on
two challenging tasks: temporal prediction of mesh deformations under non-linear dynamics and generative models using a variational autoencoder framework with encoders/decoders given by SNs