Learning Barycentric Representations of 3D Shapes for Sketch-based 3D Shape
Retrieval
Abstract
Retrieving 3D shapes with sketches is a challenging
problem since 2D sketches and 3D shapes are from two
heterogeneous domains, which results in large discrepancy
between them. In this paper, we propose to learn barycenters of 2D projections of 3D shapes for sketch-based 3D
shape retrieval. Specifically, we first use two deep convolutional neural networks (CNNs) to extract deep features of
sketches and 2D projections of 3D shapes. For 3D shapes,
we then compute the Wasserstein barycenters of deep features of multiple projections to form a barycentric representation. Finally, by constructing a metric network, a discriminative loss is formulated on the Wasserstein barycenters of 3D shapes and sketches in the deep feature space
to learn discriminative and compact 3D shape and sketch
features for retrieval. The proposed method is evaluated
on the SHREC’13 and SHREC’14 sketch track benchmark
datasets. Compared to the state-of-the-art methods, our
proposed method can significantly improve the retrieval
performance