Abstract
Most of the unsupervised dependency parsers
are based on probabilistic generative models
that learn the joint distribution of the given
sentence and its parse. Probabilistic generative models usually explicit decompose the
desired dependency tree into factorized grammar rules, which lack the global features of
the entire sentence. In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence
and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence. We propose two approaches to model the latent representation: the first deterministically summarizes the representation from the sentence
and the second probabilistically models the
representation conditioned on the sentence.
Our approach can be regarded as a new type
of autoencoder model to unsupervised dependency parsing that combines the benefits
of both generative and discriminative techniques. In particular, our approach breaks the
context-free independence assumption in previous generative approaches and therefore becomes more expressive. Our extensive experimental results on seventeen datasets from various sources show that our approach achieves
competitive accuracy compared with both generative and discriminative state-of-the-art unsupervised dependency parsers.