Unsupervised Bilingual Word Embedding Agreementfor Unsupervised Neural Machine Translation
Abstract
Unsupervised bilingual word embedding
(UBWE), together with other technologies
such as back-translation and denoising,
has helped unsupervised neural machine
translation (UNMT) achieve remarkable
results in several language pairs. In previous
methods, UBWE is first trained using nonparallel monolingual corpora and then this
pre-trained UBWE is used to initialize the
word embedding in the encoder and decoder
of UNMT. That is, the training of UBWE
and UNMT are separate. In this paper, we
first empirically investigate the relationship
between UBWE and UNMT. The empirical
findings show that the performance of UNMT
is significantly affected by the performance
of UBWE. Thus, we propose two methods
that train UNMT with UBWE agreement.
Empirical results on several language pairs
show that the proposed methods significantly
outperform conventional UNMT