Abstract
Multilingual neural machine translation
(Multi-NMT) with one encoder-decoder
model has made remarkable progress due to
its simple deployment. However, this multilingual translation paradigm does not make
full use of language commonality and parameter sharing between encoder and decoder.
Furthermore, this kind of paradigm cannot
outperform the individual models trained on
bilingual corpus in most cases. In this paper,
we propose a compact and language-sensitive
method for multilingual translation. To
maximize parameter sharing, we first present
a universal representor to replace both encoder
and decoder models. To make the representor
sensitive for specific languages, we further
introduce language-sensitive embedding,
attention, and discriminator with the ability
to enhance model performance. We verify
our methods on various translation scenarios,
including one-to-many, many-to-many and
zero-shot. Extensive experiments demonstrate that our proposed methods remarkably
outperform strong standard multilingual
translation systems on WMT and IWSLT
datasets. Moreover, we find that our model
is especially helpful in low-resource and
zero-shot translation scenarios.