When a Good Translation is Wrong in Context: Context-Aware Machine
Translation Improves on Deixis, Ellipsis, and Lexical Cohesion
Abstract
Though machine translation errors caused by
the lack of context beyond one sentence have
long been acknowledged, the development of
context-aware NMT systems is hampered by
several problems. Firstly, standard metrics are
not sensitive to improvements in consistency
in document-level translations. Secondly, previous work on context-aware NMT assumed
that the sentence-aligned parallel data consisted of complete documents while in most
practical scenarios such document-level data
constitutes only a fraction of the available parallel data. To address the first issue, we perform a human study on an English-Russian
subtitles dataset and identify deixis, ellipsis
and lexical cohesion as three main sources of
inconsistency. We then create test sets targeting these phenomena. To address the second
shortcoming, we consider a set-up in which a
much larger amount of sentence-level data is
available compared to that aligned at the document level. We introduce a model that is
suitable for this scenario and demonstrate major gains over a context-agnostic baseline on
our new benchmarks without sacrificing performance as measured with BLEU