Abstract
Many common character-level, string-tostring transduction tasks, e.g. graphemeto-phoneme conversion and morphological
inflection, consist almost exclusively of
monotonic transduction. Neural sequence-tosequence models with soft attention, which
are non-monotonic, often outperform popular
monotonic models. In this work, we ask the
following question: Is monotonicity really
a helpful inductive bias in these tasks? We
develop a hard attention sequence-to-sequence
model that enforces strict monotonicity and
learns a latent alignment jointly while learning
to transduce. With the help of dynamic programming, we are able to compute the exact
marginalization over all monotonic alignments. Our models achieve state-of-the-art
performance on morphological inflection. Furthermore, we find strong performance on two
other character-level transduction tasks. Code
is available at https://github.com/
shijie-wu/neural-transducer