Abstract
Neural architectures based on self-attention,
such as Transformers, recently attracted interest from the research community, and obtained
significant improvements over the state of the
art in several tasks. We explore how Transformers can be adapted to the task of Neural Question Generation without constraining
the model to focus on a specific answer passage. We study the effect of several strategies
to deal with out-of-vocabulary words such as
copy mechanisms, placeholders, and contextual word embeddings.
We report improvements obtained over the
state-of-the-art on the SQuAD dataset according to automated metrics (BLEU, ROUGE), as
well as qualitative human assessments of the
system outputs.