Abstract
In this paper, we present a simple, yet effective,
attention and memory mechanism that is reminiscent of Memory Networks and we demonstrate it
in question-answering scenarios. Our mechanism
is based on four simple premises: a) memories can
be formed from word sequences by using convolutional networks; b) distance measurements can
be taken at a neuronal level; c) a recursive softmax function can be used for attention; d) extensive
weight sharing can help profoundly. We achieve
state-of-the-art results in the bAbI tasks, outperforming Memory Networks and the Differentiable
Neural Computer, both in terms of accuracy and
stability (i.e. variance) of results