NTM-Lasagne is a library to create Neural Turing Machines (NTMs) in Theano using the Lasagne library. If you want to learn more about NTMs, check out our blog post.
This library features:
A Neural Turing Machine layer NTMLayer, where all its components (controller, heads, memory) are fully customizable.
Two types of controllers: a feed-forward DenseController and a "vanilla" recurrent RecurrentController.
A dashboard to visualize the inner mechanism of the NTM.
Generators to sample examples from algorithmic tasks.
Getting started
To avoid any conflict with your existing Python setup, and to keep this project self-contained, it is suggested to work in a virtual environment with virtualenv. To install virtualenv:
sudo pip install --upgrade virtualenv
Create a virtual environment called venv, activate it and install the requirements given by requirements.txt. NTM-Lasagne requires the bleeding-edge version, check the Lasagne installation instructions for details. The latest version of Lasagne is included in the requirements.txt.
For more detailed examples, check the examples folder. If you would like to train a Neural Turing Machine on one of these examples, simply run the corresponding script, like
PYTHONPATH=. python examples/copy-task.py
Tests
This projects has a few basic tests. To run these tests, you can run the py.test on the project folder
venv/bin/py.test ntm -vv
Known issues
Graph optimization is computationally intensive. If you are encountering suspiciously long compilation times (more than a few minutes), you may need to increase the amount of memory allocated (if you run it on a Virtual Machine). Alternatively, turning off the swap may help for debugging (with swapoff/swapon).
Note: Unlucky initialisation of the parameters might lead to a diverging solution (NaN scores).
Paper
Alex Graves, Greg Wayne, Ivo Danihelka, Neural Turing Machines, [arXiv]