This repository holds the basic implementation of a RNN.

It is based on the blog post The Unreasonable Effectiveness of Recurrent Neural Networks from Andrej Karpathy

The code is basically a transcript from his gist.

I also got some help from Daniel Whitenack's Building a Neural Net from Scratch in Go

For more information, please refer to this blog post

Configuration

Hyper parameters of the neural nerwork

RNN_INPUTNEURONS      Integer
RNN_OUTPUTNEURONS     Integer
RNN_HIDDENNEURONS     Integer    100        trueRNN_LEARNINGRATE      Float      1e-1       trueRNN_ADAGRADEPSILON    Float      1e-8       trueRNN_RANDOMFACTOR      Float      0.01

Parameters of the executable

MIN_CHAR_SAMPLESIZE         Integer    100        trueMIN_CHAR_SAMPLEFREQUENCY    Integer    1000       trueMIN_CHAR_INFOFREQUENCY      Integer    100        trueMIN_CHAR_BACKUPFREQUENCY    Integer    1000       trueMIN_CHAR_BACKUPPREFIX       String
MIN_CHAR_BACKUPSUFFIX       String

Parameters of the char codec

CHAR_CODEC_CHOICE     hard|soft (default hard)
CHAR_CODEC_EPOCH      100
CHAR_CODEC_VOCAB_FILE
CHAR_CODEC_INPUT_FILE
CHAR_CODEC_BATCHSIZE  default 25

Usage

Example:

This will train the RNN with Shakespeare inputs and save every now and then the model to shakespeare.bin

export CHAR_CODEC_INPUT_FILE=data/shakespeare/input.txtexport CHAR_CODEC_VOCAB_FILE=data/shakespeare/vocab.txtexport RNN_ADAGRADEPSILON=1e-8export RNN_RANDOMFACTOR=0.1export RNN_LEARNINGRATE=1e-1export MIN_CHAR_CHOICE=hardexport RNN_HIDDENNEURONS=66export MIN_CHAR_BATCHSIZE=25export MIN_CHAR_SAMPLEFREQUENCY=1000export MIN_CHAR_EPOCHS=100export MIN_CHAR_SAMPLESIZE=500export MIN_CHAR_BACKUPPREFIX=shakespeareexport MIN_CHAR_BACKUPFREQUENCY=1000export CHAR_CODEC_CHOICE=softecho "starting sequence for the sampling" | ./min-char-rnn -train

To use the pre-train model:

echo "Initial sample" | ./min-char-rnn -restore shakespeare.bin