dsl-char-cnn
Character-level CNN model for the DSL 2016 shared task.
Requirements
Keras
TensorFlow. Not tested with Theano, but should work fine except for some imports in the beginning of the code.
Cuda GPU card will make the code much faster, especially with cudnn.
Usage
To train a model, go to the src
dir and run:
python cnn_multifilter.py
This will train a model, save it to disk, and report some scores.
to test a model on raw texts, go to the src
dir and run:
python predict_test_data_with_trained_model.py
This will create a file with predictions under data
. It will also create a file with the posterior probabilitites. See example files under data
.
Note: file names are currently hard-coded in several places (e.g. model files in the train and test scripts, and data files in data.py
.
Citing
If you use this code in your work, please consider citing our paper: "A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects", Yonatan Belinkov and James Glass, VarDial 2016.
@InProceedings{belinkov-glass:2016:VarDial,
author = {Belinkov, Yonatan and Glass, James},
title = {A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects},
booktitle = {Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)},
month = {December},
year = {2016},
address = {Osaka, Japan}
}
TODO
Acknowledgements
This implementation uses code from Franois Chollet's IMBC CNN example.