Run python preprocess.py to get the wordpiece based preprocessed data.
Run python train_QANet.py to start training.
Updates
Add EMA (with about 3% improvement)
Add multi gpu (speed up)
Support adding handcraft features
Revised the MultiHeadAttention and PositionEmbedding in keras
Support parallel multi-gpu training and inference
Add layer dropout and revise the dropout bug (with about 2% improvement)
Update the experimental results and related hyper-parameters (in train_QANet.py)
Revise the output Layer QAoutputBlock.py(with about 1% improvement)
Replace the BatchNormalization with the LayerNormalization in layer_norm.py(about 0.5% improvement)
Add slice operation to QANet (double speed up)
Add Cove (about 1.4% improvement)
Implement the EMA in keras-gpu. (30% spped up)
Add WordPiece in keras (from BERT) (0.5% improvement)
Add data augmentation
I find that EMA in keras is hard to implement with GPU, and the training speed is greatly affected by it in keras. Besides, it's hard to add the slice op in keras too, so the training speed is further slower(cost about twice as much time compared with the optimized tensorflow version...).
Now, the gpu-version EMA can work perporly in keras.