GPT2 Generator Demo in GluonNLP

First, you need to convert the pretrained models in openai/gpt-2 to GluonNLP

Step-1: Download the pretrained models provided by OpenAI.

git clone https://github.com/openai/gpt-2.git cd gpt-2 python download_model.py 117M python download_model.py 345M cd ..

Step-2: Convert the model to GluonNLP (Make sure the current directory is gluonnlp-gpt2)

python gpt2_tf_to_gluonnlp.py --src_dir gpt-2/models --dst_dir models --model 117M python gpt2_tf_to_gluonnlp.py --src_dir gpt-2/models --dst_dir models --model 345M

Step-3: Test the correctness of the pretrained models:

nosetests3 test_transform.py python test_model.py

Step-4: Try the generation demo

For the conditional generator, use the following command

python sampling_demo.py --model 117M

For the unconditional generator, use the following command

python sampling_demo.py --model 117M --unconditional

You can also try the large model with 345M parameters

python sampling_demo.py --model 345M python sampling_demo.py --model 345M --unconditional

Dependencies

mxnet gluonnlp regex tensorflow