This is an implementation of Semi-supervised generative adversarial network in the paper Improved Techniques for Training GANs for Mnist
dataset.
This method and its extensions have marvellous performance on
traditional CV datasets, and remain state-of-art (by the end of
November, 2017).
Working Principle
Inspired by Good Semi-supervised Learning that Requires a Bad GAN,
semi-supervised GAN with feature matching actually generates
unrealistic fake samples around high-density region. With the inborn
continuity, the fake region in feature space split the bounds of different classes.
Default configs include CPU, saving and autoloading, generating logfile in tensorboard format, etc. You can use python ImprovedGAN.py --cuda to run it on GPU.
The latesttorch(1.2 version), tensorboardX, torchvision are needed.
Result
Default configs can train models achieving 98.5% accuracy on test dataset with 100 labeled data(10 per class) and other 59,000 unlabeled data after 100 epochs.
Loss curve during training
loss_label => red, loss_unlabel => blue, loss_gen => green
It must be noted that OpenAI implementation(theano) demonstrates a different curve, where loss_gen is nearly zero and loss_unlabel increase gradually.
But I found it hard to reproduce expected results and suffered from
exploding gradients. I changed the final layer in generator from Sigmoid to Softplus, and therefore fixed it.
./models includes the trained model, you can simply delete it for retraining.
The archectures of networks are elaborately designed, among them Weight Normalization is very important.
Thank Jiapeng Hong for discussing with me.
Change Logs
(Nov 27, 2019) Update to pytorch 1.2 and Python 3.7. The version for
pytorch 0.3 and Python 2.7 can be found in the history versions. Delete
pretrained models.