Text To Image Synthesis

This is an experimental tensorflow implementation of synthesizing images. The images are synthesized using the GAN-CLS Algorithm from the paper Generative Adversarial Text-to-Image Synthesis. This implementation is built on top of the excellent DCGAN in Tensorflow.

Model architecture

Image Source : Generative Adversarial Text-to-Image Synthesis Paper

Requirements

TensorFlow 1.0+
TensorLayer 1.4+
NLTK : for tokenizer

Datasets

The model is currently trained on the flowers dataset. Download the images from here and save them in 102flowers/102flowers/*.jpg. Also download the captions from this link. Extract the archive, copy the text_c10 folder and paste it in 102flowers/text_c10/class_*.

N.B You can downloads all data files needed manually or simply run the downloads.py and put the correct files to the right directories.

python downloads.py

## Codes - `downloads.py` download Oxford-102 flower dataset and caption files(run this first). - `data_loader.py` load data for further processing. - `train_txt2im.py` train a text to image model. - `utils.py` helper functions. - `model.py` models. ## References - [Generative Adversarial Text-to-Image Synthesis][2] Paper - [Generative Adversarial Text-to-Image Synthesis][11] Torch Code - [Skip Thought Vectors][1] Paper - [Skip Thought Vectors][12] Code - [Generative Adversarial Text-to-Image Synthesis with Skip Thought Vectors](https://github.com/paarthneekhara/text-to-image) TensorFlow code - [DCGAN in Tensorflow][3] ## Results - the flower shown has yellow anther red pistil and bright red petals. - this flower has petals that are yellow, white and purple and has dark lines - the petals on this flower are white with a yellow center - this flower has a lot of small round pink petals. - this flower is orange in color, and has petals that are ruffled and rounded. - the flower has yellow petals and the center of it is brown - this flower has petals that are blue and white. - these white flowers have petals that start off white in color and end in a white towards the tips.