Deep AutoEncoders for Collaborative Filtering

This is not an official NVIDIA product. It is a research project described in: "Training Deep AutoEncoders for Collaborative Filtering"(https://arxiv.org/abs/1708.01715)

The model

The model is based on deep AutoEncoders.

Requirements

• Python 3.6

• Pytorch: pipenv install

• CUDA (recommended version >= 8.0)

Training using mixed precision with Tensor Cores

• You would need NVIDIA Volta-based GPU

• Checkout mixed precision branch

• For theory on mixed precision training see Mixed Precision Training paper

Getting Started

Run unittests first

The code is intended to run on GPU. Last test can take a minute or two.

$ python -m unittest test/data_layer_tests.py
$ python -m unittest test/test_model.py

Tutorial

Checkout this tutorial by miguelgfierro.

Get the data

Note: Run all these commands within your DeepRecommender folder

Netflix prize

• Download from here into your DeepRecommender folder

$ tar -xvf nf_prize_dataset.tar.gz
$ tar -xf download/training_set.tar
$ python ./data_utils/netflix_data_convert.py training_set Netflix

Data stats

Train the model

In this example, the model will be trained for 12 epochs. In paper we train for 102.

python run.py --gpu_ids 0 
--path_to_train_data Netflix/NF_TRAIN 
--path_to_eval_data Netflix/NF_VALID 
--hidden_layers 512,512,1024 
--non_linearity_type selu 
--batch_size 128 
--logdir model_save 
--drop_prob 0.8 
--optimizer momentum 
--lr 0.005 
--weight_decay 0 
--aug_step 1 
--noise_prob 0 
--num_epochs 12 
--summary_frequency 1000

Note that you can run Tensorboard in parallel

$ tensorboard --logdir=model_save

Run inference on the Test set

python infer.py 
--path_to_train_data Netflix/NF_TRAIN 
--path_to_eval_data Netflix/NF_TEST 
--hidden_layers 512,512,1024 
--non_linearity_type selu 
--save_path model_save/model.epoch_11 
--drop_prob 0.8 
--predictions_path preds.txt

Compute Test RMSE

python compute_RMSE.py --path_to_predictions=preds.txt

After 12 epochs you should get RMSE around 0.927. Train longer to get below 0.92

Results

It should be possible to achieve the following results. Iterative output re-feeding should be applied once during each iteration.

(exact numbers will vary due to randomization)