Kaggle CIFAR-10

Code for CIFAR-10 competition. http://www.kaggle.com/c/cifar-10

Summary

Neural Network Configurations

Developer Environment

• Ubuntu 14.04

• 15GB RAM (This codebase can run on g2.2xlarge!)

• CUDA (GTX760 or more higher GPU)

• Torch7 latest

• cuda-convnet2.torch

Installation

(This document is outdated. See: Getting started with Torch)

Install CUDA (on Ubuntu 14.04):

apt-get install nvidia-331
apt-get install nvidia-cuda-toolkit

Install Torch7 (see Torch (easy) install):

curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-all | bash

Install(or upgrade) dependency packages:

luarocks install torch
luarocks install nn
luarocks install cutorch
luarocks install cunn
luarocks install https://raw.githubusercontent.com/soumith/cuda-convnet2.torch/master/ccn2-scm-1.rockspec

Checking CUDA environment

th cuda_test.lua

Please check your Torch7/CUDA environment when this code fails.

Convert dataset

Place the data files into a subfolder ./data.

ls ./data
test  train  trainLabels.csv

• th convert_data.lua

Local testing

th validate.lua

dataset:

Generating the submission.txt

th train.lua
th predict.lua

MISC

Model Averaging

Training with different seed parameter for each nodes.

(same model, same data, different initial weights, different training order)

th train.lua -seed 11
th train.lua -seed 12
...
th train.lua -seed 16

Mount the models directory for each nodes. for example, ec2/node1, ec2/node2, .., ec2/node6.

Edit the path of model file in predict_averaging.lua.

Run the prediction command.

th predict_averaging.lua

Network In Network

./nin_model.lua is an implementation of Network In Network [3]. This model gives score of 0.92400.

My NIN implementation is 2-layer NIN. Its differ from mavenlin's implementation. I tried to implement the mavenlin's 3-layer NIN. However, I did not get good result.

My implementation of 3-layer NIN is here.

Bug

global_contrast_normalization in ./lib/preprocessing.lua is incorrect implementation (This function is just z-score). but I was using this implementation in the competition.

Figure

data augmentation + preprocessing

References

• [1] Karen Simonyan, Andrew Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", link

• [2] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", link

• [3] Min Lin, Qiang Chen, Shuicheng Yan, "Network In Network", link

• [4] R. Collobert, K. Kavukcuoglu, C. Farabet, "Torch7: A Matlab-like Environment for Machine Learning"