Create a conda environment from the environment.yml file using the following command in terminal: $ conda env create -f environment.yml
In your home directory, create a .theanorc file containing:
[global]
floatX = float32
device = cuda0
Dataset
Download the NYU dataset and unzip it. The code is designed to process the data in HDF5 format using h5py. To convert the dataset in HDF5 format run the following code in your terminal:
from datasets_preprocessing.datasets import NYU_Datasetnyu = NYU_Dataset('/path/NYU/dataset', '/path/NYU/hdf5')
nyu.convert_to_hdf5()
where /path/NYU/ should be replaced with the location of the unziped file from above. In datasets_preprocessing.datasets, there are also classes for converting to HDF5 the ICVL and MSRA datasets. Only NYU contains RGB-D images, while ICVL and MSRA contain only depth images, so experiments have been done only for NYU. Nevertheless, you may want to train just the depth stream for ICVL and MSRA.
For a full description of the input arguments do python training_script.py -h.
Publication
Please reference this publication if you find this code useful:
@inproceedings{kazakos_fusion_icip2018,
author={E. Kazakos and C. Nikou and I. A. Kakadiaris},
booktitle={25th IEEE International Conference on Image Processing (ICIP)},
title={On the Fusion of RGB and Depth Information for Hand Pose Estimation},
year={2018},
pages={868-872},
month={Oct},
}
Citations
J. Tompson, M. Stein, Y. LeCun, and K. Perlin, “Real- Time Continuous Pose Recovery of Human Hands Using Convolutional Networks,” ACM Transactions on Graphics, vol. 33, pp. 169:1–169:10, 2014.