This is a tensorflow based implementation for our ICML 2017 paper on curiosity-driven exploration for reinforcement learning.
Idea is to train agent with intrinsic curiosity-based motivation (ICM)
when external rewards from environment are sparse. Surprisingly, you can
use ICM even when there are no rewards available from the environment,
in which case, agent learns to explore only out of curiosity: 'RL
without rewards'. If you find this work useful in your research, please
cite:
@inproceedings{pathakICMl17curiosity,
Author = {Pathak, Deepak and Agrawal, Pulkit and
Efros, Alexei A. and Darrell, Trevor},
Title = {Curiosity-driven Exploration by Self-supervised Prediction},
Booktitle = {International Conference on Machine Learning ({ICML})},
Year = {2017}
}
1) Installation and Usage
This code is based on TensorFlow. To install, run these commands:
# you might not need many of these, e.g., fceux is only for mariosudo apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb
libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig python3-dev
python3-venv make golang libjpeg-turbo8-dev gcc wget unzip git fceux virtualenv
tmux# install the codegit clone -b master --single-branch https://github.com/pathak22/noreward-rl.gitcd noreward-rl/
virtualenv curiositysource $PWD/curiosity/bin/activate
pip install numpy
pip install -r src/requirements.txt
python curiosity/src/go-vncdriver/build.py# download modelsbash models/download_models.sh# setup customized doom environmentcd doomFiles/# then follow commands in doomFiles/README.md