Gym-TORCS

Gym-TORCS is the reinforcement learning (RL) environment in TORCS domain with OpenAI-gym-like interface. TORCS is the open-rource realistic car racing simulator recently used as RL benchmark task in several AI studies.

Gym-TORCS is the python wrapper of TORCS for RL experiment with the simple interface (similar, but not fully) compatible with OpenAI-gym environments. The current implementaion is for only the single-track race in practie mode. If you want to use multiple tracks or other racing mode (quick race etc.), you may need to modify the environment, "autostart.sh" or the race configuration file using GUI of TORCS.

This code is developed based on vtorcs (https://github.com/giuse/vtorcs) and python-client for torcs (http://xed.ch/project/snakeoil/index.html).

The detailed explanation of original TORCS for AI research is given by Daniele Loiacono et al. (https://arxiv.org/pdf/1304.1672.pdf)

Because torcs has memory leak bug at race reset. As an ad-hoc solution, we relaunch and automate the gui setting in torcs. Any better solution is welcome!

Requirements

We are assuming you are using Ubuntu 14.04 LTS/16.04 LTS machine and installed

• Python 3

• xautomation (http://linux.die.net/man/7/xautomation)

• OpenAI-Gym (https://github.com/openai/gym)

• numpy

• vtorcs-RL-color (installation of vtorcs-RL-color is explained in vtorcs-RL-color directory)

Example Code

The example code and agent are written in example_experiment.py and sample_agent.py.

Initialization of the Race

After the insallation of vtorcs-RL-color, you need to initialize the race setting. You can find the detailed explanation in a document (https://arxiv.org/pdf/1304.1672.pdf), but here I show the simple gui-based setting.

So first you need to run

sudo torcs

in the terminal, the GUI of TORCS should be launched. Then, you need to choose the race track by following the GUI (Race --> Practice --> Configure Race) and open TORCS server by selecting Race --> Practice --> New Race. This should result that TORCS keeps a blue screen with several text information.

If you need to treat the vision input in your AI agent, you have to set the small image size in TORCS. To do so, you have to run

python snakeoil3_gym.py

in the second terminal window after you open the TORCS server (just as written above). Then the race starts, and you can select the driving-window mode by F2 key during the race.

After the selection of the driving-window mode, you need to set the appropriate gui size. This is done by using the display option mode in Options --> Display. You can select the Screen Resolution, and you need to select 64x64 for visual input (our immplementation only support this screen size, other screen size results the unreasonable visual information). Then, you need to shut down TORCS to complete the configuration for the vision treatment.

Simple How-To

from gym_torcs import TorcsEnv#### Generate a Torcs environment# enable vision input, the action is steering only (1 dim continuous action)env = TorcsEnv(vision=True, throttle=False)# without vision input, the action is steering and throttle (2 dim continuous action)# env = TorcsEnv(vision=False, throttle=True)ob = env.reset(relaunch=True)  # with torcs relaunch (avoid memory leak bug in torcs)# ob = env.reset()  # without torcs relaunch# Generate an agentfrom sample_agent import Agent
agent = Agent(1)  # steering onlyaction = agent.act(ob, reward, done, vision=True)# single stepob, reward, done, _ = env.step(action)# shut down torcsenv.end()

Add Noise in Low-dim Sensors

If you want to apply sensor noise in low-dimensional sensors, you should

os.system('torcs -nofuel -nodamage -nolaptime -vision -noisy &')
os.system('torcs -nofuel -nolaptime -noisy &')

at 33 & 35th lines in gym_torcs.py

Great Application

gym-torcs was utilized in DDPG experiment with Keras by Ben Lau. This experiment is really great!

https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html

Acknowledgement

gym_torcs was developed during the spring internship 2016 at Preferred Networks.