A Deep Reinforcement Learning Approach to Behavioral Planning for Autonomous Vehicles
Behavioral planning for autonomous vehicles in scenarios such as intersections and lane-changing/merging is a challenging task. The code in this repository is for a couple of the experiments performed for this paper.
Installation
The code of this repo has been tested on Ubuntu 14.04 and with Python 2.7
Include export SUMO_HOME=/usr/share/sumo in your ~/.bashrc and source ~/.bashrc
Training
To train the DRQN: Add a folder called Center in the gym-traffic folder. In the Center folder, create a subdirectory called frames. This is where the learning result frames are stored. Create a folder in the examples folder called drqn. This is where the weights checkpoint files are stored. The checkpoint files are created every 100 episodes, so you must train at least 100 in order to run the DRQN.
cd [repo_root]/gym-traffic/examples
python example_gym_traffic_drqn.py train
Experiments
For each of these experiments, you will probably want to set the delay to 0ms once the GUI opens. This will decrease the amount of time you spend waiting for the ego-vehicle to reach the intersection. The simulation should slow down when the ego-vehicle arrives, but you may still need to manage the speed manually via the delay setting.
To test a random routine:
cd [repo_root]/gym-traffic/examples
python example_gym_traffic_random.py
To test a TTC (Time to Collision) rule based approach:
cd [repo_root]/gym-traffic/examples
python example_gym_traffic_ttc.py
To test the DRQN (Deep Recurrent Q-Network) based approach (you must have trained at least 100 episodes before running this):