Dynamic Electronic Toll Collection via Multi-Agent Deep Reinforcement Learning
with Edge-Based Graph Convolutional Networks
Abstract
Over the past decades, Electronic Toll Collection
(ETC) systems have been proved the capability of
alleviating traffic congestion in urban areas. Dynamic Electronic Toll Collection (DETC) was recently proposed to further improve the efficiency
of ETC, where tolls are dynamically set based on
traffic dynamics. However, computing the optimal DETC scheme is computationally difficult and
existing approaches are limited to small scale or
partial road networks, which significantly restricts
the adoption of DETC. To this end, we propose a
novel multi-agent reinforcement learning (RL) approach for DETC. We make several key contributions: i) an enhancement over the state-of-the-art
RL-based method with a deep neural network representation of the policy and value functions and
a temporal difference learning framework to accelerate the update of target values, ii) a novel edgebased graph convolutional neural network (eGCN)
to extract the spatio-temporal correlations of the
road network state features, iii) a novel cooperative
multi-agent reinforcement learning (MARL) which
divides the whole road network into partitions according to their geographic and economic characteristics and trains a tolling agent for each partition. Experimental results show that our approach
can scale up to realistic-sized problems with robust
performance and significantly outperform the stateof-the-art method