Source-linked AI summary
CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario
Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, Zhenhui Li
TL;DR
Traffic-signal reinforcement learning needs large-scale simulation data, but SUMO is not scalable to city-sized networks and traffic flows. The paper presents CityFlow, a microscopic, multithreaded simulator with reinforcement-learning interfaces, and reports more than twenty times the speed of SUMO while supporting city-wide simulation. CityFlow is also positioned as a testbed for traffic-signal control and broader transportation studies.
Problem
Traffic-signal reinforcement learning requires large data sets, while SUMO is not scalable to large road networks and traffic flows needed for city-scale studies.
Method
The paper creates CityFlow, a microscopic traffic simulator using optimized data structures, algorithms, and multithreading, with a Python interface for reinforcement learning.
Results
More than twenty times faster than SUMO, CityFlow supports city-wide traffic simulation and provides vehicle-duration differences from SUMO within a reasonable range.
Takeaways & Limitations
CityFlow can serve as a testbed for traffic-signal control and other large-scale transportation reinforcement-learning studies.
Abstract
from arXiv · showhide
Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.
1 INTRODUCTION
CityFlow is introduced to address the need for scalable traffic simulation in reinforcement-learning research, where real-world data are insufficient and SUMO cannot support city-scale networks and flows. It provides a faster, flexible simulator intended for city-wide traffic simulation and broader transportation studies.
- Traffic signal control is complex because traffic flow is dynamic and thousands of signals must be coordinated.
- Reinforcement-learning methods require large data sets that cities cannot safely or directly generate at sufficient scale.Using each minute as a sample yields only 1,440 samples per day, which is insufficient for training a powerful deep reinforcement-learning model.
- SUMO is not scalable to large road networks and traffic flows, limiting its use for city-scale reinforcement-learning studies.On a 30 × 30 grid with tens of thousands of vehicles, SUMO performs around three simulation steps per second; cities may contain thousands of intersections and hundreds of thousands of vehicles.
- CityFlow supports city-wide traffic simulation, flexible road-network, vehicle-model, and signal-plan definitions, and a reinforcement-learning interface.The simulator uses multithreading and is reported as more than twenty times faster than SUMO.
- CityFlow is intended to support transportation research beyond traffic signals, including large-scale studies and benchmark reinforcement-learning environments.The authors also plan to calibrate simulation parameters using real-world observations.
2 BRIEF DESCRIPTION
CityFlow combines microscopic vehicle simulation with optimized data structures, algorithms, and multithreading, while exposing stepwise control and state access for multi-agent reinforcement learning. Its design covers vehicle dynamics, intersections, lane changes, interfaces, and visualization.
- System Design: CityFlow is a microscopic simulator that updates each vehicle at every time step and accelerates simulation through multithreading and optimized structures and algorithms.
- Car Following Model: The car-following model selects each vehicle’s desired speed under safety, signal, leading-vehicle, and other constraints.It uses a modified Krauß model and ballistic position updates rather than Euler updates.
- Car Following Model: Collision-free speed is computed from the following vehicle’s speed, leading vehicle’s speed, decelerations, gap, and time-step interval by solving a quadratic equation.
- Intersection Logic: Intersection behavior enforces stopping at red signals, stopping when possible at yellow signals, and yielding to higher-priority vehicles.The simulator precomputes information to avoid costly brute-force collision checks.
- Lane Change Model: Lane changes use lane segments to search only nearby vehicles, while shadow vehicles notify surrounding traffic during a change.The segment structure supports related-vehicle searches in constant time, and shadow vehicles can become leaders in car-following.
- Interfaces and Visualization: The pybind11 interface supports stepwise simulation, state retrieval, traffic-signal control, and on-the-fly vehicle insertion.A web-based WebGL interface renders replay output for large-scale simulations.
3 PERFORMANCE
CityFlow is evaluated against SUMO for simulation efficiency and vehicle-duration effectiveness across traffic scenarios. It provides substantially higher simulation speed while producing vehicle-duration differences described as within a reasonable range.
- Efficiency: 25 times speedup is achieved on large-scale 30×30 road networks with tens of thousands of vehicles using 8 threads.This corresponds to 72 steps of simulation per second, and speedup becomes more significant with more threads.
- Efficiency: CityFlow outperforms SUMO across scenarios ranging from small to large traffic with a single thread.The advantage is even more significant when additional threads are used.
- Efficiency: CityFlow retrieves simulation information more efficiently through its Python interface than SUMO.The passage attributes this difference to CityFlow’s pybind11 integration versus SUMO’s socket-based interaction.
- Effectiveness: Vehicle-duration differences between CityFlow and SUMO remain within a reasonable range under different traffic-volume settings.Vehicle duration is measured as the time required for a vehicle to enter and leave the road network.
4 DEMO DETAIL
The demo presents CityFlow across synthetic and real-world traffic scenarios, spanning small networks to large-scale settings. It also showcases reinforcement-learning training, interactive signal controls, and simulator performance comparisons.
- CityFlow is demonstrated across synthetic and real-world scenarios, from small road networks with dozens of vehicles to large networks with tens of thousands.
- Figure 1 shows CityFlow screenshots from different scenarios, while Figure 2 presents its speedup compared with SUMO.
- The demonstration includes a complete reinforcement-learning episode that optimizes traffic signal plans and displays gradual traffic-condition improvement.
- Participants can adjust signal cycle length, green ratio, and traffic volume while observing instant feedback on traffic conditions.
- The project is software-based, requires no special hardware, and is demonstrated using a laptop, ideally with an external monitor.
5 SUMMARY
CityFlow is proposed as an efficient, multi-agent reinforcement-learning environment for large-scale city traffic. It supports traffic-signal-control research and broader urban-mobility studies, with additional reinforcement-learning scenarios planned.
- CityFlow provides an efficient, multi-agent reinforcement-learning environment for large-scale city traffic and a testbed for traffic-signal-control research.
- The project is being developed to support broader transportation studies and additional scenarios such as dynamic vehicle routing and reversible or limited lanes.