Source-linked AI summary
Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems
Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang, Yuxuan Liang
TL;DR
Traffic-signal control still faces challenges in generalizing across networks, explaining decisions, and handling unusual incidents. Traffic-R1 uses a compact model trained with expert guidance and simulated exploration, with reasoning and inter-intersection communication. Its live deployment reports reduced queues and operator workload.
Problem
Operational traffic-signal control still relies on heuristics and substantial human oversight, while current methods face generalization, interpretability, and out-of-distribution incident challenges.
Source: §1 · p. 2Method
Traffic-R1 is a 3B-parameter model trained with offline expert-informed and online simulated reinforcement learning, generating reasoning alongside actions and coordinating intersections through asynchronous message passing.
Source: §1 · p. 2Results
>5% reduction in average queues and >50% reduction in operator workload were reported in live deployment serving over 55,000 drivers daily.
Source: §1 · p. 2Takeaways & Limitations
Evaluation on standard benchmarks and traffic-incident handling showed improvement over prior methods, and real-world tests demonstrated value for industrial deployment.
Source: §7 · p. 8Traffic observations and incident information must be converted to structured text, which can be inconvenient and risk information loss in dense deployments.
Source: §8 · p. 9
Abstract
from arXiv · showhide
We introduce Traffic-R1, a 3B-parameter foundation model with human-like reasoning for Traffic signal control (TSC), developed via self-exploration and iterative reinforcement of LLM with expert guidance in a simulated traffic environment. Compared with traditional reinforcement learning and recent LLM-based methods, Traffic-R1 offers three main advantages: zero-shot generalization, transferring unchanged to new road networks and out-of-distribution incidents by leveraging internal traffic-control policies and reasoning; a compact 3B-parameter design that supports real-time inference on mobile-class chips for edge deployment; and an explainable TSC process that enables multi-intersection coordination through communication and an asynchronous communication network. Extensive benchmarks show Traffic-R1 outperforms strong baselines and training-intensive RL controllers. In production, the model now manages signals affecting over 55,000 drivers daily, reduces average queue lengths by more than 5%, and halves operator workload. Our model is available at https://huggingface.co/Season998/Traffic-R1.
1 Introduction
Traffic-R1 is introduced as a compact, general-purpose traffic-signal controller intended to address gaps in generalization, interpretability, and incident response. Its two-stage reinforcement-learning approach combines expert guidance and simulated exploration with reasoning and asynchronous communication.
1 Introduction
- Existing controllers and RL methods face cross-region transfer, interpretability, and out-of-distribution incident challenges that limit field deployment.The introduction also identifies operational reliance on heuristics and substantial human oversight.Source: §1 · pp. 1–2
- LLM Explainers inherit an underlying RL policy’s performance ceiling, while LLM Assistants add prompt engineering and repeated queries that increase latency and computation.The introduction says post-hoc explanations may diverge from controller logic and everyday timing benefits are limited.Source: §1 · p. 2
- Traffic-R1 uses offline expert-informed and online simulated-environment RL to train a 3B-parameter model to generate reasoning alongside control actions.Rewards address output format and action correctness; online action rewards come from simulated traffic feedback.Source: §1 · p. 2
- Self-generated training samples help mitigate catastrophic forgetting, while an asynchronous message-passing network coordinates agents across intersections.The introduction links retained language communication skills and explainable reasoning with multi-intersection coordination.Source: §1 · p. 2
- >5% reduction in average queues and >50% reduction in operator workload were reported in live deployment serving over 55,000 drivers daily.The workload reduction concerns phase planning and incident response.Source: §1 · p. 2
2 Related Works
Related work traces traffic-signal control from fixed and pressure-based rules to RL and LLM integrations, then reviews instruction and reinforcement fine-tuning for language models.
Traffic Signal Control
- FixedTime uses expert-set cycles and phase allocations, while MaxPressure prioritizes movements using queue-based pressure; both remain constrained under varying scenarios.RL-based methods improve simulated performance but often fall short of real-world deployment.Source: §2 · p. 3
- LLM Explainers verbalize an RL controller’s policy, whereas LLM Assistants consult an LLM for out-of-distribution incidents; both paradigms retain practical deployment challenges.Explainers inherit the RL policy’s coverage and performance limits, while Assistants add prompt engineering and repeated queries.Source: §2 · p. 3
RL finetuning for LLM
- Instruction fine-tuning is widely used for task- and format-specific adaptation, while RL fine-tuning is less adopted because of pipeline complexity and compute cost.Prior work associates RL fine-tuning with self-exploration, capability acquisition, and KL-constrained updates that can mitigate catastrophic forgetting.Source: §2 · p. 3
3 Preliminaries
The preliminaries model traffic networks as directed graphs and define signal phases and multi-agent traffic-signal control.
Road Network
- A road network is a directed graph of intersections and lanes, with lanes classified by turn type and divided into distance-based segments.The lane types are go-through, left-turn, and right-turn.Source: §3 · p. 3
Signal Phase
- At each signal-switching step, a signal phase selects permitted lanes from a predefined set, giving those lanes green while conflicting lanes remain red.The permitted lane set defines the phase.Source: §3 · p. 3
Traffic Signal Control System
- The control system comprises agents managing individual intersections and collaborating with neighbors through observations and message passing at signal-switching steps.Coordination examples include green-wave synchronization and emergency response.Source: §3 · p. 3
4 Methodology
Traffic-R1’s methodology combines offline human-informed RL, online open-world RL, and asynchronous communication to support multi-intersection coordination.
4.1 Human-Informed RL
- Offline human-informed RL uses real traffic experts rather than an RL-model teacher, addressing limits in the teacher’s performance, generalization, and reasoning.The approach combines an expert-collaborative dataset with offline policy optimization.Source: §4.1 · p. 4
4.1.1 Expert-Collaborative TSC Dataset
- 3,000 validated traffic scenarios form a dataset built with 11 experts, using simulator-checked actions and excluding reasoning steps to encourage the model’s own reasoning.A base model proposes each action; two traffic-control experts review it, and experts provide a correction when it is rejected.Source: §4.1.1 · p. 4
4.1.2 Offline Policy Optimization
- Offline policy optimization rewards exact matches to expert actions and adherence to the required reasoning-and-answer format.The model generates structured responses from textual traffic scenarios, and Group Relative Policy Optimization is used for stable updates.Source: §4.1.2 · p. 4
- Group-normalized advantages center rewards among completions for the same prompt, while a KL penalty constrains policy updates.The KL penalty’s strength is set by β; the reference policy is typically a snapshot of the previous policy.Source: §4.1.2 · pp. 4–5
4.2 Open-World Reinforcement Learning
- Online RL trains in simulated multi-intersection traffic using cumulative queue-length and waiting-time rewards; STPO spreads trajectory reward across steps to reduce computational overhead.The simulation uses a 4 × 4 network, and trajectory concatenation is described as impractical because it increases context, compute, and memory costs.Source: §4.2 · p. 5
4.3 Asynchronous Communication Network
- Traffic intersections are divided into parity-based groups activated in two inner steps, letting the first group send messages that the second receives and processes.This communication structure is designed to use LLM language capabilities for multi-intersection coordination.Source: §4.3 · pp. 5–6
5 Experiments
The experiments evaluate Traffic-R1 on conventional traffic control, out-of-distribution incidents, reinforcement-learning fine-tuning, and component ablations. They report zero-shot performance against multiple baseline families and test how training stages and communication contribute to results.
5.1.1 Dataset
- The evaluation uses two public traffic-flow datasets and 200 textual examples of out-of-distribution incidents, including accidents and emergency scenarios.Hangzhou1 is also modified with a 5% proportion of emergency-vehicle flow for emergency coordination scenarios.Source: §5.1.1 · p. 6, Table 1 · p. 6
5.1.2 Implementation Details
- Traffic-R1 is evaluated in CityFlow using four signal phases, 15-second green periods, and one-hour traffic-flow simulations.Training uses four H100 GPUs; inference is reported on a single Tesla T40 GPU.Source: §5.1.2 · p. 6
5.1.3 Baseline Methods
- Comparisons include traditional, reinforcement-learning, LLM-based, and zero-shot controllers, with learning-based baselines trained on the same simulated network and traffic-flow data.LLMLight and CoLLMLight also receive the expert-collaborative dataset in their training instructions.Source: §5.1.3 · p. 6
5.1.4 Evaluation Protocols
- Average Travel Time and Average Waiting Time are the evaluation metrics, and lower values indicate better traffic efficiency.Table 2 reports zero-shot performance on conventional traffic signal control tasks.Source: §5.1.4 · pp. 6–7, Table 2 · p. 7
5.2 Conventional TSC Evaluation (RQ1)
- Traffic-R1 significantly outperforms all baselines on conventional TSC tasks in zero-shot evaluation, while zero-shot RL baselines can lag behind traditional methods.The authors also report that Traffic-R1 exceeds RL methods trained on each test dataset in full-shot evaluation.Source: §5.2 · p. 7, Table 2 · p. 7
5.3 OOD Incident Dealing (RQ2)
- Traffic-R1 outperforms larger general LLMs and LLMLight by over 30% across OOD metrics, with particularly strong performance on network-wide incidents.The authors attribute coordinated multi-intersection control to the asynchronous communication design.Source: §5.3 · p. 7, Table 3 · p. 7
5.4 Discussion of RL-based Finetuning (RQ3)
- RL-based fine-tuning outperforms instruction fine-tuning on zero-shot TSC, while self-rollouts and a KL-divergence penalty help preserve general capabilities.The authors characterize instruction fine-tuning as encouraging imitation of synthetic data and RL as fostering internal reasoning through self-exploration and iteration.Source: §5.4 · p. 8, Figure 5 · p. 8
5.5 Ablation Study (RQ4)
- Ablations indicate that expert-informed RL supports foundational TSC knowledge, open-world RL improves performance, and asynchronous communication matters for multi-intersection but not conventional traffic tasks.Figure 6 presents the ablation results on TSC and OOD tasks.Source: §5.5 · p. 8, Figure 6 · p. 8
6 Real-world deployment
Traffic-R1 is deployed on a city traffic platform, where a six-week A/B test reports improvements in traffic control and reduced labor.
6 Real-world deployment
- Traffic-R1 controls 10 key intersections and serves over 55,000 drivers daily in a Chinese city.The city name is withheld for privacy.Source: §6 · p. 8, Figure 7 · p. 8
- A six-week A/B test reports more efficient decision-making, better performance than human experts, and consistent operation with reduced labor.Table 6 lists average queue, maximum queue, and working hours as outcome measures.Source: §6 · p. 8, Table 4 · p. 8
- Table 6 reports average queue, maximum queue, and working hours as deployment evaluation measures.The supplied passage identifies these measures but does not provide their values.Source: Table 4 · p. 8
7 Conclusion and Future Work
The paper presents a two-stage reinforcement-learning strategy and a communication framework for a foundational traffic-control model. It reports improved benchmark and incident-handling evaluations, real-world deployment value, and reinforced vision-language models as future work.
7 Conclusion and Future Work
- A two-stage reinforcement-learning strategy and network communication framework convert an LLM into a foundational traffic-control model operating like a human traffic agent.Source: §7 · p. 8
- Evaluation on standard TSC benchmarks and traffic-incident handling shows improvement over prior methods, while real-world tests demonstrate industrial deployment value.Source: §7 · p. 8
- Future work proposes reinforced vision-language models for direct road-vision reasoning without structured textual traffic data.Source: §7 · p. 8
8 Limitations
The paper identifies structured-text dependence and limited large-scale field evaluation as limitations of Traffic-R1. It also reports that extended trials and systematic driver-response collection were constrained by practical and agency requirements.
8 Limitations
- Traffic observations and incident information must be converted into structured text, which may be inconvenient and lose information in dense real-world deployments.Source: §8 · p. 9
- The paper cannot provide detailed analysis of hallucination-related instability across large-scale traffic scenarios or assess driver feedback to the agent-based network.Source: §8 · p. 9
- Access, privacy, safety, cost, and trial complexity prevented extended field evaluations and systematic collection of driver-response data.Source: §8 · p. 9
A.1 Input Template for Traffic-R1
The rollout template presents structured intersection and lane observations for selecting a signal, with optional incident information. It directs the agent to reason internally before returning a boxed choice.
A.1 Input Template for Traffic-R1
- The template structures observations by signal phase, allowed lanes, and queued-vehicle counts across segments.The examples cover east-west and north-south through and left-turn movements.Source: §A.1 · p. 12
- Incident information is optional and explicitly unused during training.Source: §A.1 · p. 12
- The format instruction requests internal reasoning in <think> tags and the final signal choice in \boxed{ }.Source: §A.1 · p. 12
A.2 Introduction of baselines
The section groups comparison methods into traditional, RL-based, and zero-shot approaches, then lists representative controllers and language models. The zero-shot group is evaluated without training on the TSC evaluation datasets.
Traditional Methods
- Traditional methods include FixedTime, which uses predetermined phase splits, and MaxPressure, which selects the phase with the highest pressure.Source: §A.2 · p. 12
RL-based Methods
- RL-based methods normally train and interact with policies on each evaluation dataset; LLMLight-7B uses Advanced-CoLight actions and GPT-4 explanations for instruction finetuning.Source: §A.2 · p. 12
- The RL comparison set includes attention- and graph-based controllers such as AttendLight, CoLight, Efficient-CoLight, and Advanced-CoLight.Source: §A.2 · p. 12
Zero-shot Methods
- Zero-shot methods, including Traffic-R1, control each evaluation dataset using internal policies and knowledge without training on those datasets.Source: §A.2 · p. 12
- The listed zero-shot LLM baselines include CoLLMLight-8B, Llama3.3-70B, Qwen2.5-72B, GPT-3.5-turbo, and GPT-4o.Source: §A.2 · pp. 12–13
A.3 Traffic Incident Data
Across the supplied sections, Traffic-R1 is tested on incident and conventional-control cases and described in a real-world deployment with a hybrid dispatch process. The reported results include zero-shot benchmark performance and reductions in queues and operator labor against manual control.
A.3 Traffic Incident Data
- Incident testing used 200 representative textual examples derived from traffic-management emergency recordings to simulate out-of-distribution incidents.Examples include accidents, pedestrian events, emergency vehicles, and other disruptions.Source: §A.3 · p. 13
A.4 More Experiment Result
- Traffic-R1 in zero-shot configuration outperformed all baselines, including extensively trained RL methods, and performed better than DeepSeek-R1-671B at 1% of its parameter size.Source: §A.4 · p. 14
A.5 Reasoning Cases of Traffic-R1
- Reasoning examples describe choices that weigh queued and approaching vehicles and, during incidents, balance traffic efficiency with incident needs.Source: §A.5 · p. 14, §A.4 · p. 14
A.6.1 Environment and System Adaption.
- The deployment covers 10 interconnected intersections serving over 55,000 vehicles daily, with lane aggregation and mandatory peak-hour pedestrian phases adapting the system to arterial roads.Source: §A.6.1 · pp. 14–15
- Roadside-camera detections from a fine-tuned Grounding DINO model are combined with millimeter-wave radar distances to represent traffic state.Source: §A.6.1 · p. 15
- A hybrid dispatch framework uses offline forecasts and expert-approved preliminary timing plans for routine control, reserving online adjustments for incident response under expert supervision.The offline forecast uses 30 days of historical data and a LightGBM model.Source: Figure 8 · p. 14, §A.6.1 · p. 15
A.6.2 In-Situ Performance Evaluation
- 9.3% and 4.4%: Traffic-R1 reduced average and maximum queue lengths, respectively, and cut daily human operational labor by 75% versus manual control.The A/B test alternated the systems weekly and measured weekday evening peak hours.Source: §A.6.2 · p. 15, Table 6 · p. 15