PaperlayerView exact revision on arXiv

Source-linked AI summary

LLMLight: Large Language Models as Traffic Signal Control Agents

Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, Hui Xiong

arXiv:2312.16044v5cs.AI

TL;DR

Traffic signal control methods can struggle to generalize across scenarios and explain their decisions, while direct use of LLMs as traffic-control agents remains underexplored. LLMLight prompts LLM agents with real-time traffic conditions and task knowledge, and LightGPT adds traffic-specific training and policy refinement. Across ten traffic datasets, LLMLight with LightGPT significantly outperforms nine baseline methods and ten leading LLMs, with reported generalization and interpretability benefits.

  • Problem

    Traditional transportation and reinforcement-learning approaches can generalize poorly across varied traffic scenarios and lack interpretable control rationales, while direct LLM decision-making for TSC remains largely unexplored.

    Source: §1 · pp. 1–2

  • Method

    LLMLight prompts LLM agents with verbalized intersection conditions, task descriptions, and commonsense traffic knowledge, while LightGPT adds traffic-specialized fine-tuning and critic-guided policy refinement.

    Source: §1 · p. 2, §3.2.2 · p. 3

  • Results

    LLMLight with LightGPT significantly outperforms nine baseline methods and ten leading LLMs across ten traffic datasets, with reported effectiveness, generalization, and interpretability.

    Source: §6 · p. 9

  • Takeaways & Limitations

    The paper concludes that tailored prompting and domain-specific fine-tuning can improve control policies, and that LLM-based traffic control can provide detailed decision explanations across diverse scenarios.

    Source: §6 · p. 9

    The approach uses only target-intersection traffic features, omits camera imagery and other modalities, and does not model pedestrians or bicycles.

    Source: §6 · p. 9

Abstract

from arXiv · show

Traffic Signal Control (TSC) is a crucial component in urban traffic management, aiming to optimize road network efficiency and reduce congestion. Traditional TSC methods, primarily based on transportation engineering and reinforcement learning (RL), often struggle with generalization abilities across varied traffic scenarios and lack interpretability. This paper presents LLMLight, a novel framework employing Large Language Models (LLMs) as decision-making agents for TSC. Specifically, the framework begins by instructing the LLM with a knowledgeable prompt detailing real-time traffic conditions. Leveraging the advanced generalization capabilities of LLMs, LLMLight engages a reasoning and decision-making process akin to human intuition for effective traffic control. Moreover, we build LightGPT, a specialized backbone LLM tailored for TSC tasks. By learning nuanced traffic patterns and control strategies, LightGPT enhances the LLMLight framework cost-effectively. Extensive experiments conducted on ten real-world and synthetic datasets, along with evaluations by fifteen human experts, demonstrate the exceptional effectiveness, generalization ability, and interpretability of LLMLight with LightGPT, outperforming nine baseline methods and ten advanced LLMs.

1 INTRODUCTION

Traffic signal control can bring economic, environmental, and societal benefits, but complex urban networks remain difficult to manage. LLMLight uses prompted LLM agents for signal decisions and introduces LightGPT, a traffic-specialized model intended to improve control cost-effectively.

1 INTRODUCTION

  • Existing transportation heuristics depend heavily on manual design, while reinforcement-learning methods can generalize poorly to larger networks or uncommon traffic and lack interpretable action rationales.The paper links this opacity to reduced feasibility for real-world deployment.Source: §1 · p. 1
  • Direct LLM control remains underexplored, and LLMs must bridge the gap between real-time traffic observations and linguistic understanding while addressing limited traffic expertise and high usage costs.Generalist models may also hallucinate in professional fields.Source: §1 · p. 2
  • LLMLight verbalizes intersection conditions and combines them with task descriptions and commonsense knowledge in a prompt that guides an LLM’s chain-of-thought selection of the next signal configuration.The framework treats TSC as a partially observable Markov Game.Source: §1 · p. 2
  • LightGPT uses GPT-4-derived control actions and rationales for imitation fine-tuning, then critic-guided policy refinement to produce more effective, generalizable policies cost-effectively.The paper reports experiments on ten datasets and evaluations by fifteen human experts, comparing LLMLight and LightGPT with nine baseline methods and ten advanced generalist LLMs.Source: §1 · p. 2

2 PRELIMINARY

The preliminaries define the directed road-network representation and signal-control actions used in the traffic-control task. They also point to a figure illustrating a commonly used intersection setting.

2 PRELIMINARY

  • A road network is a directed graph of intersection joints and lanes, with lanes categorized by vehicle movement and divided into segments.The lane categories are go-through, left-turn, and right-turn.Source: §2 · p. 2
  • A traffic signal control action selects a signal phase that activates a group of lanes without conflicting movements; other phases show red.At each signal-switching time step, the agent chooses from the available actions.Source: §2 · p. 2
  • Figure 8 provides more detail about the commonly used intersection, lane, and signal-phase setting.Source: §2 · p. 2

3 LLM FOR TRAFFIC SIGNAL CONTROL

This section formulates traffic signal control as an LLM decision task and describes LLMLight’s prompting and action workflow. It then details LightGPT’s three-stage training procedure for adapting a backbone LLM to traffic signal control.

3.1 Task Formulation

  • Traffic signal control is formulated as a partially observable Markov Game in which an LLM agent maps intersection observations and task context to reasoning and a signal action.The stated long-term goal is maximizing transportation efficiency across the road network.Source: §3.1 · p. 3

3.2 Workflow of LLMLight

  • LLMLight collects intersection traffic conditions, builds a commonsense-augmented prompt, then uses the LLM’s reasoning to select and execute a signal action.The workflow consists of observation collection, prompt construction, and reasoning with action execution.Source: §3.2 · p. 3

3.2.1 Observation Feature Construction.

  • The observation represents queuing and approaching vehicle counts on incoming lanes, verbalized as human-readable text for the LLM.Queued vehicles are below a speed threshold; approaching vehicles are above it and counted in a lane segment.Source: §3.2.1 · p. 3

3.2.2 Commonsense Augmented Prompt Construction.

  • The prompt combines real-time observation with scenario, task, and action-space descriptions, plus commonsense guidance to prioritize long queues over distant vehicles.The composed prompt is denoted X.Source: §3.2.2 · p. 3

3.2.3 Reasoning and Action Execution.

  • LLMLight uses zero-shot reasoning to analyze lane conditions, select a signal for the most congested lanes, and provide rationales for its decisions.The authors identify these rationales as contributing to a more explainable and transparent traffic control system.Source: §3.2.3 · p. 3

3.3 Backbone LLM Training

  • LLMLight’s training procedure collects and filters GPT-4 reasoning trajectories, imitation-fine-tunes a backbone LLM on the selected trajectories, then refines its policy with critic feedback.The filtering critic estimates long-term rewards, with negative queue length given as an example reward.Source: §3.3 · p. 3, §3.3.1 · pp. 3–4

3.3.1 Trajectory Collection and Filtering.

  • GPT-4 interacts with simulated traffic environments to generate reasoning trajectories, which are filtered for alignment with long-term traffic-control goals using a pre-trained action-value network.The filtering criterion includes minimizing future queue length; the critic estimates action values from rewards such as negative queue length.Source: §3.3.1 · pp. 3–4

3.3.2 Imitation Fine-tuning.

  • Imitation fine-tuning trains the backbone LLM on filtered GPT-4 reasoning trajectories and their selected actions, using the prompt as instruction and negative log-likelihood as the loss.The training description also specifies parameter-efficient fine-tuning with LoRA to reduce trainable parameters and computational and storage costs.Source: §3.3.2 · p. 4

4 EXPERIMENTS

Across Jinan and Hangzhou, LLMLight with LightGPT achieves state-of-the-art or comparable performance against baselines; experiments also examine ablations, transfer, scalability, extreme traffic, interpretability, and deployment. Results report strong performance across these settings, including expert approval of the tested scenarios.

4.1.1 Datasets.

  • Experiments cover seven real-world and three synthetic datasets, evaluate travel time, queue length, and waiting time, and compare transportation, RL, and LLM-based methods.The datasets include New York, extreme high-traffic scenarios, and a 24-hour cycle; results also use uniform RL hyperparameters and standardized LLM settings.Source: §4.1.1 · p. 5, §4.2 · p. 5, §4.3 · p. 5

4.5 Comparison Between the Traditional Methods and LLMLight (RQ1)

  • LLMLight with LightGPT achieves state-of-the-art or comparable performance against traditional and RL baselines on five Jinan and Hangzhou datasets.The comparison is reported in Table 2.Source: Table 2 · p. 6, §4.5 · p. 6

4.7.1 Transferability.

  • After transfer to a distinct road network, RL methods decline notably, while LLMLight consistently achieves superior performance across datasets.Figure 4 marks transferred models with “-T”; other models are trained and tested on the same dataset.Source: Figure 4 · p. 7, §4.7.1 · p. 7

4.6 Abalation Study

  • Each LightGPT training stage progressively improves control policies, and critic-guided refinement yields state-of-the-art or competitive performance across datasets, significantly surpassing GPT-4.The ablation compares original models, imitation fine-tuning variants, and imitation fine-tuning plus critic-guided policy refinement.Source: §4.6 · p. 7

4.7.2 Scalability.

  • On the larger New York network, LLMLight achieves the shortest travel and waiting times; the best RL methods’ waiting times are 57.80% and 49.80% longer.The compared models were pretrained on Jinan 1.Source: §4.7.2 · p. 7, Figure 5 · p. 7

4.7.3 Extreme High-traffic Scenarios.

  • Under approximately fourfold higher vehicle arrivals per five minutes, RL methods degrade and perform worse than Maxpressure, while LLMLight remains superior.The extreme-traffic datasets are synthetic and based on Jinan and Hangzhou; models were pretrained on regular traffic from the same road network.Source: §4.7.3 · pp. 7–8, Figure 6 · p. 8

4.8 Interpretability of LLMLight (RQ3)

  • In a Hangzhou scenario, LightGPT selects NLSL after identifying the greatest congestion among its allowed lanes and provides detailed explanations for its control decisions.Figure 7 illustrates the decision process; the authors contrast LLMLight’s explanations with RL methods’ lack of them.Source: §4.8 · p. 8, Figure 7 · p. 8

4.9 Deployment Analysis

  • In evaluations across examined traffic scenarios, all participants—four traffic police officers, nine experienced drivers, and two AI specialists—acknowledged LLMLight’s reasoning and decisions, praising its real-world applicability and interpretability.Appendix A.5 reports a 24-hour robustness experiment under daily long-term traffic fluctuations, and Appendix A.4 details a cost-effectiveness analysis comparing LightGPT with GPT-4.Source: §4.9 · p. 8

5 RELATED WORK

Related work spans conventional and RL-based traffic signal control, broader intelligent-transportation forecasting and control, and emerging uses of LLMs as planners for control tasks.

5.1 Traffic Signal Control

  • Transportation methods use fixed-time cycles or pressure-based phase selection, while PressLight combines Maxpressure with RL and Advanced-MP develops richer traffic-state features.The examples include FixedTime, Maxpressure, PressLight, and Advanced-MP.Source: §5.1 · p. 8

5.2 Intelligent Transportation

  • Intelligent-transportation research emphasizes forecasting and control, with examples spanning personalized vehicle-energy prediction and dynamic EV charging prices.The examples are meta-learning for personalized energy predictions and multi-agent graph-convolutional RL for EV pricing.Source: §5.2 · p. 8

5.3 Large Language Models for Decision Making

  • LLMs are increasingly used as central planners in control tasks, including urban-domain benchmarking and end-to-end autonomous-driving systems.Examples include CityBench [18] and DriveGPT4 [55].Source: §5.3 · p. 8

6 CONCLUSION AND LIMITATIONS

The paper introduces LLMLight and LightGPT for traffic signal control, reports performance across diverse scenarios, and identifies limits in the framework’s scope.

6 CONCLUSION AND LIMITATIONS

  • Across ten traffic datasets, LLMLight with LightGPT significantly outperforms nine baseline methods and ten leading LLMs.The paper reports effectiveness, generalization, and interoperability across diverse traffic scenarios.Source: §6 · p. 9
  • Traditional TSC methods have limited generalization in unseen scenarios, while tailored prompting and domain-specific fine-tuning can improve generalist LLM control policies.LLM-empowered TSC also provides detailed explanations for decisions, supporting a more comprehensible and accountable system.Source: §6 · p. 9
  • The framework uses target-intersection traffic features without multi-agent cooperation, and omits camera inputs and pedestrian and bicycle effects.Its focus is primarily vehicle traffic flow.Source: §6 · p. 9

A.1 Settings of Intersection, Lanes, and Signal Phases

Figure 8 presents the most-used setting of the intersection, lanes, and signal phases.

A.1 Settings of Intersection, Lanes, and Signal Phases

  • Figure 8 presents the most-used setting of the intersection, lanes, and signal phases.Source: §A.1 · p. 11

A.2.1 Transportation Methods.

The transportation baselines include random phase switching, fixed-time cycles, and pressure-based phase selection.

A.2.1 Transportation Methods.

  • Random switches signal phases with a fixed duration.Source: §A.2.1 · p. 11
  • FixedTime uses a predefined cycle length and phase time and is widely implemented in most traffic scenarios.Source: §A.2.1 · p. 11
  • Maxpressure greedily relieves lanes with the highest pressure, derived from upstream and downstream queue-length differences.Source: §A.2.1 · p. 11

A.2.2 RL-based Methods.

The section surveys RL-based traffic-signal-control methods, which use pressure, attention, or neighboring-intersection information to guide control.

A.2.2 RL-based Methods.

  • MPLight uses pressure as both its observation and reward, with FRAP designed to handle unbalanced traffic flow.FRAP is its foundational model.Source: §A.2.2 · p. 11
  • AttendLight uses attention to construct observation features and predict phase-transition probabilities.Source: §A.2.2 · p. 11
  • PressLight applies deep reinforcement learning to optimize intersection pressure using the MaxPressure concept.Source: §A.2.2 · p. 11
  • CoLight uses a graph attention network to enable communication among neighboring intersections for coordinated decisions.Source: §A.2.2 · p. 11
  • Efficient-CoLight builds on CoLight by adding efficient pressure as an observation to optimize decisions.Source: §A.2.2 · p. 11
  • Advanced-CoLight builds on CoLight with advanced traffic-state features and is described as state of the art for TSC tasks.Source: §A.2.2 · p. 11

A.2.3 Large Language Models.

The appendix reports additional deployment, evaluation, and comparison results for LLMLight and LightGPT. It also describes supporting system materials and practical deployment challenges.

A.2.3 Large Language Models.

  • The appendix catalogs tested LLM families, explains lane-color and signal-phase mappings, and provides prototype, road-network, and prompt-template materials.The prototype displays current traffic conditions, vehicle statistics, and LightGPT’s reasoning.Source: §A.2.3 · p. 11, Figure 8 · p. 11, §A.3 · p. 11

A.4 Cost-effectiveness of LightGPT

  • LightGPT is reported to combine the best effectiveness and real-time control with the lowest deployment cost; one model can asynchronously serve up to 100 intersections.LightGPT (Llama3-8B) serves up to 100 intersections asynchronously; LightGPT (Llama2-13B) serves up to 50.Source: §A.4 · pp. 11–12, Table 4 · p. 12

A.5 24-hour Cycle Experiment

  • Across a full day with morning and evening rush hours, LLMLight consistently achieves the shortest travel and waiting times, while RL methods fare worse than MaxPressure under changing flows.The authors also report that LLMLight significantly outperforms FixedTime in real-world scenarios.Source: Figure 10 · p. 12, §A.5 · p. 12, Figure 11 · p. 12

A.6 Human Expert Evaluation

  • Fifteen evaluators—four traffic police officers, nine experienced drivers, and two AI specialists—gave overwhelmingly positive feedback on applicability and interpretability.The evaluation covered five randomly selected traffic scenarios.Source: §A.6 · p. 12, Figure 12 · p. 14, Figure 13 · p. 14

A.7.1 Challenges in Real-world Deployment.

  • Deployment may require camera or sensing-coil integration changes, regional regulatory adjustments, and attention to driver behavior; captured images are kept on traffic-department servers.The system extracts vehicle counts as nonpersonal data, and captured images are not made public.Source: §A.7.1 · p. 12, §A.7.2 · p. 13

A.8 Failure of Other LLMs

  • Llama-2 models may ignore commonsense guidance and prioritize approaching vehicles over queues, while ChatGPT-3.5’s reasoning includes a claim the authors identify as hallucination.The analyzed case has severe north–south congestion, left-turn queues, and heavy go-through inflow.Source: §A.8 · p. 13, Figure 17 · p. 16

A.9 Experiment on Policy Network Learning by Using the Actor-Critic Paradigm

  • PPO with an MLP value-function head and negative queue-length reward struggled to converge, so the authors adopted critic-guided policy refinement for supervised alignment.The comparison is reported for the Jinan 1 and Hangzhou 1 datasets.Source: §A.9 · p. 13, Table 9 · p. 13

A.10 Comparison with the SOTA RL-based Method Under a Fair Setting

  • On Hangzhou, LLMLight outperforms Advanced-CoLight without cooperation on ATT, AQL, and AWT, using only target-intersection observations.The authors note Advanced-CoLight’s marginal advantage on Hangzhou datasets when cooperation is included.Source: §A.10 · p. 13, Table 10 · p. 13

Related papers

Loading 2312.16044v5…