PaperlayerView exact revision on arXiv

Source-linked AI summary

Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions

Doaa Mahmud, Hadeel Hajmohamed, Shamma Almentheri, Shamma Alqaydi, Lameya Aldhaheri, Ruhul Amin Khalil, Nasir Saeed

arXiv:2501.04437v1eess.SYcs.AIcs.ET

TL;DR

ITS needs efficient, adaptive, and responsive ways to manage complex transportation networks. This paper reviews how LLMs support traffic management, prediction, communication, and related ITS applications, while examining deployment challenges. It concludes that progress in data handling, computational resources, and ethical safeguards is needed to realize their potential.

  • Problem

    LLM-assisted ITS must handle vast, dynamic, and incomplete traffic data, substantial real-time computational demands, and ethical risks involving personal data and bias.

  • Method

    The paper comprehensively reviews LLM applications, model integrations, traffic-control systems, multimodal transportation, deployment methods, and future research directions for ITS.

  • Results

    The review identifies improvements in traffic prediction, signal optimization, real-time communication, traffic-control decision-making, and multimodal transportation through LLM-assisted ITS.

  • Takeaways & Limitations

    Integrating LLMs with edge computing, IoT, quantum computing, and beyond-5G networks is presented as a path toward more intelligent, adaptive, and resilient transportation systems.

  • Takeaways & Limitations

    LLM-assisted ITS remains limited by dynamic data management, real-time computational requirements, and secure handling of personal data.

Abstract

from arXiv · show

Intelligent Transportation Systems (ITS) are crucial for the development and operation of smart cities, addressing key challenges in efficiency, productivity, and environmental sustainability. This paper comprehensively reviews the transformative potential of Large Language Models (LLMs) in optimizing ITS. Initially, we provide an extensive overview of ITS, highlighting its components, operational principles, and overall effectiveness. We then delve into the theoretical background of various LLM techniques, such as GPT, T5, CTRL, and BERT, elucidating their relevance to ITS applications. Following this, we examine the wide-ranging applications of LLMs within ITS, including traffic flow prediction, vehicle detection and classification, autonomous driving, traffic sign recognition, and pedestrian detection. Our analysis reveals how these advanced models can significantly enhance traffic management and safety. Finally, we explore the challenges and limitations LLMs face in ITS, such as data availability, computational constraints, and ethical considerations. We also present several future research directions and potential innovations to address these challenges. This paper aims to guide researchers and practitioners through the complexities and opportunities of integrating LLMs in ITS, offering a roadmap to create more efficient, sustainable, and responsive next-generation transportation systems.

I. INTRODUCTION

The introduction presents ITS as a means to improve transportation efficiency, safety, and sustainability, then reviews how LLMs extend these capabilities through interaction, prediction, communication, and situational awareness. It also situates the paper among related AI, generative-AI, and LLM-focused surveys while identifying deployment concerns.

  • Motivation: ITS targets transportation efficiency, safety, and sustainability, while market projections indicate expanding interest in both ITS and LLM technologies.The Middle East ITS market was valued at USD 2.82 billion in 2017, while the U.S. LLM market is projected to rise from $50 million in 2020 to $1.4 billion by 2030.
  • ITS Background: Traditional ITS uses traffic signals, adaptive management, variable message signs, ramp metering, sensors, and cameras to manage traffic flow and reduce accidents.These approaches often rely on predetermined algorithms.
  • ITS Background: Deep learning adds predictive and adaptive capabilities, including traffic prediction, incident detection, and advanced driver assistance through CNN- and RNN-based analysis.CNNs process traffic-camera images, whereas RNNs predict traffic patterns from historical data.
  • LLM Opportunities: LLMs extend ITS through natural-language interaction, predictive analytics, traffic forecasting, emergency communication, multimodal planning, and improved situational awareness.Applications include traffic prediction, social-media sentiment analysis, emergency response, disaster management, and multimodal transportation planning.
  • LLM Opportunities: LLMs can integrate traffic, GPS, social-media, weather, and sensor information to adjust signals, personalize travel guidance, support predictive maintenance, and coordinate autonomous vehicles.The described applications aim to improve traffic management, safety, reliability, and sustainability.
  • Related Work: Prior surveys cover AI for road safety, vehicle and traffic control, generative AI for perception and decision-making, explainable AI, and LLM risks such as bias, privacy, and hallucination.Reviews of vision-language models additionally emphasize data scale and quality, real-time processing, hardware support, and safety alignment.

B. Contributions of this paper

The paper contributes a broad review of ITS foundations, LLM techniques, applications, challenges, and future directions. It is designed to connect model fundamentals with transportation use cases and responsible deployment needs.

  • ITS Foundations: The paper reviews ITS components, operational principles, and overall effectiveness as a foundation for examining LLM integration.This overview establishes the transportation-system context for later discussions.
  • LLM Foundations: It explains the theoretical background of GPT, T5, CTRL, BERT, and other LLM techniques within ITS.The discussion is intended to help readers understand how these models can support ITS applications.
  • ITS Applications: The paper analyzes LLM applications in traffic-flow prediction, vehicle detection and classification, autonomous driving, traffic-sign recognition, and pedestrian detection.The stated objective is to examine how these applications address ITS challenges affecting traffic management and safety.
  • Challenges: It examines challenges involving data availability, computational constraints, and ethical considerations, alongside approaches for responsible and effective implementation.The paper also discusses future research directions for overcoming these challenges.
  • Future Directions: Future directions include improved data integration, stronger real-time processing, integration with emerging technologies, and ethical frameworks addressing privacy concerns.These directions are presented as pathways toward more efficient, sustainable, and responsive transportation systems.
  • Scope: The paper reviews centralized and decentralized LLMs and their relevance to traffic prediction, signal optimization, route planning, autonomous vehicles, public transport, V2X, ADAS, and traffic-control centers.The model background includes transformer-based LLMs, pre-training, fine-tuning, and in-context learning.

A. Generative Pre-Trained Transformer (GPT)

GPT is presented as a pretrained, transformer-based language model that can be adapted to diverse tasks and transportation data. The section also contrasts GPT’s generative approach with T5’s unified text-to-text formulation.

  • GPT Fundamentals: GPT is pretrained on large text datasets and fine-tuned for language generation, sentiment analysis, language modeling, translation, and text categorization.Its transformer architecture advances beyond earlier RNN and CNN approaches.
  • GPT Fundamentals: GPT uses self-attention to incorporate sentence-wide context when generating the next word, supporting natural-language understanding and production.The decoder generates output from the input representation.
  • GPT Variants: The GPT family includes GPT-1, GPT-2, GPT-3, InstructGPT, ChatGPT, GPT-4, CODEX, and WebGPT, with later models including closed-source API-accessed systems.GPT-1 established decoder-only generative pre-training on unlabeled text.
  • GPT-3: 175 billion parameters distinguish GPT-3, whose pretraining supports in-context learning without gradient updates or task-specific fine-tuning.GPT-3 is described as handling translation, question answering, cloze tests, reasoning, and domain adaptation through textual interaction.
  • Transportation Applications: GPT can analyze weather reports, traffic updates, and user feedback to derive transportation insights and recommend routes for commuters and logistics operations.The section presents scenario simulation as another transportation-planning use.
  • T5 Comparison: T5 uses an encoder-decoder architecture that converts tasks such as question answering, summarization, classification, and translation into a unified text-to-text format.This allows shared losses, hyperparameters, and training protocols across tasks.
  • T5 Comparison: T5’s generative formulation supports domain-specific models such as SciFive and has been used to derive pretrained representations for network traffic from unlabeled data.The cited network-traffic model preserves generative capabilities while capturing global information.

C. BERT

BERT uses bidirectional Transformer self-attention to represent contextual word interactions and dependencies. The surrounding model discussion contrasts open-source LLaMA-2 and scalable Falcon architectures relevant to ITS.

  • C. BERT: BERT captures short- and long-term dependencies through bidirectional self-attention over input sequences.It provides context-aware representations of words or word fragments based on their positions.
  • C. BERT: BERT pretraining combines masked language modeling and next-sentence prediction on large unlabeled text corpora.Pretraining learns broad language representations and supports convergence and generalization.
  • D. LLaMA: LLaMA-2 Chat is trained through pretraining, supervised fine-tuning, and iterative reinforcement learning from human feedback.The paper connects these dialogue-optimized models to more responsive human-machine interaction in ITS.
  • E. Falcon LLM: Falcon models span 7B, 40B, and 180B parameters and are trained primarily on RefinedWeb corpora.Falcon-180B used more than 3.5 trillion text tokens, while the family targets hardware, data, and performance scalability.
  • E. Falcon LLM: Falcon’s multi-query attention supports efficient large-task processing, with Falcon 40B also offering multilingual capabilities.The model supports English, German, Spanish, and French, plus rudimentary knowledge of several additional languages.

III. DECENTRALIZED LLMS

Decentralized LLMs distribute computation, communication, and model updates across networked devices rather than relying on centralized infrastructure. This can improve privacy and scalability in ITS, but complexity and communication overhead remain central constraints.

  • Architecture and rationale: Decentralized LLMs distribute computation and communication across nodes, reducing reliance on centralized servers.The approach is presented as relevant to distributed ITS environments and learning tasks.
  • ITS integration: Model updates can be shared securely across devices while keeping data localized, improving privacy, security, and deployment scalability.Connected vehicles and devices can also contribute spare computing resources to the network.
  • ITS integration: Decentralized LLMs can support traffic control, safety, and flow management across smart-city hubs, vehicles, roadside units, and traffic centers.They can analyze localized vehicle and infrastructure data and disseminate traffic conditions and alerts without central databases.
  • Computational complexity: Data parallelism has temporal complexity O(n/p), but communication and synchronization overhead can offset distributed-computation gains.Complexity depends on model size, node count, communication overhead, and synchronization requirements.
  • Computational complexity: Federated Averaging complexity is approximated as O(kn), depending on dataset size and the number of updates.The large dataset-size term makes computational and communication costs important for federated decentralized LLMs.
  • ITS applications: Distributed GPT-NeoX can process massive real-time datasets for traffic prediction, signal adjustment, rerouting, and accident-risk analysis.Its distributed training and large-scale processing capabilities are framed as useful for transportation decision-making.

B. OpenFlamingo

Open-source and distributed model frameworks offer different ways to scale multimodal processing, training, and deployment for ITS. Their potential spans flexible vision-language interaction, multilingual adaptation, distributed tensor computation, and collaborative fine-tuning.

  • OpenFlamingo: OpenFlamingo is an open-source decentralized multimodal model that combines image and text inputs for vision-language tasks.Its intended tasks include image classification and visual question answering.
  • OpenFlamingo: OpenFlamingo accepts multiple images and text inputs in arbitrary order, supports few-shot and multi-round interaction, and generalizes across tasks.These properties distinguish it from several open-source models that typically process one image at a time.
  • BLOOM: BLOOM provides an open-access multilingual model with 176 billion parameters and generation in 46 natural and 13 programming languages.Its open resources and responsible-use framework are presented as useful for public-sector ITS experimentation.
  • Distributed training frameworks: Colossal-AI scales large-model training across multiple GPUs and nodes to address substantial computational-resource demands.The framework is positioned as particularly beneficial where ITS models require distributed computing.
  • Distributed training frameworks: Mesh TensorFlow distributes tensor computations across processor meshes, improving memory and computational efficiency for large-model training.Model parallelism avoids replicating entire models on every device and supports TPUs, GPUs, and CPUs.
  • Collaborative deployment: Petals enables collaborative inference, fine-tuning, and parameter-efficient adaptation of large models over the internet.Users can host model components and share fine-tuned adapters or prompts without centralized control.

IV. LLMS IN ITS

LLMs are applied across ITS tasks including traffic forecasting, signal optimization, and multimodal decision support. The reviewed approaches combine contextual data processing with tool-assisted or vision-language reasoning to improve responsiveness and management.

  • A. Traffic Prediction and Forecasting: Traffic forecasting supports travel advice and transportation management, but traditional methods may miss nonlinear spatial-temporal traffic structure.RNNs and LSTMs model temporal dependencies, while GCNs model geographical dependencies.
  • A. Traffic Prediction and Forecasting: LLMs combine historical traffic, current updates, events, and weather to produce short- and long-term forecasts.Short-term forecasts can support dynamic routing and traffic-signal timing in real time.
  • A. Traffic Prediction and Forecasting: TrafficBERT uses transformer self-attention and large-scale traffic data to predict flow across varied road conditions without road-specific or meteorological data.The passage states that it outperforms conventional statistics and deep-learning models.
  • A. Traffic Prediction and Forecasting: Multimodal LLM systems embed spatial-temporal traffic data and combine visual, linguistic, and probabilistic reasoning for forecasting and accident-response applications.Examples include partially frozen attention LLMs with fusion convolutions and LLaVA-based autonomous-driving systems.
  • B. Traffic Signal Optimization: Traffic signal optimization uses traffic volumes and patterns to coordinate signal timing, reducing congestion and commuter travel times.The section frames optimized timing as a means of improving intersection movement efficiency.
  • B. Traffic Signal Optimization: LA-Light combines conventional traffic-management techniques with LLM-based cognition and perception tools for hybrid signal-control decisions.The LLM selects tools, evaluates collected environmental information, and recommends traffic-signal timing after sufficient data gathering.

C. Route Planning and Navigation

LLMs support route planning and navigation by interpreting real-time traffic, maps, and user instructions to make higher-level decisions. Their use extends to autonomous-vehicle planning and public-transport scheduling, while environmental perception remains a limitation for autonomous driving.

  • Route Planning: LLMs analyze real-time traffic information, traffic reports, and maps to support high-level route-planning decisions.
  • Route Planning: Chain-of-Thought prompting supports multi-step reasoning and decision-making for navigation tasks such as route planning.
  • Autonomous Vehicles: Hybrid LLM and rule-based planners address complex self-driving scenarios, while NavGPT performs zero-shot sequential action prediction for vision-and-language navigation.
  • Autonomous Vehicles: LLMs lack innate environmental perception and may therefore produce suboptimal or hazardous autonomous-driving decisions without sensory modules supplying live data.
  • Public Transportation: LLMs can analyze traffic patterns and forecast congestion in real time, enabling proactive adjustments to public-transport routes and schedules.

F. LLMs-assisted Vehicle-to-Everything Communication

LLMs extend vehicle-to-everything communication by interpreting complex messages and producing context-aware responses for vehicles, infrastructure, pedestrians, and networks. Across connected transportation systems, they also support traffic control, ADAS interactions, smart-city data processing, and pedestrian-flow applications, subject to data, bias, privacy, and safety constraints.

  • V2X Communication: V2X connects vehicles with other vehicles, infrastructure, pedestrians, and networks for real-time data exchange that supports traffic management and safety.
  • V2X Communication: LLMs interpret road signs, traffic signals, vehicle messages, and infrastructure alerts, improving context-aware responses, hazard detection, and route guidance.
  • V2X Communication: LLMs process large volumes of real-time data from connected vehicles, infrastructure, and road users to enhance situational awareness in autonomous vehicles.
  • Broader ITS Applications: LLM integration across ITS applications supports driver assistance, in-car voice interaction, smart-city data analysis, and pedestrian-trajectory prediction.
  • Traffic Control Centers: LLM-based traffic-control systems use real-time and historical data for adaptive signal timing and proactive congestion management.
  • Traffic Control Centers: Traffic-control applications remain constrained by high-quality data requirements, model bias, and data-privacy concerns affecting reliability, safety, and fairness.

K. Multi-model Transportation

LLM-assisted ITS is presented as a way to coordinate complex multimodal transportation, traffic control, and pedestrian-management tasks. The reviewed applications emphasize improved efficiency, connectivity, usability, and adaptive responses, while real-world deployment remains limited.

  • Multimodal transportation: LLM-assisted multimodal transportation can improve network efficiency, dependability, and usability by optimizing routes, traffic management, and intermodal connectivity.The approach targets reduced congestion, shorter travel times, and smoother transitions among bicycles, buses, trains, and ridesharing services.
  • Deployment boundary: Most LLM-based ITS prototypes remain conceptual or at the research stage, with few documented real-world implementations.This limits the evidence currently available for assessing deployment at operational scale.
  • Traffic forecasting: Traffic forecasting models address dynamic spatial and temporal dependencies, with STPT outperforming competing models by 10-34% across RMSE, MAE, and MAPE.STPT uses coupled graph transformer-convolution units and multiple adjacency graphs to generate noise-resistant embeddings.
  • Pedestrian and smart-city applications: LLM-assisted ITS is also linked to pedestrian-flow management, smart-city services, and improved autonomous-system performance.Pedestrian movement modeling supports robot planning, behavioral analysis, and self-driving applications.
  • Multimodal transportation: Multimodal route models must address increasing network complexity and generalized path overlap across shared modes and physical links.The MLK model separates route-specific and link-specific utilities to better represent traveler behavior and route choices.

D. TransTTE

TransTTE applies transformer-based spatiotemporal modeling to travel-time estimation in complex urban road networks. Its goal is to estimate travel duration from route and departure information while addressing network constraints and traffic variability.

  • Model design: TransTTE uses transformers and spatiotemporal techniques to estimate travel time in complex urban road networks.The model addresses the constraints imposed by road-network structure and the unpredictable dynamics of urban traffic.
  • Purpose: Accurate travel-time estimation is treated as important for managing unpredictable urban traffic dynamics.The measure helps describe traffic flow and supports transportation management.
  • Limitations: The section identifies high-quality data requirements, model bias, and data privacy as key limitations for transportation applications.These constraints affect the reliability and responsible use of the model in ITS settings.
  • Model design: Given an origin, destination, and departure time, TransTTE estimates the associated travel duration from historical traffic and road-network data.Its self-attention mechanism focuses on relevant spatial features while encoding network centrality.

E. BERT4ITS

BERT4ITS is introduced to address the difficulty of analyzing heterogeneous transportation Big Data containing textual and numerical information. The section also places this approach within broader constraints involving data quality, computation, privacy, and real-time processing.

  • BERT4ITS: BERT4ITS applies the BERT model to transportation Big Data that combines textual and numerical information.The framework is motivated by the difficulty of interpreting diverse traffic flows, sensor data, and real-time incidents.
  • Data availability: LLM-based ITS requires complex, dynamic, and often incomplete traffic data to capture traffic patterns and behaviors accurately.Relevant data come from diverse sources, including sensors, cameras, and mobile devices.
  • Challenges: LLM deployment in ITS is constrained by substantial computational requirements and ethical risks involving sensitive data and unintended bias.The paper identifies scalability, privacy, and fairness as central concerns for real-time transportation use.
  • Advanced data processing: Advanced sensors, connected vehicles, computer vision, and Floating Car Data are proposed to improve real-time traffic-data collection and management.FCD uses GPS or mobile phones to monitor vehicle locations, speeds, and directions.
  • Advanced data processing: Distributed and edge-computing architectures could process transportation data closer to its sources, reducing reliance on continuous centralized transmission.The proposed direction targets more responsive real-time analysis.

B. Computational Constraints and Adaptive LLMs

The paper identifies bias, computational cost, privacy, security, and employment impacts as constraints on deploying LLMs in ITS. It discusses mitigation through bias evaluation, model-efficiency techniques, data governance, security measures, and policy responses.

  • Bias: Bias in LLM-based transportation systems can affect safety studies and decision-making, with potentially unsafe consequences for minority groups.FACTS and the Background Comparison Metric are cited as approaches for identifying problematic data slices and evaluating bias.
  • Computational constraints: LLMs require substantial computation for training and real-time inference because of their large parameter counts and complex architectures.Pruning, quantization, knowledge distillation, efficient architectures, and hardware accelerators are proposed to reduce deployment cost.
  • Privacy: LLM-assisted ITS raises privacy risks because transportation data can reveal vehicle locations, driving patterns, queries, and travel identities.Re-identification may remain possible even after anonymization or aggregation, particularly through contextual clues and external data.
  • Privacy: Data governance measures include minimization, anonymization, privacy-impact assessment, secure handling, differential privacy, and synthetic data generation.The paper links these measures to protecting users and supporting public trust in LLM-based traffic systems.
  • Security: LLM systems face security threats including data extraction, gradient leaking, model theft, and black-box model replication.The paper describes extraction of training-set information and model replication from prediction access as concrete risks.
  • Employment and public policy: AI adoption in ITS may displace routine or manual jobs while increasing demand for new skills and regulatory oversight.The paper cites an estimate that nearly 40% of jobs worldwide may be affected by AI and calls for retraining and inclusive policy responses.

E. Integration with Existing Systems and Emerging Technologies

Integrating LLMs into ITS requires addressing latency, bandwidth, scalability, and computational constraints across cloud, edge, and heterogeneous network architectures. Emerging technologies and distributed approaches offer pathways toward more responsive, scalable, and privacy-aware transportation systems.

  • Integration challenges: Cloud-dependent LLMs can introduce latency and bandwidth issues when integrated with sensor-rich ITS environments and multimodal data streams.Real-time ITS applications require rapid processing, while cloud computing and multimodal inputs increase network and computational demands.
  • Integration challenges: Real-time inference at the edge remains constrained by the substantial computational and memory requirements of LLMs.Models may contain hundreds of layers and millions of parameters, although lightweight models such as MobileBERT aim to reduce these demands.
  • Network constraints: High latency, limited bandwidth, and unreliable wireless coverage can delay data transmission and impair LLM-based traffic optimization.These constraints are especially consequential in high-traffic areas and rural regions with network dead zones.
  • Network constraints: Improving wireless infrastructure, including resilient 5G and alternative technologies, is presented as necessary for accurate traffic management in challenging conditions.The paper links network reliability, latency, bandwidth, and real-time adaptability to effective LLM-based decision-making.
  • Scalability: LLMs must operate efficiently across cellular, DSRC, and other network architectures without compromising scalability or real-time performance.Centralized cellular architectures can introduce latency, motivating advances in both network technology and machine-learning design.
  • Scalability: Model compression through pruning, quantization, and knowledge distillation can reduce LLM demands and support deployment on resource-limited edge devices.The paper identifies ITS-specific neural architectures and edge deployment as directions for reducing latency and improving real-time traffic management.

H. LLMs for Edge Computing-assisted ITS

Edge computing can move LLM-based ITS processing closer to vehicles, infrastructure, and users, supporting faster, more resilient, and more personalized traffic decisions. The paper also identifies 6G and quantum computing as future directions for expanding connectivity and computational capability, while reviewing broader LLM applications and remaining research needs.

  • Edge-assisted ITS: Edge-deployed LLMs can process traffic data near its source, enabling low-latency congestion prediction, guidance, and adaptive traffic optimization.Processing on vehicles or roadside infrastructure can help transportation systems respond more quickly to changing conditions.
  • Edge-assisted ITS: Edge computing can preserve critical traffic optimization functions during network disruptions or intermittent connectivity.Reduced reliance on stable, high-bandwidth connections supports continued operation in areas with limited communication infrastructure.
  • Edge-assisted ITS: Combining LLMs with edge computing can produce more personalized and context-aware traffic guidance for individual drivers and communities.Local processing allows systems to account for user needs, preferences, and community-specific contexts.
  • Emerging technologies: 6G could improve LLM responsiveness through ultra-reliable low-latency communication, massive connectivity, and high data rates.These capabilities are discussed for real-time traffic management, congestion prediction, routing optimization, and vehicle-infrastructure communication.
  • Emerging technologies: Quantum computing could accelerate complex transportation optimization, traffic prediction, rerouting, and signal-adjustment decisions.The paper presents quantum computing as a potential source of additional computational power for LLM-based ITS.
  • Scope and future directions: The review concludes that LLM-assisted ITS offers potential across traffic prediction, signal optimization, communication, public transportation, and V2X applications while retaining data, computational, and ethical challenges.Future work emphasizes edge computing, IoT, quantum computing, and beyond-5G networks alongside secure data governance.
Loading 2501.04437v1…