PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Explainable Traffic Flow Prediction with Large Language Models

Xusen Guo, Qiming Zhang, Junyue Jiang, Mingxing Peng, Meixin Zhu, Hao, Yang

arXiv:2404.02937v5cs.LGcs.AI

TL;DR

Traffic forecasting must capture complex, changing conditions, while deep-learning architectures can be difficult to interpret. xTP-LLM turns multimodal traffic data into language prompts for LLM-based prediction; it reports competitive performance with explanations, though nuanced flow fluctuations remain challenging.

  • Problem

    Traffic forecasting must model nonlinear, spatial-temporal variation, while deep-learning architectures require intricate designs and provide limited insight into input-to-prediction mappings.

    Source: INTRODUCTION · p. 3

  • Method

    xTP-LLM converts multimodal traffic data and external factors into structured prompts and fine-tunes an LLM for traffic prediction and explanations.

    Source: INTRODUCTION · p. 3, Problem Description · p. 6

  • Results

    xTP-LLM achieves competitive performance compared with state-of-the-art models while offering explanations for its predictions.

    Source: CONCLUSION AND FUTURE WORK · p. 24

  • Takeaways & Limitations

    The language-based framework offers a transparent and adaptable approach for urban prediction tasks relevant to transportation planning and management.

    Source: CONCLUSION AND FUTURE WORK · p. 24

    xTP-LLM can miss nuanced traffic-flow fluctuations, including changes during the cited afternoon and evening periods.

    Source: Result-explainable studies · p. 21

Abstract

from arXiv · show

Traffic forecasting is crucial for intelligent transportation systems. It has experienced significant advancements thanks to the power of deep learning in capturing latent patterns of traffic data. However, recent deep-learning architectures require intricate model designs and lack an intuitive understanding of the mapping from input data to predicted results. Achieving both accuracy and explainability in traffic prediction models remains a challenge due to the complexity of traffic data and the inherent opacity of deep learning models. To tackle these challenges, we propose a Traffic flow Prediction model based on Large Language Models (LLMs) to generate explainable traffic predictions, named xTP-LLM. By transferring multi-modal traffic data into natural language descriptions, xTP-LLM captures complex time-series patterns and external factors from comprehensive traffic data. The LLM framework is fine-tuned using language-based instructions to align with spatial-temporal traffic flow data. Empirically, xTP-LLM shows competitive accuracy compared with deep learning baselines, while providing an intuitive and reliable explanation for predictions. This paper contributes to advancing explainable traffic prediction models and lays a foundation for future exploration of LLM applications in transportation. To the best of our knowledge, this is the first study to use LLM for explainable prediction of traffic flows.

INTRODUCTION

Traffic forecasting must capture complex, changing conditions, yet deep-learning models often rely on hand-crafted architectures and provide limited interpretability. xTP-LLM uses multimodal traffic descriptions with an LLM to pursue accurate, explainable predictions.

INTRODUCTION

  • Traffic predictions must account for nonlinear dynamics, spatial-temporal variation, and changing factors such as accidents and weather to support informed decisions.Traffic network prediction supports applications including route planning, traffic management, and intelligent transportation systems (1).Source: INTRODUCTION · p. 3
  • Deep-learning traffic models require specifically designed network structures, while abstract representations can obscure generalization and leave predictions difficult to explain.Prior approaches to reliability and robustness also face difficulty providing credentials for predictions where real-world tasks lack labeled data [14–16].Source: INTRODUCTION · p. 3
  • LLMs can describe multimodal urban data and generate explanations, but formulating spatial-temporal tasks for accurate LLM prediction remains challenging.The paper contrasts these capabilities with established deep-learning spatial-temporal formats in Figure 1.Source: INTRODUCTION · p. 3, Figure 1 · p. 4
  • xTP-LLM converts traffic data and external factors into structured prompts, and the paper reports competitive accuracy, generalization, and prediction explanations.The inputs include points of interest, weather, dates, and holidays; the contributions also introduce the CATraffic text dataset and report generalization without additional training.Source: INTRODUCTION · pp. 3–4

Spatial-temporal prediction

The paper situates xTP-LLM among spatial-temporal prediction and explainability methods, then frames traffic forecasting as language modeling. Its prompts combine spatial, historical, and external information with task instructions to guide predictions and explanations.

Spatial-temporal prediction

  • Spatial-temporal deep-learning models use CNNs, RNNs, GNNs, and Transformers to capture spatial or temporal structure, but may fall short in explainability and generalization.The reviewed approaches also combine architectures to model different dependencies among spatial-temporal features.Source: Spatial-temporal prediction · p. 5

Explainable prediction

  • Explainability methods examine influential features, causal graphs, or counterfactuals, while LLMs offer natural-language explanations alongside predictions.Counterfactual time-series explanations substitute selected training-set points to illustrate alternative outcomes.Source: Explainable prediction · p. 5

Large language models

  • LLMs are pretrained on text and can be fine-tuned for domain tasks, adapting general knowledge without training a foundation model from scratch.Some approaches freeze LLM parameters and train only extended encoders and decoders.Source: Large language models · pp. 5–6

METHODOLOGY

  • The methodology uses an LLM to forecast traffic flow and produce explanations from historical values and external factors.The paper describes prompt construction and fine-tuning as central steps in the approach.Source: METHODOLOGY · p. 6

Problem Description

  • Traffic forecasting is framed as autoregressive token generation from historical flow and external factors, with explanations generated after the predicted sequence.Prompts encode task and traffic-domain knowledge, regional attributes derived from nearby PoIs, the past 12 hours of flow, and factors such as date, holidays, and weather. Spatial-temporal CoT prompts guide consideration of location, time periods, and links between spatial context and historical-flow changes.Source: Problem Description · pp. 6, 7, 9, 10

Supervised Fine-tuning

Supervised fine-tuning adapts pretrained LLMs to traffic forecasting using labeled examples and a next-token objective; xTP-LLM uses LoRA updates to reduce the number of parameters involved in fine-tuning.

Supervised Fine-tuning

  • Supervised fine-tuning adapts a pretrained LLM to a specific task by training it on relevant labeled examples and optimizing a loss against true outputs.For traffic prediction, the examples pair historical traffic data with accurate forecasts.Source: Supervised Fine-tuning · p. 10
  • Traffic forecasting is framed as autoregressive next-token generation, with the model trained on the annotated dataset using a next-token prediction objective.The target token is compared with the ground-truth next token, conditioned on historical input tokens and previously generated tokens.Source: Supervised Fine-tuning · p. 10
  • xTP-LLM uses LoRA low-rank updates to adapt Llama2-7B-chat, reducing the number of parameters involved in updates from d × k to r × (d + k).The rank r is chosen much smaller than the dimensions of the original weight matrix.Source: Supervised Fine-tuning · pp. 10–11

Explanations Generation

Prompted explanation generation initially produced explanations that could diverge from predictions, so xTP-LLM uses few-shot examples to guide their alignment during inference.

Explanations Generation

  • Adding explanation instructions to prompts enabled xTP-LLM to generate explanations, but they often did not cohere with predictions because only predictions were included in fine-tuning loss calculations.The authors attribute the mismatch to insufficient alignment between explanations and predictions.Source: Explanations Generation · p. 11
  • The model receives 2–3 examples pairing traffic predictions with aligned explanations before the instruction fine-tuning prompt, using them as inference-time guidance.The examples are generated with ChatGPT using ground-truth predictions, and the approach is described as improving coherence without extensive new data or additional fine-tuning.Source: Explanations Generation · p. 11

EXPERIMENTAL SETTINGS AND RESULTS

The experiments use CATraffic, a multimodal California traffic dataset, and evaluate forecasts across a full year and on a separate zero-shot San Diego set.

EXPERIMENTAL SETTINGS AND RESULTS

  • The experimental section compares xTP-LLM with baseline models and examines spatial-temporal performance, ablations, generalization, and generated explanations.The supplied passage outlines the planned analyses but does not report their outcomes.Source: EXPERIMENTAL SETTINGS AND RESULTS · p. 11

Dataset Description

  • CATraffic combines traffic volumes from 1,000 sensors in Greater Los Angeles and Greater Bay Area with weather, points of interest, and holiday information.It uses hourly data from January 1, 2018, to December 30, 2019, selected from the five-year LargeST traffic dataset.Source: Dataset Description · pp. 11–12
  • The 1,000 CATraffic sensors were selected from 8,600 by clustering sensors on nearby points-of-interest features and choosing one representative sensor per cluster.The procedure uses 20 PoI categories normalized within a 5 km range.Source: Dataset Description · p. 12
  • The study trains on 2018 data and validates on 2019, predicts 12 hours from 12 historical hours, and evaluates zero-shot generalization on 100 San Diego sensors.The San Diego set covers November 1 through December 31, 2019, and was not used for fine-tuning; samples with 24 consecutive zero values were filtered out.Source: Dataset Description · p. 12

Evaluation Metrics

The paper evaluates forecasting accuracy with three error metrics and compares xTP-LLM with nine deep-learning baselines spanning temporal, recurrent, convolutional, and attention-based approaches.

Evaluation Metrics

  • RMSE emphasizes larger errors, MAE averages absolute errors equally, and MAPE expresses average percentage differences but is sensitive to zero denominators.The authors describe using a combination of these metrics to assess forecasting performance.Source: Evaluation Metrics · pp. 12–13

Baseline Models

  • The nine baselines include LSTM, DCRNN, AGCRN, STGCN, GWNET, ASTGCN, STTN, STGODE, and DSTAGNN.They represent temporal-only, recurrent, TCN-based, attention-based, neural-ODE, and dynamic sensor-correlation approaches.Source: Baseline Models · p. 13

Experiment Settings

xTP-LLM outperforms the reported baselines on average and across prediction steps, while showing consistent performance across locations and days. The experiments use CATraffic and a 12-hour history to forecast the next 12 hours.

Experiment Settings

Overall Performance

Spatial and Temporal Homogeneity

  • The model's reported advantages across locations and days support its robustness to the spatial and temporal variation examined in these analyses.Source: Spatial and Temporal Homogeneity · p. 15
  • xTP-LLM shows relatively consistent prediction performance across Greater Los Angeles locations and lower daily average MAPE throughout November 2019.The spatial comparison covers four prediction horizons; the temporal comparison uses daily averages.Source: Spatial and Temporal Homogeneity · p. 15

Ablation Studies

The ablations find gains from date, weather, and PoI inputs, with date producing the largest individual improvement. Combinations of external inputs and domain knowledge with CoT prompting also show benefits.

The impact of input components on model performance

The impact of different horizons on model performance

Longer historical input is associated with lower MAE, while prediction performance generally worsens at longer forecast horizons. The reported robustness analyses cover time periods, weather conditions, and zero-shot comparisons on unseen datasets.

The impact of different horizons on model performance

  • The 12-hour history yields the lowest MAE and the 4-hour history the highest; performance also deteriorates at longer prediction horizons.The authors link longer forecasts to greater uncertainty and describe early prediction errors as accumulating across steps.Source: The impact of different horizons on… · p. 18, Figure 7 · p. 18

Robustness analysis on the test dataset

  • Average MAE ranges from 21.25 to 22.57 across peak/off-peak hours and weekdays/weekends, while MAE remains relatively stable across weather conditions.The authors report good test performance for storm and snow conditions absent from training, despite those conditions being underrepresented.Source: Robustness analysis on the test dataset · pp. 18–19, Table 3 · p. 19, Table 4 · p. 19
  • On unseen CATraffic and TaxiBJ zero-shot evaluations, xTP-LLM outperforms the compared LLMs despite its smaller parameter scale.The comparisons include the CATraffic zero-shot dataset and TaxiBJ inflow and outflow tasks.Source: Table 5 · p. 19, zero-shot testing analysis · p. 21
  • In selected CATraffic zero-shot samples from three sensors and four time periods, xTP-LLM captures traffic trends and prediction values better than the compared LLMs.The authors attribute the differences to prompt design and fine-tuning.Source: Figure 8 · p. 20, zero-shot testing analysis · p. 21

Case Studies for Explainable Predictions

The case studies pair traffic-flow forecasts with textual explanations across different time slots and external conditions. They illustrate weekday, rain, and holiday-related predictions, while noting difficulty with nuanced flow fluctuations.

Result-explainable studies

  • The examples cover different prediction time slots and external factors, pairing ground truths and predictions with explanatory text.The plotted forecasts include light-red 95% confidence intervals; the displayed explanations were shortened with ChatGPT.Source: Result-explainable studies · p. 21, Table 6 · p. 22
  • The model describes a typical weekday rise followed by a decline, identifies rain as an impact factor associated with a delayed traffic peak, and accounts for holiday impacts.The complete output for example (d) appears in Appendix A.2.Source: Result-explainable studies · p. 21
  • The authors report difficulty capturing nuanced flow fluctuations, including those at 14–18 hours in Example 2 and 20–22 hours in Example (d).They nevertheless report that the model provides textual rationales for its forecasts.Source: Result-explainable studies · p. 21

What-if Analysis

The what-if analysis varies prompts to represent an accident or sandstorm and compares the resulting predictions with normal predictions and ground truth. The conclusion identifies stronger spatial modeling and broader external inputs as future directions, alongside substantial resource and engineering demands for city-level LLM systems.

What-if Analysis

  • The analysis prompts two hypothetical conditions—a nearby accident at 10 p.m. and a severe sandstorm at 9 a.m.—and compares normal and what-if predictions with ground truth.Other prompt sections remain consistent with the original prediction tasks.Source: What-if Analysis · p. 23, Table 7 · p. 23
  • In the accident scenario, nearby traffic volume falls from 530 to 430; in the sandstorm scenario, vehicle counts fall from 532 to 352 before traffic gradually recovers.The authors attribute these changes to accident congestion and to reduced visibility and road passability, respectively.Source: What-if Analysis · p. 23

CONCLUSION AND FUTURE WORK

  • The conclusion presents xTP-LLM as a traffic prediction model combining competitive performance with interpretability through multimodal inputs and language-based representations.It describes the framework as transparent and adaptable for various urban prediction tasks.Source: CONCLUSION AND FUTURE WORK · p. 24
  • Future work includes improving LLM use of spatial information and sensor relationships, while adding factors such as accidents, human activities, and major events.City-level multimodal applications are described as challenging and requiring substantial computational resources and engineering capabilities.Source: CONCLUSION AND FUTURE WORK · p. 24

APPENDIX

The prompt design uses a persistent system prompt for role, traffic context, and chain-of-thought guidance, with user inputs and ground truth varying. An explanatory example shows a 12-hour Christmas Day forecast that accounts for holidays and weather-related effects on human activity.

Prompt Design

  • The system prompt sets the LLM’s role, supplies traffic-prediction context, and provides chain-of-thought guidance; user input and corresponding ground truth vary across the dataset.The system prompt remains the same throughout the dataset.Source: Prompt Design · p. 29, Table 8 · p. 30

Explanatory Example

  • For a 12-hour forecast made at 5 PM on Christmas Day 2019, explanatory demand instructions let xTP-LLM output both predictions and explanations while accounting for holidays and weather’s impact on human activities.The example response is presented in Table 5.Source: Explanatory Example · p. 29
  • Table 9 presents an example of the complete response with explanatory output from the proposed xTP-LLM; this example corresponds to example (d) in Table 6.Source: Table 9 · p. 31

Related papers

Loading 2404.02937v5…