PaperlayerView exact revision on arXiv

Source-linked AI summary

LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Models

Mingxing Peng, Xusen Guo, Xianda Chen, Meixin Zhu, Kehua Chen

arXiv:2403.18344v2cs.AI

TL;DR

Lane-change prediction needs advance intention and trajectory forecasts, while existing approaches have limitations in long-term accuracy and interpretability. LC-LLM turns scenario information into prompts for a fine-tuned LLM and reports improvements across intention and trajectory prediction on highD.

  • Problem

    Existing lane-change prediction approaches have limited long-term accuracy and interpretability, despite the need to forecast surrounding vehicles’ intentions and trajectories in advance.

    Source: §I · p. 1

  • Method

    LC-LLM converts vehicle-state and map information into natural-language prompts, fine-tunes an LLM with CoT reasoning, and requests explanations during inference.

    Source: §I · pp. 1–2

  • Results

    On highD, LC-LLM reports a 17.7% improvement in lane change intention prediction, a 64.4% improvement in lateral trajectory prediction, and a 66.1% improvement in longitudinal trajectory prediction over baseline models.

    Source: §I · p. 2

  • Takeaways & Limitations

    The paper evaluates CoT reasoning accuracy as a quantitative measure of LC-LLM’s interpretability.

    Source: §I · p. 2

    LC-LLM has been tested exclusively on highD, which primarily comprises highway scenarios; performance on more diverse urban and suburban datasets remains to be evaluated.

    Source: §I · p. 11

Abstract

from arXiv · show

To ensure safe driving in dynamic environments, autonomous vehicles should possess the capability to accurately predict lane change intentions of surrounding vehicles in advance and forecast their future trajectories. Existing motion prediction approaches have ample room for improvement, particularly in terms of long-term prediction accuracy and interpretability. In this paper, we address these challenges by proposing LC-LLM, an explainable lane change prediction model that leverages the strong reasoning capabilities and self-explanation abilities of Large Language Models (LLMs). Essentially, we reformulate the lane change prediction task as a language modeling problem, processing heterogeneous driving scenario information as natural language prompts for LLMs and employing supervised fine-tuning to tailor LLMs specifically for lane change prediction task. Additionally, we finetune the Chain-of-Thought (CoT) reasoning to improve prediction transparency and reliability, and include explanatory requirements in the prompts during inference stage. Therefore, our LC-LLM model not only predicts lane change intentions and trajectories but also provides CoT reasoning and explanations for its predictions, enhancing its interpretability. Extensive experiments based on the large-scale highD dataset demonstrate the superior performance and interpretability of our LC-LLM in lane change prediction task. To the best of our knowledge, this is the first attempt to utilize LLMs for predicting lane change behavior. Our study shows that LLMs can effectively encode comprehensive interaction information for driving behavior understanding.

I. INTRODUCTION

The introduction identifies limited long-term prediction and interpretability as challenges in lane-change prediction. It presents LC-LLM as a language-modeling approach that predicts intentions and trajectories while generating explanations, and reports improvements across three prediction tasks.

I. INTRODUCTION

  • Existing methods have limited interaction modeling and interpretability, while early lane-change prediction depends on surrounding-vehicle interactions.The introduction identifies weak long-term prediction as a consequence of limited interaction modeling.Source: §I · p. 1
  • LC-LLM converts target-vehicle state, surrounding-vehicle state, and map information into natural-language prompts, then fine-tunes an LLM with Chain-of-Thought reasoning.Inference prompts also request explanations for prediction results.Source: §I · pp. 1–2
  • The paper presents LC-LLM as the first LLM for lane-change prediction and reports generated explanations alongside predictions, with CoT accuracy evaluated as an interpretability measure.The model predicts lane-change intentions and trajectories.Source: §I · p. 2
  • 17.7% improvement in lane change intention prediction, 64.4% improvement in lateral trajectory prediction, and 66.1% improvement in longitudinal trajectory prediction are reported on highD.The introduction states that LC-LLM outperforms all baseline models.Source: §I · p. 2

II. RELATED WORK

Related work covers vehicle-state prediction methods and the emerging use of LLMs in autonomous driving. It reviews intention and trajectory prediction approaches, noting that LLM use for lane-change intention and trajectory prediction remains unexplored in the reviewed work.

II. RELATED WORK

  • Prior work groups vehicle-state prediction into intention and trajectory prediction, and reviews LLM applications in autonomous driving,,, [15].The review describes these as two categories based on output type.Source: §II · p. 2

A. Intention Prediction

  • Intention-prediction studies span SVMs, CNNs, recurrent networks, and Transformers,,,,,,,,, but advance prediction remains a challenge.The review notes that these approaches detect maneuvers that have already started more effectively than they predict intentions in advance.Source: A. Intention Prediction · p. 2

B. Trajectory Prediction

  • Trajectory-prediction methods progress from rasterized CNNs and LSTMs to vectorized GNNs and Transformers [22],,,,,,,,, [28], yet often lack interpretability.The review also describes discrete motion tokens as a language-modeling approach.Source: B. Trajectory Prediction · p. 2

C. LLMs for Autonomous Driving

  • Although LLMs have been explored for autonomous-driving perception and planning [15], their use for intention and trajectory prediction remains unexplored; Fig. 1 traces LC-LLM from prompted observations through fine-tuning to explanatory inference.The figure describes natural-language observation prompts, fine-tuning, and inference prompts with explanatory requirements.Source: C. LLMs for Autonomous Driving · p. 3, Figure 1 · p. 3

III. METHODOLOGY

LC-LLM formulates intention and trajectory prediction as language modeling, using driving observations as inputs and producing predictions with CoT reasoning. It trains the model to predict output tokens from the ground-truth sequence.

III. METHODOLOGY

  • LC-LLM reformulates intention and trajectory prediction as language modeling and uses supervised fine-tuning to tailor an LLM to the task.At inference, explanatory requirements are added to prompts so the model provides CoT reasoning and explanations.Source: §III · p. 3

A. Problem Formulation

  • The task is to forecast the target vehicle’s trajectory at t future timesteps and its lane-change intention over the same window.The scenario illustration distinguishes the target vehicle, surrounding vehicles, and the target’s future trajectory.Source: A. Problem Formulation · p. 4
  • The model takes target-vehicle state, surrounding-vehicle state, and map information as inputs, and outputs CoT reasoning, intention, and future trajectory.The formulation denotes the model process as F.Source: A. Problem Formulation · p. 4
  • Inputs and prediction results are represented as token sequences, making the prediction objective a language-modeling loss over the ground-truth output tokens.The model predicts the next ground-truth token given its historical tokens.Source: A. Problem Formulation · p. 4

B. Prompting

The prompts divide instructions from scenario observations: a consistent system message specifies the model’s role and expected output, while a user message describes the current driving scene. The scene description supplies vehicle and map context for prediction.

B. Prompting

  • The system message stays consistent across scenarios and specifies the model’s role, coordinate system, and expected output, while the user message describes scenario-specific observations.The expected output includes intention, trajectory points over four seconds, and reasoning.Source: B. Prompting · p. 4
  • The prompt requests one of three intention modes and four trajectory points, one per second, for the future four-second horizon.The coordinate frame is centered on the target vehicle, with positive y to its left and x parallel to its facing direction.Source: B. Prompting · pp. 4–5
  • User prompts describe map and target-vehicle information alongside surrounding vehicles’ spatial relationships, speeds, types, and relative distances in eight directions.The target-vehicle description includes its historical trajectory over the past two seconds, current velocity, and vehicle type.Source: B. Prompting · pp. 4–5

C. Reasoning

LC-LLM’s CoT reasoning is labeled using driving knowledge and traffic rules, and consists of notable features and potential behaviors. The annotations encode measurable vehicle cues and classify possible driving patterns.

C. Reasoning

  • CoT labels draw on driving knowledge, traffic rules, and traditional lane-change rules to support the model’s reasoning and explanations.The reasoning comprises notable features and potential behaviors.Source: C. Reasoning · p. 5

1) Labeling of Notable feature:

  • Notable features include lateral velocity above 1.5 km/h, longitudinal acceleration above 0.4 m/s2, relative speeds of nearby vehicles, and trucks ahead within 100 meters.A target vehicle being a truck is also labeled notable in right-lane-change scenarios.Source: 1) Labeling of Notable feature · p. 5

2) Labeling of Potential behavior:

  • Eight potential-behavior labels cover overtaking, fast- or slow-lane changes, irregular changes, following, and normal lane keeping.The labels distinguish scenarios using cues such as a slower or blocked lane ahead, acceleration, deceleration, and truck status.Source: 2) Labeling of Potential behavior · p. 6

D. Fine-tuning

LC-LLM is fine-tuned on natural-language versions of driving scenarios and produces predictions with CoT reasoning and explanations. The paper evaluates it on highD, including its interpretability and prediction performance.

D. Fine-tuning

  • LC-LLM uses Llama-2-13b-chat with LoRA and supervised fine-tuning, aligning predicted outputs with highD ground truth while masking prompt-token loss.Samples pair an input prompt with an answer containing automatically labeled future trajectories, driving intentions, and CoT reasoning.Source: D. Fine-tuning · p. 6

E. Interpretability of Prediction

  • Fine-tuned CoT reasoning and inference prompts requiring explanations provide predictions with corresponding reasoning and explanations.The model weights remain fixed when explanatory requirements are incorporated at inference.Source: E. Interpretability of Prediction · p. 6

IV. EXPERIMENTAL RESULTS AND ANALYSIS

A. Dataset Processing

  • The study trains and tests on highD, splitting recordings 50/10 and normalizing positions in target-centered coordinates.Lane-change samples span 0–4 seconds ahead, divided into four prediction-time intervals; the sampled datasets contain 144,000 training and 24,000 test samples.Source: A. Dataset Processing · p. 7

B. Experimental Setup

Experiments use repeated runs of LC-LLM and reimplemented baselines, with intention prediction assessed by classification metrics and trajectory prediction by RMSE.

B. Experimental Setup

  • Experiments are repeated five times, with average results reported; LC-LLM training uses eight A800 GPUs and takes 11 hours for the 13B model.LSTM- and Transformer-based models are reimplemented as baselines.Source: B. Experimental Setup · p. 7

C. Evaluation Metrics

1) Evaluation Metrics of Intention Prediction: •

  • Precision measures correctly predicted positives among predicted positives, while recall measures correctly predicted positives among actual positives.The respective denominators are total predicted positives and total actual positives.Source: 1) Evaluation Metrics of Intention… · p. 7

2) Evaluation Metrics of Trajectory Prediction: •

Trajectory prediction is evaluated with RMSE for lateral and longitudinal errors, and the paper compares LC-LLM with baselines on highD.

2) Evaluation Metrics of Trajectory Prediction: •

  • RMSE measures average deviation between predicted and actual values; RMSE (lat) and RMSE (lon) assess lateral and longitudinal prediction error.The figure compares trajectory-prediction performance across the proposed model and baselines on highD.Source: 2) Evaluation Metrics of Trajectory… · p. 7, Figure 6 · p. 8

D. Overall Performance

  • The study evaluates LC-LLM on both intention prediction and trajectory prediction.The paper introduces these as its two evaluation tasks.Source: D. Overall Performance · p. 8

1) Intention Prediction Results and Analysis:

LC-LLM outperforms baselines in long-term lane-change prediction, with reported improvements in intention and trajectory prediction. Ablations also report gains from contextual inputs and CoT reasoning.

1) Intention Prediction Results and Analysis:

  • At T ∈ (3, 4], average F1 exceeds Transformer by 17.7% and LSTM by 25.7%, with advantages also reported at T ∈ (2, 3].The reported advantage over baselines is especially pronounced at longer prediction times.Source: 1) Intention Prediction Results and… · p. 8

2) Trajectory Prediction Results and Analysis:

  • Lateral and longitudinal RMSE improve over baselines, with LC-LLM's advantage increasing at larger prediction horizons.The authors describe this pattern as indicating better robustness for long-term prediction horizons.Source: 2) Trajectory Prediction Results and… · p. 8
  • At 4 s, LC-LLM outperforms all baselines by 17.7% in intention prediction, 64.4% in lateral trajectory prediction, and 66.1% in longitudinal trajectory prediction.These are the reported long-term prediction improvements across the three tasks or trajectory axes.Source: 2) Trajectory Prediction Results and… · pp. 8–9

1) Components Ablation Studies:

  • Against target-only prompting, full contextual prompts plus CoT reduce lateral RMSE by 10.64% and longitudinal RMSE by 29.57%.The authors identify surrounding-vehicle information as crucial and report further trajectory-prediction enhancements from CoT reasoning.Source: Figure 7 · p. 9, 1) Components Ablation Studies · p. 9

2) Multi-task Ablation Study:

The section reports benefits from joint intention-and-trajectory prediction, illustrates scenario-grounded explanations, and evaluates reasoning accuracy and fine-tuning against zero-shot performance.

2) Multi-task Ablation Study:

  • Multi-task learning outperforms isolated trajectory prediction, while adding trajectory prediction also improves intention-prediction accuracy.The authors attribute the trajectory benefit to integrating high-level intention prediction and the intention benefit to capturing more nuanced details.Source: 2) Multi-task Ablation Study · p. 9

1) Visualization of Results:

  • Visual examples pair driving scenarios with predictions and explanations, including reasoning consistent with surrounding-vehicle speeds and a slow truck.The paper contrasts these outputs with prior methods that generate prediction results alone.Source: 1) Visualization of Results · p. 9, Figure 8 · p. 10

2) Quantitative Evaluation:

  • The model scores 97.2 for Chain-of-Thought reasoning accuracy, based on manual scoring of 100 test samples.The scoring deducts points for errors, omissions, or additions of notable features and for potential behavior errors.Source: 2) Quantitative Evaluation · p. 9

G. Zero-shot Study

  • Fine-tuning yields markedly higher F1 scores and lower RMSE than zero-shot performance across two tasks.Table IV presents the zero-shot and fine-tuned comparison.Source: G. Zero-shot Study · p. 9, Table IV · p. 11

H. Robustness Evaluations

The robustness evaluation tests LC-LLM in four safety-critical scenarios outside the HighD dataset. The authors also identify dataset scope and inference speed as limitations and describe future extensions.

H. Robustness Evaluations

  • Four safety-critical scenarios test emergency braking or acceleration by nearby vehicles; examples report collision-avoidance adjustments and trajectory changes.The scenarios vary lane-change direction, vehicle speeds, and relative distances; the reported examples include a stopping right-front vehicle and an accelerating right-rear vehicle.Source: H. Robustness Evaluations · p. 11

I. Limitations

  • Evaluation is limited to HighD highway scenarios; urban and suburban generalization and detailed-map prompting remain unresolved.The authors call for evaluation on more diverse datasets and describe accurate high-definition-map prompts as future work.Source: §I · p. 11
  • LC-LLM-13B takes 26.72 seconds per batch of 32, versus 0.011 seconds for LSTM and 0.014 seconds for Transformer, challenging real-time use.The authors partly attribute the slower inference to the model's larger parameter count.Source: §I · p. 11

V. CONCLUSION

  • The conclusion reports improved highD accuracy and interpretability, with urban extension, faster inference, and simultaneous multi-vehicle prediction left to future work.The proposed future work includes knowledge distillation to improve speed and efficiency.Source: §V · pp. 11–12

Related papers

Loading 2403.18344v2…