Making Large Language Models Better Planners with Reasoning-Decision Alignment

1 Introduction

• Prior LLM-based AD methods use CoT to enhance explainability and scene understanding, but the paper identifies inconsistencies between reasoning and decisions as an unaddressed problem.The motivating example describes correct recognition of a stopped front car followed by a wrong plan to move forward.Source: §1 · pp. 2–3, Figure 1 · p. 2
• End-to-end autonomous driving systems face interpretability and robustness challenges, while LLM-based AD methods leave reasoning-decision misalignment unaddressed.The paper describes misalignment as inconsistency between scores assigned to CoTs and decision-making accuracy.Source: §1 · pp. 2–3
• 0.80 L2 error and 0.32 collision rates on nuScenes, and 0.82 L2 error and 0.38 collision rate on DriveLM-nuScenes, are reported as state-of-the-art and leading results, respectively.The paper also says RDA-Driver outperforms or is comparable to prior works across two nuScenes evaluation methods.Source: §1 · p. 3
• RDA-Driver uses a multimodal LLM to produce CoT reasoning and planning results together, with a ranking loss to align them.The model transforms multi-view images into BEV features and redesigns CoT reasoning.Source: §1 · p. 3
• The redesigned CoTs cover perception, prediction, and decision-making to support interpretable driving decisions.The paper presents these as part of RDA-Driver's reasoning design.Source: §1 · p. 3

End-to-End Autonomous Driving

• Conventional modular AD separates perception, prediction, and planning, while end-to-end systems generate routes from raw sensor data but raise interpretability and robustness challenges.Examples discussed include ST-P3, UniAD, and VAD.Source: §2 · pp. 3–4

Autonomous Driving with Large Language Models

• LLM-based AD work includes driving datasets, language-assisted planning, and systems integrating logical reasoning with decision tasks, but often overlooks reasoning-decision inconsistency.Examples include LMDrive, DriveLM, GPT-Driver, DME-Driver, and Reason2drive.Source: §2 · p. 4

Alignment in Large Language Models

• Prior alignment research uses reinforcement or contrastive learning, but has focused chiefly on math and commonsense reasoning rather than AD decisions.The paper describes its work as the first attempt to use contrastive learning to align reasoning and planning in AD scenarios with safety and timeliness requirements.Source: §2 · p. 4

3 Method

• RDA-Driver takes multi-view images, ego status, and multi-turn CoT prompts as input, then produces reasoning and planning results together; training contrasts positive and generated negative samples.The framework uses token-average scores for CoT answers and a contrastive loss to rank positive samples higher.Source: Figure 2 · p. 5

3.1 RDA-Driver Overview

• The system encodes multi-view images, maps visual representations to language-aligned tokens with an adapter, and feeds visual and prompt tokens to an LLM decoder.The paper adopts Llama as its decoder.Source: §3.1 · p. 5

Vision Encoder.

• The vision encoder converts multi-perspective images into BEV features using lift-splat operations and is pretrained on object detection for semantic features.Lift projects pixels into 3D voxels using depth distributions, while splat aggregates voxel features by sum pooling.Source: Vision Encoder · pp. 5–6

Adapter Module.

• Grid-based region encoding flattens each h×w BEV grid into a visual token, reducing token count to H/h × W/w before a two-layer MLP maps tokens to the language dimension.The authors contrast this approach with cross-attention, which may lose fine-grained scene details, and pooling, which sacrifices more visual information.Source: Adapter Module · p. 6

3.2 Chain-of-Thought Alignment Tuning

• Causal CoT reasoning establishes relationships between events to help predict potential outcomes and plan driving routes and behaviors.The passage uses an obstacle ahead as an example, noting possible emergency braking, lane changes, and reactions by surrounding vehicles.Source: §3.2 · p. 6
• The authors state that errors or inconsistencies in causal reasoning may lead to incorrect decisions, making the system unusable.Source: §3.2 · p. 6
• A consistency constraint loss is adopted to keep the model’s explanations and conclusions aligned during reasoning.Source: §3.2 · p. 6

Vanilla Chain-of-Though Fine-tuning.

• Each training sample contains visual input V_i, text input T_i, and a ground-truth trajectory A_i.Source: Vanilla Chain-of-Though Fine-tuning · p. 6
• The text input includes ego status and CoT prompts; ego status records velocity, heading angular velocity, acceleration, and the recent three-frame trajectory.Source: Vanilla Chain-of-Though Fine-tuning · p. 7
• The VLM is fine-tuned with cross-entropy loss to predict response tokens from visual input and multi-turn CoT prompts.The loss is written over tokens of the response, conditioned on the preceding response tokens, CoT sub-prompts, and visual input.Source: Vanilla Chain-of-Though Fine-tuning · p. 7

Dataset Generation.

• The procedure generates multiple multi-turn CoTs per training sample through both model-based and data-based methods.Source: Dataset Generation · p. 7
• For model-based generation, candidates are ranked by the L2 distance between predicted and ground-truth trajectories.Source: Dataset Generation · p. 7
• For data-based generation, annotated CoT labels from similar scenes with different decisions are combined into positive and negative samples.Samples from the same scene at different timestamps can be permuted and combined to form alternatives.Source: Dataset Generation · p. 7
• The two data sources are intended to support decision-making and reasoning while teaching consistency between causal reasoning and conclusions.Source: Dataset Generation · p. 7

Alignment Function.

• For model-generated candidates, CoT quality is ranked by trajectory output, and a ranking alignment loss favors higher-quality CoTs.The model computes a token-average score for each sample; the ranking uses predicted-trajectory L2 distance to ground truth.Source: Alignment Function · pp. 7–8
• For data-generated examples, binary alignment loss is used to make positive-sample scores higher than negative-sample scores.The passage identifies the negative generated data and says the detach operation is employed.Source: Alignment Function · p. 8
• The final training loss combines vanilla, ranking-alignment, and binary-alignment losses.Source: Alignment Function · p. 8

3.3 Driving with Chain-of-Thought Prompting

• The prompt chain identifies important objects and their locations before using them in future reasoning and driving decisions.The paper presents this as the perception stage of its logical chain.Source: §3.3 · p. 8, Figure 3 · p. 9
• Prediction prompts ask which noticed object matters and what action the ego vehicle should take, with logical descriptions guiding judgments and risk avoidance.The prompt distinguishes objects by whether the ego vehicle notices them first, second, or third.Source: §3.3 · p. 9
• The decision stage summarizes perception and prediction to select the ego vehicle’s direction and speed, such as turning or stopping.The prompt asks the system to predict the ego vehicle’s behavior.Source: §3.3 · p. 9
• The final planning stage forecasts a safe, feasible three-second ego-vehicle trajectory as six waypoints.Trajectory output is the system’s final planning prompt.Source: §3.3 · p. 9

4.1 Experimental Setting

• Planning is evaluated on nuScenes and DriveLM-nuScenes using L2 error and collision rate over 1-, 2-, and 3-second horizons.RDA-Driver uses 4,072 DriveLM-nuScenes training samples, and the BEV encoder is kept frozen during fine-tuning.Source: §4.1 · p. 10

4.2 Main Results

• On nuScenes, RDA-Driver outperforms or matches prior work on L2 and collision rate; on DriveLM-nuScenes, it shows only 0.01–0.06 fluctuations.The authors also report comparable or better performance than UniAD and VAD with 1/7 of their training size, and better results than GPT-Driver without relying on other foundation-model priors.Source: Table 1 · p. 11, Table 2 · p. 11, §4.2 · pp. 11–12

Effectiveness of Alignment Loss.

• 62 to 71: the CoT response score improves with alignment loss.The loss experiments use ranking across generated samples and binary loss on similar samples; CoT scoring uses 200 training samples because validation CoT labels are unavailable, with GPT3.5 ratings.Source: Table 3 · p. 12, Effectiveness of Alignment Loss · pp. 12–13

Comparison between Visual Encoders.

• Partitioning and flattening BEV features into regions performs better than CLIP, whereas simply pooling BEV features has adverse effects on decision-making.The authors emphasize preserving BEV features when converting them to LLM tokens, particularly for fine-grained trajectory regression.Source: Table 5 · p. 13, Comparison between Visual Encoders · pp. 13–14

Effectiveness of chain-of-Thought Redesign.

• Multi-round CoTs support logical reasoning for complex decisions, and omitting perception, prediction, or decision-making weakens reasoning and trajectory prediction.The reported comparison varies both CoT category and number of QA rounds.Source: Effectiveness of chain-of-Thought… · p. 14, Table 6 · p. 14

• RDA-Driver constrains CoT reasoning and subsequent planning results through contrastive learning.Source: §5 · p. 14
• Multi-turn CoT encourages holistic reasoning about driving scenarios.Source: §5 · p. 14
• The model uses only multi-view images; exploring video input is future work.Source: §5 · p. 14