Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving

Pengxiang Li; Yinan Zheng; Yue Wang; Huimin Wang; Hang Zhao; Jingjing Liu; Xianyuan Zhan; Kun Zhan; Xianpeng Lang

Source-linked AI summary

Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving

Pengxiang Li, Yinan Zheng, Yue Wang, Huimin Wang, Hang Zhao, Jingjing Liu, Xianyuan Zhan, Kun Zhan, Xianpeng Lang

arXiv:2509.20109v1Sep 24, 2025cs.ROcs.AIcs.CL

TL;DR

Imitation-learning-based driving planners struggle to enforce physical safety rules, motivating a more controllable planning approach. ReflectDrive discretizes trajectories for diffusion-language-model planning and adds gradient-free reflective regeneration; on NAVSIM, it is reported to improve safety-critical trajectory generation, while inference efficiency remains a limitation.

Problem
Imitation learning does not inherently encode physical rules, so driving systems can generate trajectories that violate safety constraints.
Method
ReflectDrive discretizes two-dimensional trajectories into action-codebook tokens and uses goal-conditioned generation, local safety search, and inpainting-based reflection for gradient-free correction.
Results
ReflectDrive is evaluated on NAVSIM and is reported to enforce hard safety constraints without compromising behavioral coherence.
Takeaways & Limitations
Discrete diffusion with reflective inference offers a scalable and reliable approach for safety-critical autonomous-driving trajectory generation.
Takeaways & Limitations
More inference iterations do not necessarily improve results because the output token count is relatively small, and future work targets computational efficiency.

Abstract

from arXiv · show

End-to-End (E2E) solutions have emerged as a mainstream approach for autonomous driving systems, with Vision-Language-Action (VLA) models representing a new paradigm that leverages pre-trained multimodal knowledge from Vision-Language Models (VLMs) to interpret and interact with complex real-world environments. However, these methods remain constrained by the limitations of imitation learning, which struggles to inherently encode physical rules during training. Existing approaches often rely on complex rule-based post-refinement, employ reinforcement learning that remains largely limited to simulation, or utilize diffusion guidance that requires computationally expensive gradient calculations. To address these challenges, we introduce ReflectDrive, a novel learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion. We first discretize the two-dimensional driving space to construct an action codebook, enabling the use of pre-trained Diffusion Language Models for planning tasks through fine-tuning. Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient computation. Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors. Based on this, we apply local search methods to identify unsafe tokens and determine feasible solutions, which then serve as safe anchors for inpainting-based regeneration. Evaluated on the NAVSIM benchmark, ReflectDrive demonstrates significant advantages in safety-critical trajectory generation, offering a scalable and reliable solution for autonomous driving systems.

1 INTRODUCTION

ReflectDrive addresses safety gaps in imitation-learning-based E2E driving by combining discrete diffusion planning with inference-time reflection. It uses discrete actions and safety validation to guide trajectory generation while preserving behavioral coherence.

Imitation learning can produce probable trajectories that violate collision-avoidance and drivable-area constraints because physical rules are not inherently encoded.
ReflectDrive discretizes the two-dimensional driving space into an action codebook, enabling pre-trained Diffusion Language Models to generate trajectories through fine-tuning.
Discrete diffusion enables safety constraints to be incorporated through search, masking, and sampling during trajectory generation.
The reflection mechanism provides inference-time guidance for denoising by combining external safety validation with efficient discrete token optimization.
ReflectDrive is evaluated on real-world driving benchmarks and is reported to enforce hard safety constraints without compromising behavioral coherence.

2 RELATED WORK

Related work positions E2E and VLA systems as scalable alternatives to modular and rule-based driving, while highlighting unresolved safety limitations in imitation-learning pipelines. Existing remedies use trajectory anchors or post-selection to improve predicted paths.

E2E systems have largely replaced modular and rule-based approaches because of their scalability and reduced reliance on manually engineered rules.
VLA models incorporate pretrained VLM world knowledge and can accept human instructions for long-tail scenarios and preferred driving behaviors.
Imitation-learning pipelines remain affected by causal confusion and lack verifiable safety guarantees.
Existing approaches use clustered or rule-based trajectory anchors as conditioning inputs or candidates for post-selection and refinement.

3 PRELIMINARIES

The planning task models future trajectories conditioned on scene context, while discrete diffusion corrupts and reconstructs token sequences through masked denoising. Its discrete representation supports inpainting, search, and safety-constraint integration.

Autonomous driving planning learns a conditional trajectory distribution p(τ | c), with context comprising multi-view images, instructions, and ego state.
Discrete diffusion generates structured sequences through a forward masking process and a learned reverse denoising process.
The denoising model predicts original tokens at masked positions using unmasked tokens, timestep s, and external context c, trained with a negative log-likelihood objective.
Inference starts from a fully masked sequence and iteratively samples, fixes, and remasks tokens across refinement steps.
Inpainting reconstructs masked segments while preserving unmasked context, and discrete tokens support efficient search and safety-constraint integration.

4 METHOD

ReflectDrive combines discretized trajectory planning with a two-stage reflective inference process. It generates diverse goal-conditioned candidates, selects a trajectory, then iteratively repairs unsafe waypoints using local token search and diffusion inpainting.

4.1 DISCRETE DIFFUSION FOR AUTONOMOUS DRIVING PLANNING: ReflectDrive quantizes each two-dimensional waypoint into codebook token pairs, flattening the trajectory into a discrete sequence that supports efficient BEV search.
4.1 DISCRETE DIFFUSION FOR AUTONOMOUS DRIVING PLANNING: A pretrained Diffusion Language Model is fine-tuned as a VLA planner to generate tokenized trajectories conditioned on images, language instructions, and ego state.
4.2 REFLECTIVE INFERENCE: Reflective inference first generates diverse goal-conditioned trajectories, evaluates them with a global scorer including goal progress, and selects the top-scoring candidate.
4.2 REFLECTIVE INFERENCE: The framework uses global, safety, and local scorers to evaluate complete trajectories, identify failures, and assess candidate token pairs.
4.2 REFLECTIVE INFERENCE: The safety-guided loop uses a safety scorer to locate unsafe waypoints, a local scorer to search a small discrete neighborhood, and corrected tokens as safety anchors.
4.2 REFLECTIVE INFERENCE: Diffusion inpainting regenerates trajectory segments around safety anchors, restoring global coherence until the plan is safe or the computational budget is reached.

5 EXPERIMENTS

ReflectDrive is evaluated on NAVSIM against end-to-end and VLA baselines, with reflective inference improving safety and progress metrics. Ground-truth agent information further approaches human-driving performance, while ablations show complementary roles for goal-conditioned generation and safety-guided regeneration.

5.1 BENCHMARK AND BASELINES: ReflectDrive’s base model achieves a PDMS score of 84.8, comparable to base end-to-end planners but slightly below augmented end-to-end planners.The authors identify limited training-data scale and base VLM capability as potential limiting factors.
5.2 MAIN RESULTS: +7.9 points in EP and +3.9 points in DAC over ReflectDrive (w/o R.I.), alongside +1.3 in TTC and +0.8 in NC.Safety-guided regeneration improves safety metrics, while goal-conditioned generation optimizes ego progress.
5.2 MAIN RESULTS: Safety-guided regeneration iteratively moves risky trajectories into safe regions, avoiding boundary violations, conflicts, or by yielding through deceleration.The trajectories remain kinematically feasible and smooth after discretization.
5.2 MAIN RESULTS: With ground-truth agent information, ReflectDrive reaches NC 99.7, DAC 99.5, TTC 99.1, and EP 88.9.These results match or exceed the reported human-driving trajectory scores and indicate potential gains from more accurate detection and prediction.
5.4 ABLATION STUDIES: The ablations vary generation steps, goal points, exploration steps, and maximum regeneration iterations to study reflective-inference settings.The evaluated parameters control inpainting duration, multimodal candidate count, search range, and regeneration budget.
5.4 ABLATION STUDIES: Goal-conditioned generation improves ego progress, while safety-guided regeneration improves safety metrics and progress, confirming their complementary roles.Together, the components target progress optimization and safety-constraint enforcement without compromising driving efficiency.

6 CONCLUSION

ReflectDrive combines discrete diffusion with reflection for safe trajectory generation. Its NAVSIM evaluation demonstrates effectiveness and safety advantages.

6 CONCLUSION: ReflectDrive discretizes the two-dimensional driving space into an action codebook and uses a pre-trained diffusion language model for planning.The framework fine-tunes the model for trajectory generation.
6 CONCLUSION: The reflection mechanism first generates diverse goal-conditioned trajectories, then performs gradient-free safety-guided regeneration through feasible-solution search and inpainting.This sequence supports multimodal behavior while correcting unsafe trajectories.
6 CONCLUSION: Evaluations on NAVSIM demonstrate ReflectDrive’s effectiveness and safety advantages for trajectory generation.

A VISUALIZATION OF PLANNING RESULTS

The implementation fixes trajectory length and decodes all trajectory points in parallel. ReflectDrive uses diverse goal proposals, spatially distinct goal selection, bounded safety iterations, and supervised fine-tuning from LLaDA-V.

A VISUALIZATION OF PLANNING RESULTS: ReflectDrive generates three diverse goal proposals to cover potential driving intents and applies 0.9-meter non-maximum suppression for spatial separation.
A VISUALIZATION OF PLANNING RESULTS: The output trajectory length is fixed, and all trajectory points are decoded in parallel during inference.
A VISUALIZATION OF PLANNING RESULTS: Training uses the LLaDA-V codebase, its pre-trained weights, and supervised fine-tuning.

C SCORING FUNCTION IMPLEMENTATION DETAILS

The scoring implementation balances hard safety constraints with continuous driving quality and efficiency. A trajectory’s final score combines these components multiplicatively.

C SCORING FUNCTION IMPLEMENTATION DETAILS: The evaluation framework balances hard safety constraints with continuous measures of driving quality and efficiency.
C SCORING FUNCTION IMPLEMENTATION DETAILS: The final trajectory score used by Sglobal and Slocal is the product of Hard Safety Compliance H(τ) and Performance Quality Q(τ).

C.1 HARD SAFETY COMPLIANCE TERM (H(τ))

H(τ) is a hard safety-compliance term that gates trajectory scores using inviolable driving rules. Any rule violation can nullify a trajectory’s score regardless of its performance quality.

C.1 HARD SAFETY COMPLIANCE TERM (H(τ)): H(τ) multiplies individual safety-metric scores, so violating any inviolable rule drives the term toward zero.The Safety Scorer evaluates these hard constraints at each waypoint to detect failures.
C.1 HARD SAFETY COMPLIANCE TERM (H(τ)): mNC penalizes at-fault collisions, scoring 1.0 without one, 0.5 for an at-fault static-object collision, and 0.0 for other at-fault collisions.An at-fault collision includes the ego vehicle’s front colliding with any object or colliding with a static object.
C.1 HARD SAFETY COMPLIANCE TERM (H(τ)): mDAC is binary: it scores 1.0 when the entire vehicle footprint stays within the drivable area and 0.0 otherwise.The metric enforces strict compliance with the legally designated drivable area.

C.2 PERFORMANCE QUALITY TERM (Q(τ))

Q(τ) measures the quality of trajectories that pass hard safety checks. It combines normalized weighted performance metrics covering progress, time-to-collision, and comfort.

C.2 PERFORMANCE QUALITY TERM (Q(τ)): Q(τ) evaluates trajectories after hard safety checks using a normalized weighted sum of performance metrics.The quality term applies only to trajectories that have passed the hard safety checks.
C.2 PERFORMANCE QUALITY TERM (Q(τ)): Q(τ) combines mEP, mTTC, and mC using weights wEP, wTTC, and wC.The term is expressed as Q(τ) = wEP · mEP(τ) + wTTC · mTTC(τ) + wC · mC(τ), normalized by the sum of the weights.
C.2 PERFORMANCE QUALITY TERM (Q(τ)): mEP measures progress along the intended high-level route and is normalized to [0, 1] using a scene-specific feasible upper bound.The metric captures longitudinal route progress relative to what is feasible in the scene.
C.2 PERFORMANCE QUALITY TERM (Q(τ)): mTTC provides a binary temporal-safety score, awarding 1.0 above the predefined threshold and 0.0 below it.The example safe threshold is 2.0 seconds, and the supplied weight is wTTC = 5.
C.2 PERFORMANCE QUALITY TERM (Q(τ)): mC is a binary comfort score determined by whether longitudinal and lateral acceleration and jerk remain within predefined limits.It scores 1.0 when all dynamic quantities stay within comfort limits and 0.0 when any limit is exceeded; the supplied weight is wC = 2.

D LIMITATIONS & FUTURE WORK

ReflectDrive’s limitations concern input information, reflection design, discrete trajectory representation, and efficiency, while future work targets richer observations, improved rewards, and faster inference.

Model Inputs: Single-frame, three-view inputs omit surrounding-vehicle velocity, motion direction, and speed, motivating historical images and richer information.The authors propose also predicting key-obstacle trajectories to support reward modeling and later game-theoretic interaction planning.
Reflection: More reflection iterations and online inference attempts increase inference time but do not necessarily improve final performance.The work uses the PDM scorer without task-specific adjustments for rapid validation and suggests reducing unnecessary iterations.
Reflection: Failure cases include boundary-violation and collision-avoidance oscillation in limited drivable space, likely due to discrete trajectory-representation errors.The authors identify alternative representations or mitigation methods as future work.
Reflection: Navigation errors arise because the reward function omits navigation correctness, causing incorrect correction directions in some scenarios.Iterative reward refinement and replacement of the rule-based reward with a model-based reward are proposed.
Reflection: Suboptimal goal-point selection limits correction capability when the search range is constrained.The authors suggest improving the base model through reinforcement learning or other advanced techniques, while noting that internalized search may introduce corner cases.
Sample Efficiency: Algorithm optimization and acceleration received limited attention because the study focused on method validation, leaving substantial room for improvement.KV-cache engineering is proposed to improve computational efficiency.
Conclusion: Despite these limitations, extensive experiments demonstrate ReflectDrive’s closed-loop planning capabilities and potential for safety-oriented driving behavior.The authors characterize the planner as high-performance and highly adaptable for autonomous driving systems.