Source-linked AI summary
DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving
Chenxu Dang, Sining Ang, Yongkang Li, Haochen Tian, Jie Wang, Guang Li, Hangjun Ye, Jie Ma, Long Chen, Yan Wang
TL;DR
Diffusion and token-based VLA planners each have important weaknesses in autonomous driving, including alignment, efficiency, generalization, causal-error, and irreversible-decoding challenges. DriveFine addresses this gap with a masked-diffusion VLA using decoupled block-MoE generation and refinement experts plus hybrid reinforcement learning, achieving state-of-the-art results across NAVSIM v1, v2, and Navhard.
Problem
Diffusion-based planners face modality-alignment, training-efficiency, and generalization limitations, while token-based planners suffer cumulative causal errors and irreversible decoding.
Method
DriveFine combines a masked-diffusion VLA with plug-and-play block-MoE generation and refinement experts, explicit expert selection, gradient blocking, and hybrid reinforcement learning.
Results
DriveFine achieves state-of-the-art performance on NAVSIM v1, NAVSIM v2, and Navhard benchmarks.
Takeaways & Limitations
The results support masked diffusion with decoupled refinement as an effective and robust approach for improving token-based VLA driving performance.
Takeaways & Limitations
Fully decoupling generation and refinement can dramatically increase parameters, while coupling them causes mutual interference and hinders task-specific optimization.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models for autonomous driving increasingly adopt generative planners trained with imitation learning followed by reinforcement learning. Diffusion-based planners suffer from modality alignment difficulties, low training efficiency, and limited generalization. Token-based planners are plagued by cumulative causal errors and irreversible decoding. In summary, the two dominant paradigms exhibit complementary strengths and weaknesses. In this paper, we propose DriveFine, a masked diffusion VLA model that combines flexible decoding with self-correction capabilities. In particular, we design a novel plug-and-play block-MoE, which seamlessly injects a refinement expert on top of the generation expert. By enabling explicit expert selection during inference and gradient blocking during training, the two experts are fully decoupled, preserving the foundational capabilities and generic patterns of the pretrained weights, which highlights the flexibility and extensibility of the block-MoE design. Furthermore, we design a hybrid reinforcement learning strategy that encourages effective exploration of refinement expert while maintaining training stability. Extensive experiments on NAVSIM v1, v2, and Navhard benchmarks demonstrate that DriveFine exhibits strong efficacy and robustness. The code will be released at https://github.com/MSunDYY/DriveFine.
1 Introduction
Autonomous-driving VLAs face complementary weaknesses: diffusion planners offer parallel refinement but suffer alignment, training-efficiency, and generalization problems, while token-based planners offer unified decoding but accumulate irreversible errors. DriveFine combines masked-diffusion flexibility with token-VLA refinement through decoupled block-MoE experts and hybrid reinforcement learning.
- Diffusion-based VLAs: Diffusion-based VLAs use iterative trajectory refinement and parallel decoding, but cross-modal alignment difficulties hinder training efficiency.Their additional diffusion Transformer typically requires hundreds of training epochs.
- Diffusion-based VLAs: PDMS-oriented reinforcement fine-tuning can substantially reduce diffusion planners’ EPDMS, reflecting limited robustness and generalization.The paper attributes this degradation to weak coupling between diffusion planners and the VLM, which can induce reward hacking and pretrained-knowledge loss.
- Token-based VLAs: Token-based VLAs unify vision, language, and action through autoregressive action-token decoding, but causal attention and fixed token-by-token generation are costly and error-prone.Their irreversible decoding means committed tokens cannot be modified, so point-level deviations can cause collision or off-road trajectory failures.
- Masked diffusion LLMs: Masked diffusion LLMs provide flexible decoding orders, yet early tokens may lack global consistency and remain unrevisable, causing trajectory-level failures.Unlike token decoding, diffusion planners iteratively refine trajectories.
- DriveFine: DriveFine injects token-VLA refinement into a pretrained masked diffusion planner using block-MoE expert selection and gradient isolation.Generation and refinement experts are separated during inference and training to preserve foundational capabilities and avoid cross-task interference.
- DriveFine: A hybrid online-offline reinforcement-learning strategy trains generation with GRPO while jointly supervising refinement with offline and online trajectory pairs.The approach aims to improve trajectory quality with limited parameter growth and slight inference overhead.
- Results: DriveFine achieves state-of-the-art performance across NAVSIM v1, NAVSIM v2, and Navhard benchmarks.The paper reports extensive experiments demonstrating consistent performance across these benchmarks.
2 Related Work
Related work progresses from deterministic planners to generative VLA systems and reinforcement-learning-based planning. Recent approaches improve trajectory diversity, but diffusion and token-based planners retain distinct limitations in robustness, efficiency, and decoding reversibility.
- Deterministic planners imitate single expert trajectories, limiting their ability to handle distribution shift and multimodal driving behavior.
- Diffusion policies model trajectory uncertainty and diversity, while token-based and diffusion-based planners represent alternative generative approaches for autonomous driving.
- Vision-language models evolved from scene understanding and reasoning toward VLA systems that generate meta-actions, low-frequency actions, or trajectories.
- Reinforcement learning addresses imitation learning’s lack of counterexample supervision, with methods progressing from DPO and GRPO to diffusion-specific and scoring-based strategies.
3 Method
DriveFine combines masked diffusion trajectory generation with a decoupled block-MoE refinement expert and hybrid reinforcement learning. Its pipeline uses shared representations, task-specific expert activation, and offline-online advantages to refine generated trajectories.
- 3.1 Diffusion LLM for Planning: DriveFine aligns visual and textual inputs, discretizes trajectories into language tokens, and generates actions through masked diffusion with iterative parallel unmasking.Spatial and heading ranges are discretized into bins appended to the language vocabulary.
- 3.1 Diffusion LLM for Planning: Random masking trains the generation expert with masked cross-entropy to reconstruct original trajectory tokens from contextual and noised sequences.The formulation uses sequence length L, contextual token p0, original sequence r0, and noised sequence rt.
- 3.2 Block-level MoE for Refinement: Block-MoE replicates the last n pretrained dLLM blocks as refinement blocks while sharing preceding blocks and the visual tower with generation.The design addresses interference, foundational-capability loss, and parameter overhead associated with coupled or fully separate experts.
- 3.2 Block-level MoE for Refinement: During inference, shared blocks produce context and manually activated expert blocks perform generation or refinement; during training, refinement gradients remain confined to refinement experts.This decoupling preserves pretrained knowledge and keeps refinement plug-and-play and transferable.
- 3.3 Reinforcement Finetuning: The generation expert samples trajectory groups with GRPO, while the refinement expert uses generated trajectories as offline anchors and samples refined trajectories for online exploration.Offline pairwise reward differences and online refined trajectories jointly supervise refinement.
- 3.3 Reinforcement Finetuning: The hybrid reinforcement-learning pipeline combines offline and online advantages to train the refinement expert while retaining the generation expert’s sampling process.The offline and online advantages are jointly combined into a hybrid advantage.
4 Experiments
DriveFine is evaluated across NAVSIM v1, NAVSIM v2, Navhard, ablations, robustness, and efficiency studies. It achieves strong benchmark performance while refinement improves trajectory quality and masked diffusion offers an efficiency–performance trade-off.
- Main Results: DriveFine achieves 91.9 PDMS with score-based reinforcement fine-tuning on NAVSIM v1, surpassing existing VLA planners.
- Main Results: 89.7 EPDMS on bug-fixed NAVSIM v2 establishes a new state of the art, while 86.7 EPDMS on the earlier version surpasses DriveVLA-W0 by 0.5 points.
- Main Results: On Navhard without additional training, DriveFine outperforms diffusion-based methods on both stage metrics and improves Stage 1 EPDMS over ReCogDrive by 5.5 points.
- Ablations: PDMS rises from 86.7 with SFT to 89.6 with GRPO, 0.7 points further with offline refinement training, and 90.8 with online reinforcement training.
- Ablations: Refinement corrects collision and off-road anomalies caused by individual tokens and mitigates trajectory fluctuations from noncausal decoding.
- Sensitivity: A refinement block adds 0.4 PDMS, performance improves with more blocks, and GRPO reaches its best performance at group size G = 8.
- Efficiency: With 4 diffusion steps, DriveFine reaches 90.47 PDMS at 207 ms average latency, comparable to ReCogDrive-8B and representing an efficiency–performance balance.
5 Conclusion
The paper analyzes diffusion-based and token-based VLA planners and proposes DriveFine to combine masked diffusion decoding with refinement capabilities. Experiments across NAVSIMv1, NAVSIMv2, and Navhard demonstrate effectiveness and robustness through comparative and ablation studies.
- The paper analyzes the strengths and limitations of diffusion-based and token-based planners for autonomous driving.
- DriveFine combines a plug-and-play block-MoE architecture with hybrid reinforcement training to inject refinement capabilities into token-based VLAs.
- Experiments on NAVSIMv1, NAVSIMv2, and Navhard, together with ablations and comparative analyses, demonstrate effectiveness and robustness.