PaperlayerView exact revision on arXiv

Source-linked AI summary

DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving

Yingyan Li, Shuyao Shang, Weisong Liu, Bing Zhan, Haochen Wang, Yuqi Wang, Yuntao Chen, Xiaoman Wang, Yasong An, Chufeng Tang, Lu Hou, Lue Fan, Zhaoxiang Zhang

arXiv:2510.12796v2cs.CVcs.AI

TL;DR

Large-scale VLA training is limited by sparse action supervision, motivating DriveVLA-W0’s use of future-image world modeling as dense self-supervision. The approach spans discrete-token and continuous-feature VLAs, adds a lightweight action expert, and reports stronger scaling and generalization, including amplified gains on massive datasets.

  • Problem

    VLA models receive sparse, low-dimensional action supervision despite their large capacity, creating a supervision deficit that limits rich world-representation learning.

  • Method

    DriveVLA-W0 supplements action supervision with future-image world modeling, using autoregressive prediction for discrete visual tokens and diffusion generation for continuous visual features.

  • Results

    World modeling improves generalization and amplifies data scaling, with 70M-frame training improving the VQ model’s ADE by 28.8% and the ViT model’s collision rate by 15.9%.

  • Takeaways & Limitations

    Dense predictive world modeling provides a scaling advantage that action-only supervision cannot replicate, while the MoE Action Expert reduces inference latency to 63.1% of baseline.

Abstract

from arXiv · show

Scaling Vision-Language-Action (VLA) models on large-scale data offers a promising path to achieving a more generalized driving intelligence. However, VLA models are limited by a ``supervision deficit'': the vast model capacity is supervised by sparse, low-dimensional actions, leaving much of their representational power underutilized. To remedy this, we propose \textbf{DriveVLA-W0}, a training paradigm that employs world modeling to predict future images. This task generates a dense, self-supervised signal that compels the model to learn the underlying dynamics of the driving environment. We showcase the paradigm's versatility by instantiating it for two dominant VLA archetypes: an autoregressive world model for VLAs that use discrete visual tokens, and a diffusion world model for those operating on continuous visual features. Building on the rich representations learned from world modeling, we introduce a lightweight action expert to address the inference latency for real-time deployment. Extensive experiments on the NAVSIM v1/v2 benchmark and a 680x larger in-house dataset demonstrate that DriveVLA-W0 significantly outperforms BEV and VLA baselines. Crucially, it amplifies the data scaling law, showing that performance gains accelerate as the training dataset size increases.

1 INTRODUCTION

DriveVLA-W0 addresses the supervision deficit limiting VLA scaling by adding dense future-image prediction to sparse action supervision. Across architectures and data scales, world modeling supports generalization, amplifies scaling, and enables lower-latency action generation.

  • Motivation: VLA models map high-dimensional sensory inputs to sparse, low-dimensional actions, creating a supervision deficit that limits rich world-representation learning.The passage states that increasing action-only data cannot overcome this limitation and that large VLAs can underperform smaller BEV models.
  • Approach: DriveVLA-W0 uses future-image prediction as a dense self-supervised objective that supplements action supervision and encourages learning environmental dynamics.The objective provides supervision at every timestep and builds a predictive world representation.
  • Approach: The paradigm supports discrete-token VLAs with an autoregressive world model and continuous-feature VLAs with a diffusion world model.The diffusion model generates future image pixels conditioned on current-frame vision and action features.
  • Results: World modeling improves generalization across academic benchmarks and amplifies data scaling on a 70M-frame in-house dataset.The reported benefits are attributed to dense visual supervision from future-frame prediction rather than merely increasing action-only data.
  • Deployment and analysis: A lightweight MoE-based Action Expert reduces inference latency to 63.1% of the baseline VLA and enables large-scale action-decoder comparisons.The study reports that simpler autoregressive decoders can surpass flow-matching decoders at massive data scale, reversing the smaller-scale trend.

2 RELATED WORK

Related driving VLM research progressed from language-based interpretation toward integrated VLA architectures. World-model research likewise includes both data synthesis and auxiliary representation-learning objectives.

  • VLAs in Autonomous Driving: Early driving VLMs used language models for scene explanations and high-level maneuver suggestions without producing directly executable actions.This work is described as the first stage in the development of VLMs for autonomous driving.
  • World Models in Driving and Robotics: World-model research in driving and robotics follows two philosophies: synthesizing data for simulation or learning representations through an auxiliary objective.The first stream aims to generate high-quality driving data and realistic future scenarios.
  • World Models in Driving and Robotics: Examples of generative world models include GAIA-1, DrivingGPT, and Doe-1, while Copilot4D predicts future discrete visual tokens.The cited works illustrate the data-synthesis and discrete-token prediction directions described in the passage.

3 METHODOLOGY

DriveVLA-W0 combines dense world-model supervision with VLA action prediction, supporting discrete-token and continuous-feature paradigms. A lightweight Action Expert then enables efficient action decoding through an MoE design.

  • Overview: The methodology first establishes a VLA baseline, adds world modeling for dense self-supervision, and introduces a lightweight Action Expert for real-time inference.The training sequence moves from sparse action supervision to world-model pretraining and then action-specialist integration.
  • VLA Baseline: VLA variants use either quantized discrete visual tokens or continuous visual features, with language, images, and past actions interleaved across a history window.The discrete-token variant uses an Emu3-style backbone, while the continuous-feature variant uses a Qwen2.5-VL-style backbone.
  • AR World Model: The AR World Model autoregressively predicts visual-token sequences conditioned on preceding multimodal context and jointly optimizes action and world-model losses.DriveVLA-W0 (VQ) uses a weighted sum of action prediction and AR world-model losses; generated tokens can be decoded into images for visualization.
  • Diffusion World Model: The Diffusion World Model denoises future-image latents conditioned on current visual and action features, using a joint action-plus-diffusion objective.Predicting the future frame, rather than reconstructing the present, encourages learning predictive dynamics.
  • Action Expert: The MoE pairs the full VLA Expert with a 500M lightweight Action Expert, fusing their representations through Joint Attention for efficient control.The framework compares query-based, autoregressive, and flow-matching action experts; all variants prefill previous-action features as a temporal prior.

4 EXPERIMENT

Experiments across NAVSIM and a 70M-frame in-house dataset show that world modeling improves generalization, scaling, and deployment efficiency across VLA settings.

  • 4.3 COMPARISON WITH STATE-OF-THE-ART METHODS: DriveVLA-W0 establishes state-of-the-art performance on NAVSIM v1 and v2, surpassing BEV-based WoTE and VLA-based AutoVLA.It achieves this using only a single front-view camera, while some competitors use multi-view cameras and LiDAR.
  • 4.4 WORLD MODELS AMPLIFY DATA SCALING LAW: World modeling enables transfer from NuPlan pretraining to NAVSIM despite differing action distributions and similar visual domains.Sparse-supervision baselines exhibit a detriment from pretraining, whereas VLA-W0s obtain positive knowledge transfer.
  • 4.4 WORLD MODELS AMPLIFY DATA SCALING LAW: At 70M frames, world modeling improves the VQ model’s ADE by 28.8% and the ViT model’s collision rate by 15.9%.Action-only baselines saturate, while DriveVLA-W0 models continue improving with dataset size.
  • 4.4 WORLD MODELS AMPLIFY DATA SCALING LAW: Action experts reverse their relative performance with scale: query-based and flow-matching experts excel on small data, while autoregressive decoding scales best on the massive dataset.The comparison reflects a trade-off between prediction precision and modeling capacity, with query-based and flow-matching experts limited by representational bottlenecks or sample inefficiency at scale.
  • 4.5 ABLATION STUDY: Interleaved vision-action pretraining raises PDMS from 84.1 to 85.6 over vision-only pretraining, while longer temporal context improves performance.The 6VA configuration grounds visual prediction in ego actions and better captures long-horizon dynamics.
  • 4.5 ABLATION STUDY: A query-based MoE action expert reduces latency from 117.8ms to 74.3ms and increases PDMS from 85.6 to 88.4 on an H200 GPU.The reduced latency is 63.1% of the baseline latency.
  • 4.6 VISUALIZATION: DriveVLA-W0 generates action-conditioned futures whose visual dynamics align with counterfactual deceleration, supporting its potential as a reactive simulator.The surrounding scene flows past the vehicle at a reduced rate relative to ground truth.

5 CONCLUSION

DriveVLA-W0 addresses the supervision deficit in Vision-Language-Action models through future image prediction as a dense, self-supervised objective across VQ- and ViT-based architectures. Experiments show improved data scalability and generalization, alongside a scale-dependent reversal in which simpler autoregressive action decoders prevail.

  • DriveVLA-W0 uses future image prediction as a dense, self-supervised objective for both VQ- and ViT-based architectures.
  • The approach improves data scalability and generalization compared with baselines.
  • At scale, simpler autoregressive action decoders ultimately outperform other action decoders.

B MORE EXPERIMENTS

Table 7 reports detailed results for the experiment studying whether world modeling enhances generalization to new action distributions.

  • Table 7 presents detailed results for Figure 4.
  • The experiment examines world modeling's effect on generalization to new action distributions.
  • The table focuses on results associated with differing action distributions.

B.1 LATENCY ANALYSIS

The latency analysis compares action experts with the full VLA backbone and shows that query-based and flow-matching experts maintain constant inference time, while autoregressive latency depends on generated token count.

  • The AR expert and VLA baseline scale linearly with the number of generated tokens L, unlike flow-matching and query-based experts.
  • The query-based expert maintains approximately 74ms latency, while the flow-matching expert remains constant at around 145ms.
  • With 5.6-token NAVSIM trajectories, the AR expert reaches 95ms versus 118ms for the full backbone.
  • With 17.8-token in-house trajectories, AR latency rises to 170ms but remains below the baseline's 240ms.

B.3 ABLATION STUDY

The ablation study evaluates the world model's temporal input horizon using a shared 6VA-pretrained checkpoint and three configurations with different historical visual context.

  • The study identifies the optimal temporal horizon for the world model's input.
  • All configurations use a shared 6VA-pretrained checkpoint before fine-tuning and evaluation.
  • The configurations vary historical context from only the current frame to an additional frame from 1 or 4 seconds earlier.

C.1 PLANNING TRAJECTORY

DriveVLA-W0 improves trajectory planning in complex interaction scenarios, while its autoregressive action expert produces more stable trajectories than flow matching. A 1-second temporal interval offers the best world-model input trade-off.

  • The autoregressive action expert generates markedly more stable trajectories than the flow-matching expert.The comparison emphasizes inter-frame consistency and avoidance of aggressive maneuvers.
  • World modeling helps DriveVLA-W0 avoid collisions in complex interaction scenarios where TransFuser and VLA baselines often fail.Its predictive capabilities support planning in scenes requiring interaction-aware dynamics reasoning.
  • A 1-second temporal interval is optimal for world-model inputs.The current frame alone lacks temporal context, whereas a 4-second interval introduces excessive scene variation.

C.2 FAILURE CASE

The reported failures center on ambiguous navigation instructions and inaccurate prediction of dynamic objects. These weaknesses arise in complex topologies and high-interaction scenes.

  • Instruction ambiguity: Coarse-grained commands such as “go straight” can leave the model undecided at Y-shaped intersections.The ambiguous instruction does not clearly select either branch, causing the vehicle to enter the fork area hesitantly.
  • Action-expert behavior: Flow-matching action-expert trajectories are relatively unstable and may extend beyond the drivable area.The autoregressive action expert produces more stable trajectories in the cited comparison.
  • Dynamic prediction errors: The world model can fail to anticipate oncoming vehicles at complex intersections.The resulting planner may execute an incorrect left turn because it is unaware of the potential conflict.

C.3 FUTURE IMAGE GENERATION

DriveVLA-W0 generates realistic, plausible future images across challenging driving scenes and couples these predictions with planning. Counterfactual futures also reflect learned scene geometry.

  • Future image generation: Its visual predictions are tightly coupled with the model’s planning process.The paper reports both high visual fidelity and strong action consistency.
  • Future image generation: The world model generates visually realistic and contextually plausible futures in complex intersections and dense traffic.The cited examples emphasize generative fidelity across diverse challenging scenarios.
  • Counterfactual reasoning: Conditioning on a “turn right” action produces realistic off-road imagery when the ground-truth trajectory is straight.This counterfactual behavior is presented as evidence that the model captures underlying scene geometry.
  • Counterfactual reasoning: Counterfactual generation tests alternative futures based on hypothetical actions rather than only reproducing the ground-truth trajectory.The cited case changes the vehicle’s future according to the conditioned action.

D MORE IMPLEMENTATION DETAILS

The implementation combines a latent BEV TransFuser baseline with a multi-scale cross-attention fusion module and a waypoint planning head. The related-work context situates this design within broader scaling-law research.

  • TransFuser: Latent TransFuser replaces real LiDAR features with BEV latent queries while using the camera branch for semantic and appearance features.The latent BEV branch supplies learnable global query anchors that complement the camera representation.
  • TransFuser: Multi-scale cross-attention aligns and fuses image and BEV features into a representation for planning.The fused representation is denoted Ffuse in the implementation description.
  • Planning head: A planning head predicts K future waypoints from the fused representation Ffuse.The waypoint sequence is described as {p̂t+1, . . . , p̂t+K}.
  • Scaling-law context: Deep-learning scaling-law work relates performance or loss to model size, dataset size, and computational cost.The passage cites Kaplan’s power-law formulation and Chinchilla’s compute-optimal scaling prescription.
Loading 2510.12796v2…