PaperlayerView exact revision on arXiv

Source-linked AI summary

EponaV2: Driving World Model with Comprehensive Future Reasoning

Jiawei Xu, Zhizhou Zhong, Zhijian Shu, Mingkai Jia, Mingxiao Li, Jia-Wang Bian, Qian Zhang, Kaicheng Zhang, Jin Xie, Jian Yang, Wei Yin

arXiv:2605.14696v1cs.CV

TL;DR

Manual annotations constrain the scalability of perception-based driving models, while perception-free world models rely on limited next-frame prediction for future reasoning. EponaV2 predicts future depth and semantic maps and uses flow matching GRPO for planning. It reports stronger performance than perception-free models on NAVSIMv1 and NAVSIMv2 comparisons.

  • Problem

    Perception-based driving models rely on expensive manual annotations that constrain data scaling, while perception-free models’ future reasoning relies on next-frame prediction.

    Source: §1 · pp. 1–2

  • Method

    EponaV2 predicts future depth and semantic maps for richer future representations and uses flow matching GRPO to refine its planner.

    Source: §1 · p. 2

  • Results

    EponaV2 improves over perception-free models by +1.3 PDMS on NAVSIMv1 and +5.5 EPDMS on the navhard split of NAVSIMv2.

    Source: §4.2 · p. 8

  • Takeaways & Limitations

    Future geometry and semantic predictions provide actionable information for trajectory planning, and the paper reports improved driving performance from this approach.

    Source: §1 · p. 2

Abstract

from arXiv · show

Data scaling plays a pivotal role in the pursuit of general intelligence. However, the prevailing perception-planning paradigm in autonomous driving relies heavily on expensive manual annotations to supervise trajectory planning, which severely limits its scalability. Conversely, although existing perception-free driving world models achieve impressive driving performance, their real-world reasoning ability for planning is solely built on next frame image forecasting. Due to the lack of enough supervision, these models often struggle with comprehensive scene understanding, resulting in unsatisfactory trajectory planning. In this paper, we propose EponaV2, a novel paradigm of driving world models, which achieves high-quality planning with comprehensive future reasoning. Inspired by how human drivers anticipate 3D geometry and semantics, we train our model to forecast more comprehensive future representations, which can be additionally decoded to future geometry and semantic maps. Extracting the 3D and semantic modalities enables our model to deeply understand the surrounding environment, and the future prediction task significantly enhances the real-world reasoning capabilities of EponaV2, ultimately leading to improved trajectory planning. Moreover, inspired by the training recipe of Large Language Models (LLMs), we introduce a flow matching group relative policy optimization mechanism to further improve planning accuracy. The state-of-the-art (SOTA) performances of EponaV2 among perception-free models on three NAVSIM benchmarks (+1.3PDMS, +5.5EPDMS) demonstrate the effectiveness of our methods.

1 Introduction

EponaV2 addresses the annotation costs that constrain scaling perception-based driving models and the limited future reasoning of image-prediction-based alternatives. It predicts future depth and semantic maps and applies flow matching GRPO for trajectory planning.

1 Introduction

  • Perception-based driving models depend on costly manual annotations, making it difficult to apply data scaling laws and scale their planning capabilities.The cited examples of annotations include object bounding boxes and semantic segmentations.Source: §1 · p. 1
  • EponaV2 reports state-of-the-art planning among perception-free driving world models using comprehensive future predictions.The approach is presented as not relying on manual perception labels.Source: Figure 1 · p. 2, §1 · p. 2
  • EponaV2 predicts future depth and semantic maps to represent surrounding geometry and semantic context for trajectory planning without manual perception labels.Its semantic features are derived from large-scale segmentation models.Source: §1 · p. 2
  • Flow matching GRPO uses streamlined reward functions to refine planning results and improve generated-trajectory accuracy.The mechanism is introduced for EponaV2’s flow-based planner.Source: §1 · p. 2

2 Related Work

Related work contrasts perception-based driving systems, which use manual labels, with perception-free models that avoid them but often have limited self-supervision. EponaV2’s pipeline is framed as using future depth and semantic predictions for more comprehensive reasoning.

2 Related Work

  • Perception-based approaches use labels to build perception abilities for planning, while the illustrated EponaV2 pipeline uses future depth and semantic predictions.Figure 2 contrasts these with perception-free models that plan using future-frame prediction alone.Source: Figure 2 · p. 3
  • Perception-based work includes BEV planning methods and VLM-based approaches, which rely on manual annotations for auxiliary perception tasks.Examples include occupancy prediction and visual question answering.Source: §2 · p. 3
  • Perception-free methods avoid costly manual labels through latent world models or future-image self-supervision, but can struggle with complex environments under limited self-supervision.The passage names World4Drive, Epona, DriveVLA-W0, DriveLaW, and PWM as examples.Source: §2 · p. 3

3 Methods

EponaV2 forecasts future representations for trajectory planning, supervising future geometry and semantics before fine-tuning its planner with GRPO.

3.1 Applying World Model to Driving with Flow Matching

  • DINO-Tok encodes video frames, a VLM-initialized backbone infers future representations, and a rectified flow-matching planner converts them into trajectories.The model also predicts the next frame from inferred future representations as a coarse world-model objective.Source: §3.1 · p. 4, Figure 3 · p. 5

3.2 Forecasting to Reason the Real World

  • Future metric-depth and confidence maps are decoded from inferred representations and supervised with pseudo metric-depth labels, without extra trajectory-inference cost.The depth objective is intended to make future reasoning account for 3D structure and object movement.Source: §3.2 · p. 5, Figure 3 · p. 5
  • A lightweight semantic head predicts text-aligned future-frame feature maps generated by SAM3, using object descriptions to direct attention to driving-relevant elements.The predicted features use text-fused SAM3 image features as supervision.Source: §3.2 · pp. 5–6, Figure 3 · p. 5

3.3 Flow Matching Group Relative Policy Optimization for Planner

  • GRPO fine-tunes the flow-matching planner using simulated-trajectory discrepancy rewards and an imitation loss that keeps outputs close to dataset trajectories.The authors avoid PDMS as a reward because its calculation requires manual perception labels.Source: Figure 3 · p. 5, §3.3 · p. 6
  • Training proceeds in two stages: first optimize the world model and prediction heads, then freeze other components and fine-tune the trajectory planner with GRPO.The first-stage objective combines trajectory, image, depth, and semantic losses.Source: Figure 3 · p. 5, §3.3 · p. 6

4 Experiments

EponaV2 performs competitively among perception-free models across NAVSIM benchmarks, while ablations associate future depth and semantic prediction and flow matching GRPO with distinct planning benefits. The experiments use three NAVSIM evaluations and downscaled models for the ablations.

4.1 Setup

  • Evaluation covers NAVSIMv1 navtest, NAVSIMv2 navtest and NAVSIMv2 navhard, using PDMS for NAVSIMv1 and EPDMS for NAVSIMv2.NAVSIMv2 EPDMS adds traffic-light compliance and lane keeping, and expands comfort into history and extended comfort.Source: §4.1 · p. 7
  • The 6.7B-parameter model uses a Qwen3-VL 4B backbone and is trained in two stages, with planner fine-tuning on NAVSIM navtrain.Training takes approximately 10 days across 64 H20 GPUs; evaluation uses one RTX 4090 GPU.Source: §4.1 · p. 8

4.2 Comparison

  • +1.3 improvement in PDMS over other perception-free models and +5.5 improvement in EPDMS on NAVSIMv2 navhard; EponaV2 is also comparable to perception-based models on NAVSIMv1.The NAVSIMv2 comparisons include longer simulation horizons.Source: §4.2 · p. 8, Table 3 · p. 8

4.3 Ablation Study

  • Depth loss improves NC, DAC and TTC, while semantic-map loss helps prevent collisions and guide safe, rule-compliant trajectories in the downscaled ablation.The ablation progressively adds future depth, image and SAM3-generated semantic-map prediction losses.Source: Table 4 · p. 8, §4.3 · p. 9
  • Future-depth prediction outperforms present-depth prediction for planning when only the depth predictor supervises the downscaled model.Both settings use the Qwen3-VL 2B backbone.Source: §4.3 · p. 9, Table 5 · p. 9
  • Removing flow matching GRPO reduces planning accuracy, with the ablation reporting notable improvement in EP when GRPO is included.The GRPO mechanism further optimizes the trajectory planner using reinforcement-learning techniques.Source: §4.3 · p. 9, Table 6 · p. 9

5 Conclusion

EponaV2 predicts future depth and semantic maps to support real-world future reasoning and trajectory planning, then uses flow matching GRPO to refine the planner. The paper reports improved planning performance, while noting that imprecise pseudo labels limit performance relative to perception-based models.

5 Conclusion

  • EponaV2 predicts future depth and semantic maps to represent surrounding geometry and semantic context for trajectory planning.The conclusion describes forecasting this 3D and semantic information as supporting enhanced planning performance.Source: §5 · p. 9
  • Flow matching group relative policy optimization further refines the trajectory planner and improves overall planning accuracy.Source: §5 · p. 9
  • Imprecision in pseudo depth and semantic labels from pretrained foundation models is cited as the main reason EponaV2 does not yet surpass state-of-the-art perception-based models.Source: §5 · p. 9

Related papers

Loading 2605.14696v1…