PaperlayerView exact revision on arXiv

Source-linked AI summary

DA-WAM: Decision-Aligned Future Latents for Driving World Models

Ruiguo Zhong, Benshan Ma, Xiaolong Chen, Lang Zhang, Mingyue Feng, Yaonong Wang, Pei Liu, Jun Ma

arXiv:2608.19085v1cs.ROcs.AI

TL;DR

World models must make action-specific future predictions informative for trajectory selection, rather than merely predictive. DA-WAM aligns future latent learning, counterfactual prediction, and scoring, achieving the best overall planning scores among compared learning-based planners on NAVSIM-v1 and NAVSIM-v2.

  • Problem

    World-model predictions are not consistently coupled to candidate-level trajectory scoring, limiting whether action-specific future consequences inform autonomous-driving decisions.

  • Method

    DA-WAM jointly adapts predictive representations during planning, predicts a distinct future latent for each candidate, and scores candidates using expert-matched targets and safety-critical hard negatives.

  • Results

    93.7 PDMS on NAVSIM-v1 and 87.7 EPDMS on NAVSIM-v2 are the best overall planning scores among compared learning-based planners.

  • Takeaways & Limitations

    Action-specific future latents, jointly learned representations, and hard-negative supervision are supported as components of decision-aligned trajectory planning.

  • Takeaways & Limitations

    Because offline logs record only the executed future, dense predictive supervision is restricted to the expert-matched candidate rather than all counterfactual trajectories.

Abstract

from arXiv · show

Anticipating how scenes evolve under ego actions is fundamental to safe autonomous driving, yet the full potential of world models for decision-making remains unrealized. The critical challenge lies in ensuring that future modeling is not merely predictive, but decision-informative: the predicted future must directly shape which trajectory is selected. Existing approaches decouple future representation learning from planning optimization, or share predicted states across trajectory candidates, thereby diluting the action-specific consequences that ought to guide selection. To bridge this gap, we propose DA-WAM, a framework that unifies predictive representation learning, action-conditioned future modeling, and trajectory scoring under a single decision-making objective. DA-WAM maintains predictive supervision throughout planner optimization via an online encoder and a stable momentum target, allowing future representations to co-evolve with the driving task. An action-conditioned predictor generates a distinct future latent state per trajectory candidate, which is then evaluated by a future-latent-conditioned factorized scorer. For the expert-matched trajectory, the predicted future latent is supervised by the observed future representation, while safety-critical hard negatives provide additional supervision near planning boundaries. Extensive experiments on NAVSIM-v1 and NAVSIM-v2 demonstrate state-of-the-art performance, while ablations and diagnostic analyses validate the key components.

1 INTRODUCTION

DA-WAM aligns future prediction with trajectory selection by predicting a distinct action-conditioned future latent for each candidate and conditioning its scorer on that latent. It jointly adapts predictive representations during planner optimization, using expert-matched targets and safety-critical hard negatives to distinguish candidates with different outcomes.

  • Candidate-level scoring: DA-WAM evaluates candidates against their own predicted futures so collisions, lane departures, and traffic-rule violations can influence scoring between geometrically similar trajectories.The introduction identifies candidate-level scoring as central to the planning value of a world model.
  • Safety-aware supervision: Its supervision combines observed future targets for the expert-matched candidate with safety-critical hard negatives near planning boundaries.Hard negatives are geometrically similar to the expert-matched candidate but differ in safety outcomes, discouraging geometry-only scoring.
  • Decision-aligned prediction: DA-WAM predicts a distinct future latent for each candidate trajectory and uses action-specific consequences to guide trajectory selection.This addresses mismatches caused by shared or weakly candidate-associated predicted futures.
  • Unified optimization: The framework continues predictive representation learning during planner optimization, allowing the latent space to adapt to the driving objective instead of remaining fixed after pretraining.It uses a LoRA-adapted V-JEPA 2.1 online encoder with an EMA target encoder.

2 RELATED WORK

Related work models future scene evolution with predictive latent representations and increasingly conditions predictions on ego actions or trajectory proposals. DA-WAM addresses remaining gaps by preserving predictive supervision during planner optimization and linking each candidate to its own future latent for scoring.

  • Predictive representations: JEPAs predict latent features of future states rather than reconstructing pixels, supporting efficient scene-evolution modeling for downstream ego-decision forecasting.DriveWorld and LAW apply future dynamics to enrich visual scene representations in end-to-end autonomous driving.
  • Predictive representations: Existing methods trade general predictive representations against trajectory-scoring needs through frozen pretraining, multistage pipelines, or inference-time removal.DA-WAM instead maintains predictive JEPA supervision throughout planner optimization.
  • Action-conditioned futures: Action-conditioned world models predict different futures under hypothetical maneuvers, including multiview video, trajectory-conditioned latent states, and BEV dynamics.Drive-WM, LAW, WoTE, and World4Drive represent this direction through alternative driving commands, trajectory proposals, or intention-guided forecasts.
  • Action-conditioned futures: Aggregating or weakly fusing predicted futures across proposals homogenizes scene representations and dilutes trajectory-specific safety consequences.DA-WAM establishes a one-to-one correspondence by generating a distinct future latent state for every candidate trajectory and feeding it directly into the scorer.
  • Candidate-based planning: Candidate-based planners generate multiple plausible trajectories and select among them using scene context and planning objectives.DiffusionDrive, DrivoR, GTRS, and ZTRS advance candidate generation or evaluation within this paradigm.
  • Candidate-based planning: Trajectory scoring remains predominantly anchored in current geometry and immediate motion, rarely using explicit per-candidate predicted futures to distinguish similar but safety-critical candidates.DA-WAM augments the established candidate-based planning paradigm by incorporating predicted futures directly into the scorer.

3 METHODOLOGY

DA-WAM couples action-conditioned future-latent prediction with candidate-specific trajectory scoring. Its training preserves predictive supervision while aligning observed futures, planning factors, utility, and safety-critical counterfactual supervision.

  • Architecture: DA-WAM encodes the current scene, predicts a distinct future latent for each trajectory candidate, and scores each candidate using its scene, action, and predicted future.A shared scorer predicts interpretable planning factors and an overall utility, selecting the highest-scoring candidate.
  • Expert Matching for Counterfactual Futures: Dense future-latent supervision is restricted to the candidate matched to the executed expert trajectory, while other candidates learn indirectly through trajectory-scoring losses.This avoids assigning the observed future to unexecuted counterfactual actions while retaining their influence on selection.
  • Predictive Representation Learning: The EMA target encoder provides stable future-representation targets, while only the adapted online encoder is deployed at inference.The target branch uses stop-gradient and momentum updates to prevent representational collapse.
  • Action-Conditioned Future Prediction: A shared predictor uses each action representation as a query over current scene tokens, producing candidate-specific latent futures without candidate-specific model biases.Differences among predicted futures are driven by the action queries rather than separate predictor parameters.
  • Trajectory Scoring and Safety Supervision: The scorer conditions decisions on each candidate’s predicted future latent and factorizes evaluation into collision, drivable-area, progress, time-to-collision, comfort, and utility outputs.Expert-proximate safety-critical hard negatives add counterfactual supervision near planning boundaries where geometry alone may be insufficient.

4 EXPERIMENTS

DA-WAM achieves the best overall planning scores on both NAVSIM benchmarks, with 93.7 PDMS on NAVSIM-v1 and 87.7 EPDMS on NAVSIM-v2. Matched ablations show that candidate-specific future prediction and hard-negative supervision improve planning performance, safety, and compliance.

  • NAVSIM-v1: On NAVSIM-v1, DA-WAM slightly surpasses the strongest prior learned planner, achieving 99.1 NC, 98.9 DAC, and 90.0 EP.The result balances collision avoidance, drivable-area compliance, and ego progress.
  • NAVSIM-v2: On NAVSIM-v2, DA-WAM reaches 97.9 TTC and 97.6 Lane Keeping, raising EPDMS to 87.7 and exceeding the strongest comparison by 0.2 points.The expanded metric set covers both time-to-collision and lane keeping.
  • Future-prediction ablation: The shared-global-future control lowers EP from 91.36 to 88.68 and produces the lowest PDMS of 92.81, indicating prediction–action mismatch.The no-future-prediction and current-latent controls achieve 93.31 PDMS and 93.25 PDMS, respectively.
  • Future-prediction ablation: Action conditioning raises PDMS to 93.46, while counterfactual safety supervision further increases it to 93.68 with higher NC, DAC, TTC, and Comfort.Adding hard negatives reduces EP from 90.47 to 89.97.
  • Qualitative analysis: In representative scenarios, DA-WAM follows the expert trajectory more closely in a large left turn and avoids conflicting agents in tight traffic and yielding conflicts.The baselines incur NC and TTC failures in the latter scenarios, whereas DA-WAM attains full scores across reported metrics.

5 CONCLUSION

DA-WAM makes world-model predictions directly informative for trajectory selection by jointly learning future representations with driving, modeling candidate-specific futures, and evaluating each candidate against its corresponding outcome. Expert-matched supervision preserves consistency with observed data, while safety-critical hard negatives improve discriminative learning.

  • DA-WAM jointly learns future representations with the driving task to connect prediction directly to trajectory selection.
  • The framework predicts a distinct future for each candidate trajectory and evaluates candidates against their corresponding outcomes.
  • Expert-matched supervision keeps future learning consistent with observed data, while safety-critical hard negatives improve discrimination.

A NOTATION

This section introduces the notation used throughout DA-WAM and directs readers to Table 6 for the formulation’s main symbols and meanings.

  • A NOTATION: Table 6 summarizes the main symbols used in the DA-WAM formulation.The table is titled “Main notation used in DA-WAM.”
  • A NOTATION: Readers can use Table 6 as the reference for DA-WAM notation.The passage identifies Table 6 as the summary of the formulation’s main symbols.
  • A NOTATION: The notation table pairs each symbol with its meaning.Table 6 is organized under the headings “Symbol” and “Meaning.”
Loading 2608.19085v1…