PaperlayerView exact revision on arXiv

Source-linked AI summary

VITAL: From Visual Motion to Precise Action

Ao Liang, Youquan Liu, Mingxiao Li, Wei Chow, Lingdong Kong, Yue Liao, Dongyue Lu, Yu Yang, Wei Yin, Runnan Chen

arXiv:2610.20604v1cs.CV

TL;DR

World-action models often train action experts on trajectory representations separate from visual dynamics, leaving action targets not directly derived from observed motion. VITAL pairs a visual motion scaffold with a normalized metric residual and jointly predicts both from current context; with one denoising step, it achieves strong NAVSIM scores.

  • Problem

    World-action models often define action targets independently of visual dynamics, motivating the question of whether action targets can be derived directly from observed visual motion.

    Source: §1 · p. 2

  • Method

    VITAL pairs a visually derived trajectory scaffold with a normalized residual for its remaining metric error, jointly predicting both codes from current context.

    Source: §1 · p. 2

  • Results

    With one denoising step, VITAL reaches 90.8 PDMS on NAVSIM v1, 91.1 EPDMS on v2, and 38.6 EPDMS on navhard.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The controlled reconstruction and action-learning studies indicate that visual grounding and metric fidelity are complementary properties of an action target.

    Source: §5 · p. 13

    Bounded residual refinement cannot replace a different maneuver when reactive continuations demand one, and clean-code precision does not remove conditional prediction error.

    Source: §4.3.1 · p. 11, §5 · p. 14

Abstract

from arXiv · show

World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics. This motivates deriving action targets directly from visual motion. We first examine a simpler setting in which trajectories are reconstructed from recorded future images. Even without the need for future prediction, reconstruction captures the dominant maneuver but leaves centimeter-scale errors. We therefore introduce VITAL (Vision-Induced Tokens for Action Learning), an action tokenizer that couples a visually derived motion scaffold with a compact residual encoding the trajectory error left by that scaffold. This representation makes visual motion the principal prediction target and gives small metric corrections a separate, normalized learning scale. Using flow matching, the action expert jointly predicts both components from the current image, ego state, and navigation command, without generating future images at inference. The same residual coordinates support bounded reinforcement post-training that refines execution while keeping the predicted visual scaffold fixed. With only one action denoising step, VITAL achieves 90.8 PDMS on NAVSIM v1 navtest, within 0.13 points of its ten-step score at 3.29x the end-to-end inference speed. Without retraining, it achieves 91.1 EPDMS on NAVSIM v2 navtest and 38.6 on navhard. Project page: https://worldbench.github.io/vital

1 Introduction

VITAL addresses the gap between visual dynamics and trajectory-defined action targets by pairing a visually derived motion scaffold with a normalized residual for its remaining metric error. It predicts both codes from current context and reports strong one-step NAVSIM results.

1 Introduction

  • World-action models connect visual prediction to control, but their action experts often use trajectory codes constructed separately from the video expert’s scene-evolution targets.The paper asks whether action targets can instead be derived directly from observed visual motion.Source: §1 · p. 2
  • Even with recorded future images available, trajectory reconstruction captures the dominant maneuver but leaves 123.6 mm ADE and 221.6 mm FDE across NAVSIM validation scenes.The recovered motion is smooth, with a 0.1% high-frequency velocity-energy fraction, and can miss metric detail near road boundaries.Source: §1 · p. 2
  • VITAL pairs a visual motion code that decodes to a trajectory scaffold with a normalized residual code for the demonstrated trajectory’s remaining error.With recorded images and a residual encoded from ground truth, reconstruction ADE/FDE fall to 0.3/0.5 mm, while PDMS rises from 92.1 to 94.7.Source: §1 · p. 2
  • Flow matching jointly predicts both codes from the current image, ego state, and navigation command; bounded residual updates then refine execution while keeping the predicted scaffold fixed.Inference does not generate future images, and residual refinement uses a single deterministic correction.Source: §1 · pp. 2–3
  • With equal action-training budgets, VITAL gains 5.1 PDMS points in one step over an autoencoder trained only on trajectories.Residual refinement brings the final policy to 90.8 PDMS in one step.Source: §1 · p. 3
  • 90.8 PDMS on NAVSIM v1 in one step is within 0.1 points of the ten-step score at 3.29× inference speed, including image encoding.Without retraining, VITAL also reaches 91.1 EPDMS on NAVSIM v2 and 38.6 EPDMS on reactive navhard.Source: §1 · p. 3

2 Related Work

Related work spans visual world-action models, latent and tokenized action representations, few-step generation, and policy refinement. VITAL’s distinction is to predict a visually grounded trajectory scaffold alongside its metric residual, then refine only that residual.

2.1 World-action models

  • VITAL constructs action targets from recorded visual motion and explicitly encodes the metric trajectory error left by the visual representation.Future images supervise target construction, while deployment predicts paired action codes from current context without generating future images or scoring candidates.Source: §2.1 · p. 4

2.2 Representation alignment and latent actions

  • Unlike visual-transition residuals, VITAL’s residual is the metric difference between a visually decoded trajectory scaffold and the demonstrated trajectory.Trajectory supervision grounds the scaffold in physical coordinates, and both codes are jointly predicted for planning.Source: §2.2 · p. 4

2.3 Action tokenization and reconstruction fidelity

  • VITAL assigns separate targets to visually grounded motion and the metric error remaining after scaffold reconstruction; clean-code reconstruction alone does not establish planning utility.The analysis separates clean-code reconstruction from conditional prediction, while downstream planning scores test control usefulness.Source: §2.3 · p. 5

2.4 Few-step action generation

  • VITAL retains the standard flow-matching objective but changes the action representation to jointly predict a scaffold and normalized metric residual in one step.This produces one executable trajectory without teacher distillation or candidate selection.Source: §2.4 · p. 5

2.5 Residual policy refinement

  • VITAL freezes the base action generator and decoder, restricting policy improvement to bounded corrections in the tokenizer’s metric-residual coordinates.One deterministic correction preserves the predicted visual scaffold and requires no extra denoising or reward-based search at deployment.Source: §2.5 · p. 6

3 Learning Precise Actions from Visual Motion

VITAL connects visual motion, action prediction, and reward-based refinement through a visual scaffold paired with a metric residual. The action expert predicts both components from deployment context, while refinement operates in the residual coordinates.

3.1 Problem Setup and Method Overview

  • Training pairs context with recorded future images and demonstrated trajectories, then fits the codec, learns a causal predictor, and optimizes a residual actor.At deployment, the planner predicts trajectories from context.Source: §3.1 · p. 6

3.2 Visual Motion and Metric Residual Tokenization

  • The visual tokenizer maps recorded image motion to a metric trajectory scaffold, while a standardized paired residual encodes the scaffold’s remaining error.Residual scaling gives small physical corrections an explicit learning scale.Source: §3.2 · p. 7

3.3 Joint Action Learning and One-Step Inference

  • Joint flow matching predicts visual and residual tokens from deployment features, and one Euler update decodes them into an executable trajectory.Future-video tokens are masked during action prediction.Source: §3.3 · pp. 7–8

3.4 Reconstruction and Prediction Analysis

  • The residual targets scaffold reconstruction error, with its retained span trading code size against fidelity; deployment prediction adds error beyond clean-code reconstruction.The analysis distinguishes retained information, conditional predictability, and numerical integration effects.Source: §3.4 · p. 8

3.5 Structure-Preserving Residual Policy Refinement

  • A residual actor refines the frozen policy’s trajectory within bounded residual coordinates while preserving its predicted visual tokens and scaffold.The correction is confined to a fixed subspace and does not guarantee improved reward or task-level safety for every correction.Source: §3.5 · p. 9

4 Experiments

Experiments assess one-step benchmark performance and transfer alongside representation precision, visual conditioning, joint action learning, residual refinement, and a qualitative driving scene.

4.2 Main Benchmark Results

  • 90.8 PDMS at one step is within 0.1 of the ten-step score and 3.29× faster; without retraining, VITAL scores 91.1 EPDMS on v2 navtest and 38.6 two-stage EPDMS on navhard.Ten evaluations add 0.1 PDMS on v1 and 0.1 EPDMS on v2 navtest; on navhard, VITAL scores 38.1 with ten evaluations.Source: §4.2 · pp. 10–11, Figure 4 · p. 11, Table 2 · p. 12

4.3.1 From Visual Motion to Metric Precision

  • Recorded-image reconstruction shows the visual scaffold captures the maneuver but leaves 0.12 m ADE; adding the ground-truth residual reduces ADE to 3.11 × 10−4 m.Both reconstructions use recorded future images, so the comparison measures retained codec precision rather than forecasting ability.Source: §4.3.1 · p. 11, Table 3 · p. 12

4.3.2 Visual Conditioning and Target Predictability

  • With matched vision, VITAL reaches 0.72 m ADE and 79.4 PDMS versus AE’s 0.80 m and 73.2 PDMS; shuffling vision raises VITAL’s ADE to 2.20 m and lowers PDMS to 52.3.The probes use equal-capacity ridge models and fixed frozen features.Source: §4.3.2 · p. 12, Table 4 · p. 13

4.3.3 Joint Alignment and Precision in Action Learning

  • 70.1 PDMS after one training pass gives VITAL a lead over AE (64.9) and raw trajectory prediction (55.0); its advantage over AE remains at ten evaluations, 71.4 versus 66.3 PDMS.AE scores higher on comfort at both budgets, while VITAL improves NC, DAC, EP, TTC, and DDC.Source: §4.3.3 · pp. 12–13, Table 5 · p. 13

4.3.4 Structure-Preserving Residual Post-Training

  • Residual post-training adds 0.6 PDMS at one evaluation and 0.7 at ten through local corrections, while freezing the action generator and predicted visual scaffold.On reactive continuations, refinement improves Stage 1 while slightly lowering Stage 2 EPDMS; local corrections cannot replace a changed maneuver when one is required.Source: §4.3.4 · p. 13

4.4 Qualitative Analysis

  • In the curved-road example, one-step VITAL scores 0.67 m ADE and 1.82 m FDE, versus 1.35 m and 3.77 m for ten-step SimWAM-RL.The comparison also includes SUV at ten steps and DiffusionDrive at two.Source: §4.4 · p. 13, Figure 5 · p. 13

5 Conclusion

VITAL pairs visual motion with a normalized metric residual and uses bounded residual reinforcement post-training to refine plans while preserving the predicted scaffold. One-step evaluation reaches the reported NAVSIM v1, v2, and navhard scores, while the authors identify limits to clean-code precision and local refinement.

5 Conclusion

  • VITAL pairs a visual motion scaffold with a normalized metric residual to address visual–action alignment and trajectory precision, then refines planning through bounded corrections that preserve the predicted scaffold.At deployment, both components are predicted from current observations without reward evaluation.Source: §5 · p. 13
  • With one denoising step, VITAL achieves 90.8 PDMS on NAVSIM v1, 91.1 EPDMS on v2, and 38.6 EPDMS on navhard.Source: §5 · p. 13
  • Clean-code precision does not remove conditional prediction error, and bounded residual refinement cannot replace a different maneuver when reactive continuations demand one.The authors identify broader visual contexts and more interactive planning settings as future directions.Source: §5 · p. 14

A.1 Target Representation

The target representation combines learned visual motion tokens with a fitted residual in metric trajectory coordinates. The causal planner predicts both from current context, while a bounded residual policy supplies a separate post-training correction.

A.1 Target Representation

  • Four 64-dimensional visual motion tokens decode through a velocity-and-yaw-rate decoder into a 40-point, 10 Hz trajectory; the target encoder is not used at deployment.The visual tokenizer is fitted on trajectory and motion reconstruction objectives, then frozen before fitting the paired residual.Source: §A.1 · p. 15
  • 123.60/221.60 mm ADE/FDE for visual reconstruction become 0.31/0.46 mm with the paired ground-truth residual; each interval’s residual uses a shared 12-component basis for ten pose errors.The residual is fitted in ego-reference-frame pose coordinates from training residuals.Source: §A.1 · pp. 15–16

A.2 Causal Planner Training

  • The causal action learner uses current-frame features, ego state, and navigation context to jointly predict both codes with flow matching, without letting action queries access future-video tokens.Inference reuses observed-frame features; one evaluation is an Euler update across the noise-to-data interval.Source: §A.2 · pp. 16–17

A.3 Residual Policy, Geometry and Reward Optimization

  • The residual actor is trained with scorer-based updates on hard scenes and no-op anchors from the original training partition, then deployed as one deterministic mean correction without a scorer or candidate selection.The base generator and decoder remain frozen, and the zero-initialized actor initially reproduces the base plan.Source: §A.3 · p. 18

A.4 Derivations and Interpretation

The derivations bound reconstruction and one-step prediction errors under local-coordinate, decoder, and heading assumptions. They also show that token prediction error depends on representation geometry and need not rank representations by trajectory error.

A.4 Derivations and Interpretation

  • The trajectory norm uses a fixed positive-definite Q after setting a meter-per-radian scale, with heading errors interpreted in a consistent local chart.The RL quadratic prior uses a heading scale of 5 m/rad.Source: §A.4 · p. 18
  • Reconstruction bounds assume trajectories share a local coordinate chart, with an orthonormal PCA basis and a locally Lipschitz visual decoder.Predicted and target trajectories must use the same chart.Source: §A.4 · p. 19
  • The conditional flow-matching population minimizer is the conditional mean, but predictability from visual conditioning remains an empirical question.The finite linear probe measures predictability only within its model class.Source: §A.4 · p. 19
  • Equal standardized token errors can imply different trajectory errors because normalization and decoder geometry determine the physical metric.In raw residual coordinates, that metric is M^TQM.Source: §A.4 · p. 19
  • The one-step prediction error is bounded by endpoint trajectory prediction error plus an integration-error bound.The bound uses the flow's material acceleration and decoder Lipschitz continuity.Source: §A.4 · p. 19

B.1 Data, Evaluation, and Comparison Protocol

The protocol fixes data panels and comparisons across experiments, while specifying distinct trajectory grids and score components for NAVSIM versions. It also documents planner inputs and inference-budget conventions for reproduced and published baselines.

B.1 Data, Evaluation, and Comparison Protocol

  • Action learning uses 85,109 training scenes and a fixed 4,096-scene validation panel; reconstruction uses all 18,179 validation scenes.Tokenizer and model components are fitted only on training data, and each comparison uses identical scenes.Source: §B.1 · p. 20
  • Action-learning errors use eight nominal times from 0.5 to 4.0 seconds, while reconstruction, probes, and solver diagnostics use a 40-point 10 Hz grid.Errors measured on the two grids are not compared directly.Source: §B.1 · p. 20
  • NAVSIM v1 reports PDMS with five components, whereas v2 reports EPDMS with nine components; both scores range from 0 to 100.The listed components differ between benchmark versions.Source: §B.1 · p. 20
  • VITAL replaces SimWAM's action target with four visual–residual tokens and uses a 1.23M-parameter residual actor while keeping the tokenizer and decoder frozen.Across the NFE sweep, only the number of action-flow evaluations changes.Source: §B.1 · p. 20
  • SUV and SimWAM released checkpoints are evaluated at one and ten sampling steps without retraining or candidate selection, using their native inputs and fixed seeds.Scores are computed against the PDM reference with the same v1 metric cache and scorer configuration as VITAL.Source: §B.1 · p. 20
  • Published comparisons vary in inputs, foundation models, training data, and proposal selection, and reported inference budgets can include steps beyond denoising.The protocol defines NFE as generator evaluations and plans as candidate trajectories before selection.Source: §B.1 · p. 20

B.2 NAVSIM v2 Transfer Details

The section reports NAVSIM v2 transfer details and probes of visual-feature predictability. Residual post-training improves first-stage scores but not reactive continuations, while matched visual features improve probe planning results and shuffling removes that advantage.

B.2 NAVSIM v2 Transfer Details

  • NAVSIM v2 navhard evaluation uses the official mapping of original scenes to synthetic continuations, with reactive traffic in both stages.Scores are aggregated within mappings before averaging stage-wise products.Source: §B.2 · pp. 20–21
  • GeoWAM and PerceptDrive report combined EPDMS values of 36.6 and 34.5, respectively, under the same two-stage protocol.The comparison uses their published Stage 1 and Stage 2 results.Source: §B.2 · p. 21
  • Residual post-training raises one-step Stage 1 EPDMS from 82.3 to 83.3 but lowers Stage 2 from 47.1 to 46.2, leaving the product nearly unchanged.Training on non-reactive v1 observations improves initial-stage planning without a corresponding gain on reactive continuations.Source: §B.2 · p. 21
  • The one-stage v2 diagnostic rescoring of v1 trajectories uses non-reactive traffic and is reported separately from two-stage EPDMS.Table A4 covers corrected NAVSIM v2 navtest with VITAL's v1 weights.Source: §B.2 · p. 21, Table A4 · p. 22

B.3.1 Frozen-Feature Predictability Probes

  • The trajectory-only control uses four 76-dimensional tokens for 40-point trajectories and achieves 1.80 cm held-out clean-code reconstruction ADE.Probes use frozen current-frame video-token summaries and ego/navigation inputs, with ridge regression and nested training budgets.Source: §B.3.1 · pp. 22–23
  • With 8,192 fitting scenes, adding matched visual features raises VITAL from 56.4 to 79.4 PDMS and the trajectory code from 57.3 to 73.2.VITAL's stronger planning gain accompanies lower decoded ADE despite higher standardized token prediction MSE.Source: §B.3.1 · p. 23, Table A7 · p. 23
  • Shuffling visual features lowers VITAL from 79.4 to 52.3 PDMS and raises ADE from 0.72 to 2.20 m, indicating dependence on scene-specific visual information.The trajectory-code control changes from 73.2 to 50.1 PDMS and from 0.80 to 1.38 m ADE.Source: §B.3.1 · p. 23, Table A8 · p. 23
  • For near-straight, leftward, and rightward maneuvers, matched visual features bring decoded paths closer to recorded motion than ego-only or shuffled features.The figure's lower plots trace position error over the prediction horizon.Source: §B.3.1 · p. 23, Figure A2 · p. 23

B.3.2 Learning with a Randomly Initialized Action Expert

With matched action-learning budgets, VITAL exceeds trajectory-only controls at both solver budgets, and its score advantage reflects planning components beyond comfort. The decoder-sensitivity figure examines sensitivity to normalized token noise.

B.3.2 Learning with a Randomly Initialized Action Expert

  • The comparison trains raw trajectories, AE, and VITAL for one shared pass using identical batches, a shared pretrained foundation, and no residual post-training.The evaluation fixes initial noise and uses paired bootstrap confidence intervals over validation logs.Source: §B.3.2 · pp. 23–25, Table A9 · p. 24
  • AE's advantage over raw prediction emerges by 3,000 updates and reaches 9.9 one-step PDMS points at the final checkpoint.This establishes a learned-code baseline using trajectory supervision alone.Source: §B.3.2 · p. 24, Table A10 · p. 24
  • VITAL scores 70.1 and 71.4 PDMS at one and ten steps, exceeding AE's 64.9 and 66.3 at both budgets.It retains a 5.1-point advantage over AE with iterative refinement.Source: §B.3.2 · p. 24, Table A12 · p. 25
  • VITAL improves NC, DAC, EP, TTC, and DDC at both budgets despite lower comfort scores than AE.The reported comfort scores are 51.6/57.4 for VITAL and 99.6/99.2 for AE at one/ten steps.Source: §B.3.2 · p. 24
  • AE's 9.9-point one-step improvement over raw prediction includes 7.3 comfort points, while VITAL's higher PDMS is attributed to greater combined gated progress and TTC contribution.VITAL's upper bounds on GC are 8.6/9.6 points, below AE's measured 12.1/12.5 points.Source: §B.3.2 · p. 24
  • Figure A3 contrasts final-checkpoint PDMS at one and ten evaluations with AE comfort contributions and VITAL upper bounds.Hatched bars denote bounds.Source: §B.3.2 · pp. 24–25

B.3.3 Decoder Sensitivity

  • Figure A4 examines decoder sensitivity to normalized token noise.The supplied caption states the sensitivity being examined but gives no outcome.Source: Figure A4 · p. 25

B.4 Inference Efficiency and Solver Diagnostics

The diagnostics compare inference cost with solver accuracy and examine residual-policy controls and cross-model trajectories. One-step inference is fast, while added solver steps provide little planning-score change; residual corrections and model comparisons expose both local gains and geometric differences.

B.4 Inference Efficiency and Solver Diagnostics

  • 149 ms median latency at NFE=1 with the actor rises to 490 ms at NFE=10 for only 0.1 additional PDMS.Residual post-training adds no measurable latency beyond run-to-run variation.Source: §B.4 · pp. 25–26
  • Ground-truth ADE stays between 63.10 and 63.40 cm across four solver budgets, despite the 6.61 cm one-to-80 discrepancy and 0.97 cm 40-to-80 discrepancy.The 80-step endpoint is a numerical reference, not ground truth.Source: §B.4 · p. 26, Table A14 · p. 26
  • One-step and ten-step paths retain the same maneuver, with PDMS changing by at most 0.1 points in the examples.The additional flow evaluations mainly adjust local geometry.Source: §B.4 · p. 26, Figure A5 · p. 26

B.5 Residual-Policy Controls

  • Expanding the reward-training pool from 13,105 to 17,013 hard scenes improves mean gain by about 0.4 PDMS points, while reducing prior weight has little effect.The factorial uses three seeds per setting.Source: §B.5 · p. 26, Table A15 · p. 27
  • Residual corrections illustrate increased route progress, restored road compliance with 7.9 cm maximum displacement, and improved time-to-collision around another road user.In the road-compliance example, PDMS rises from 0.0 to 100.0.Source: §B.5 · p. 26, Figure A7 · p. 27

C.2 Additional Cross-Model Comparisons

  • In one left-bend example, VITAL scores 96.3 PDMS with 0.03 m FDE, while SimWAM scores 100.0 PDMS with 1.71 m FDE.The example illustrates that geometric error and simulated driving quality provide complementary information.Source: Figure A8 · p. 28, §C.2 · p. 29

C.1 Protocol for the Cross-Model Comparison

  • In the NAVSIM scene comparison, VITAL scores 100.0 PDMS, versus 58.3 for SUV and SimWAM and 21.4 for DiffusionDrive.VITAL follows the recorded path more closely and reduces endpoint overshoot relative to the three baselines in this scene.Source: §C.1 · pp. 28–29

C.3 Failure Cases and Scope of Residual Refinement

Residual refinement can improve a plausible maneuver yet still move a trajectory outside the drivable area. The reported evaluation also leaves safety, distributional performance, and broader deployment conditions unresolved.

C.3 Failure Cases and Scope of Residual Refinement

  • An 11.40 cm correction crosses the drivable-area boundary, changing DAC from one to zero and PDMS from 98.7 to 0.0 despite NC remaining one.Across the paired one-step evaluation, 64 scenes regress on at least one of NC, DAC, TTC, or DDC.Source: Figure A9 · p. 29, §C.3 · p. 29
  • Holding the predicted visual token fixed limits the residual actor to local exploration, so it cannot reliably repair a wrong route choice or large visual-prediction error.The paper points to reactive navhard regression as motivation for refinement that can revise the scaffold.Source: §C.3 · p. 29

D.1 Potential Societal Impact

  • Benchmark averages do not establish equal performance across locations, weather conditions, or road-user groups, and individual scenes can become less safe.The paper calls for intended-domain evaluation, rare-interaction tests, independent safety constraints, and a fallback for unreliable predictions.Source: §D.1 · pp. 29–30
  • The study does not report energy or emissions measurements across pretraining, tokenizer fitting, and planner optimization.Its experiments reuse public driving recordings that may contain information about people and places, subject to source-dataset privacy protections and use conditions.Source: §D.1 · p. 30

D.2 Known Limitations

The stated limitations bound the evidence to NAVSIM planning and to local residual refinement around a predicted maneuver. Public resources are acknowledged separately.

D.2 Known Limitations

  • The experiments assess planning within NAVSIM; generalization to unseen cities, sensor failures, and prolonged real-traffic interaction remains open.The authors note stage-dependent behavior and motivate training for how surrounding traffic responds as scenes evolve.Source: §D.2 · p. 30
  • The residual policy refines locally around a predicted maneuver, leaving incorrect route choices and large visual-prediction errors for future work.Broader control over the visual scaffold and explicit safety constraints are identified as possible ways to address these limitations.Source: §D.2 · p. 30

E Public Resources Used

  • The paper acknowledges public datasets, pretrained components, evaluation software, and analysis and visualization tools; Figure 1 scenes come from public videos.Source: E Public Resources Used · p. 31

E.1 Public Datasets Used

The listed public resources include driving datasets and implementations with stated terms or licenses. The entries identify resource names and applicable terms rather than reporting experimental findings.

E.1 Public Datasets Used

  • The dataset list includes nuPlan, with nuPlan Dataset Terms, and nuScenes and OpenScene, each marked CC BY-NC-SA 4.0.Source: §E.1 · p. 31
  • NAVSIM v1/v2 is listed with Apache License 2.0.Source: §E.2 · p. 31

E.2 Public Implementations Used

  • The list also includes VGGT-Ω with a FAIR Noncommercial Research License, Depth Anything 3 with Apache License 2.0 for code, and LingBot-Map with Apache License 2.0.Source: §E.2 · p. 31
  • LingBot-VA / VA 2.0 is listed with Apache License 2.0 for code, StarVLA with MIT plus additional attribution terms, SimWAM with MIT, and Wan2.2 with Apache License 2.0.Source: §E.2 · p. 31
  • The implementation list includes UMT5-XXL, SUV, DiffusionDrive, DiffSynth-Studio, PyTorch, and Transformers, with their listed licenses.UMT5-XXL and DiffSynth-Studio are marked Apache License 2.0; SUV and DiffusionDrive are marked MIT; PyTorch is BSD 3-Clause; Transformers is Apache License 2.0.Source: §E.2 · p. 31
  • Other listed implementations are ROS 2 and NumPy, both BSD 3-Clause, and nuPlan-devkit, Apache License 2.0.Source: §E.2 · p. 31
Loading 2610.20604v1…