Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

I. INTRODUCTION

• Controller changes can make the same planned trajectory produce different ego motion, creating a gap between the frozen model’s imagined future and actual execution.The shift may arise from changes in feedback controllers, control gains, actuation constraints, or vehicle dynamics.Source: §I · p. 1
• Execution information should adapt how the future is imagined, rather than replace the planned trajectory that conditions the world model.This preserves the distinct roles of the plan as a control reference and execution as post-controller motion.Source: §I · pp. 1–2
• Even exact post-controller execution provides limited benefit when directly substituted for the plan, so execution fidelity does not necessarily imply decision fidelity.The paper attributes this mismatch to the frozen world model interpreting post-controller motion through its learned plan-to-motion mapping.Source: §I · pp. 1–2
• ExecutionBridge infers controller-specific execution characteristics from paired planned and executed trajectories to adapt the native ego latent rollout.The original planned trajectory remains the conditioning input, and the planner, world model, and reward head stay unchanged.Source: §I · p. 2
• ExecutionBridge consistently improves planning across ten unseen controller configurations, with larger gains in challenging driving scenes.The planner, world model, and reward head remain frozen.Source: §I · p. 2

A. World Models for Planning

• World-model planners use predictive dynamics to anticipate future consequences before acting, and BEV-, occupancy-, latent-, and video-based approaches predict scene evolution for downstream planning.WoTE uses imagined future states to score multiple candidate trajectories.Source: A. World Models for Planning · p. 2

B. Dynamics Identification and Adaptation

• Unlike dynamics-adaptation methods, this setting keeps the planner, world model, and reward head frozen while paired trajectories provide controller-specific execution characteristics for latent-rollout adaptation.The deployment shift is a controller change that can make the same plan produce different ego motion.Source: B. Dynamics Identification and… · p. 2, A. World Models for Planning · p. 2

C. Execution-Aware Planning and Predictive Conditioning

• Unlike approaches that modify the planning objective or model plan-to-execution transitions, ExecutionBridge keeps the planned-trajectory condition unchanged and adapts the latent future for evaluation.Prior approaches include drag-aware trajectory generation [27] and robot-factored world models [28].Source: C. Execution-Aware Planning and… · p. 2

• At time t, the planner observes world-model latent state z_t and proposes K candidate trajectories, each specifying planned ego position and yaw over H steps.Source: §III · p. 2
• The frozen world model predicts each candidate’s future from its planned trajectory, and the frozen reward head evaluates candidates using that rollout.Source: §III · pp. 2–3
• When the deployment controller executes a plan differently from the frozen model’s implicit mapping, the imagined future may differ from the future encountered.Source: §III · p. 3
• The support set contains N planned–executed pairs from controller c, sampled independently of current candidates to adapt their scores without modifying candidate trajectories or pretrained components.Source: §III · p. 3

A. Overview

• ExecutionBridge uses a support set of planned–executed pairs to adapt the latent future rather than replacing the planned trajectory supplied to the frozen world model.Source: A. Overview · pp. 3–4
• The Controller Encoder summarizes the support set into a compact representation of persistent controller-specific execution characteristics, and the Latent Bridge uses it to adapt the native ego latent.Source: A. Overview · p. 4
• The controller-conditioned rollout is evaluated by the unchanged reward head, while the planner, world model, and reward head remain frozen.Source: A. Overview · p. 4

B. Controller Representation

• The Controller Encoder maps the support set to a representation e_c that captures controller-specific execution characteristics.Source: B. Controller Representation · p. 4
• A training-only decoder predicts the execution of an independently sampled query trajectory conditioned on e_c, without requiring controller labels or explicit dynamics parameters.Source: B. Controller Representation · p. 4
• Predicting unseen query executions encourages e_c to capture characteristics that persist across planned trajectories rather than memorize support samples; the encoder is then frozen and the decoder discarded.Source: B. Controller Representation · p. 4

C. Controller-Conditioned Latent Bridge

• Compared with frozen WoTE, ExecutionBridge selects trajectories better aligned with controller-dependent execution in representative collision and drivable-area failure cases.Source: Figure 4 · p. 5
• The Latent Bridge predicts a controller-conditioned update to the native ego latent, modulated by a learned gate and projected onto an ℓ2 trust region.Source: C. Controller-Conditioned Latent Bridge · p. 5
• The bounded correction is injected only into the ego-related latent component; other components remain unchanged at that step before propagation through frozen world-model dynamics.Source: C. Controller-Conditioned Latent Bridge · p. 5

D. Bridge Training and Deployment

• The training objective combines controller-specific future-BEV supervision with candidate-level decision supervision.The future loss compares adapted BEV predictions with targets associated with each candidate’s controller-specific execution.Source: D. Bridge Training and Deployment · p. 5
• Regularization discourages unnecessary changes to the native ego latent and penalizes updates that exceed the trust-region bound.The bound is applied before projection.Source: D. Bridge Training and Deployment · p. 5
• Training first optimizes and freezes the Controller Encoder, then optimizes the Latent Bridge while the planner, world model, and reward head remain fixed.At deployment, the controller representation is computed once from the support set and cached, with no simulator supervision, online optimization, or parameter update.Source: D. Bridge Training and Deployment · p. 5
• ExecutionBridge is applied across frozen world-model planners while keeping each backbone’s native planning components frozen.The table caption identifies the comparison scope but supplies no performance values.Source: Table I · p. 6

A. Experimental Setup

• The controller-disjoint evaluation uses NAVSIM with WoTE as the primary frozen planner and gives each unseen controller 1,024 planned–executed support pairs.Evaluation controllers are excluded from encoder pretraining and bridge training, and are observed at test time only through their support sets.Source: A. Experimental Setup · p. 6

B. Planning–Execution Gap under Unseen Controllers

• Mean PDMS rises from 87.98 to 88.59 (+0.61) across ten unseen controllers, while mean TTC rises from 94.53 to 94.86 (+0.33).The low-score subset’s PDMS rises from 32.04 to 46.13 (+14.09).Source: Figure 5 · p. 6, B. Planning–Execution Gap under Unseen… · p. 6
• On the low-score subset, PDMS increases from 32.04 to 46.13 (+14.09), a substantially larger gain than across controller styles overall.The subset is defined by Frozen WoTE PDMS < 80 and fixed from the frozen baseline.Source: Figure 5 · p. 6, A. Experimental Setup · p. 6
• Controller variations in acceleration, steering response, and temporal lag can alter candidate consequences and rankings under the frozen evaluator.Representative styles include aggressive, conservative, steering-lag, and steering-gain behaviors; the examples describe safety-critical cases where controller-conditioned evaluation changes the selected trajectory and avoids failures.Source: B. Planning–Execution Gap under Unseen… · p. 6, Figure 6 · p. 7
• ExecutionBridge retains the planned trajectory as the conditioning input and adapts the imagined rollout using controller-specific execution information, addressing decision-relevant shifts rather than only tracking error.The diagnostic reports that predicted or exact execution reduces geometric error while candidate-ranking fidelity changes only marginally on the difficult-scene subset; direct substitution performs below ExecutionBridge in the candidate-level diagnostic.Source: B. Planning–Execution Gap under Unseen… · pp. 6–7, Figure 7 · p. 7

C. Execution Fidelity Does Not Imply Decision Fidelity

• Each candidate interpolates from its planned trajectory toward predicted or exact controller execution, with α = 0 retaining the plan and α = 1 fully substituting execution.The tested interpolation weights are 0, 0.25, 0.5, 0.75, and 1.Source: C. Execution Fidelity Does Not Imply… · p. 7
• Better execution fidelity substantially reduces geometric error but only marginally improves candidate ranking; ExecutionBridge yields the best decision quality.The result supports adapting the imagined future while preserving the planned trajectory as the world-model condition.Source: C. Execution Fidelity Does Not Imply… · p. 7

D. Native Compatibility and Backbone Generality

• The standard NAVSIM evaluation reports that ExecutionBridge improves WoTE while preserving its native planner, world model, reward head, and candidate generator.The section also evaluates across different frozen world-model planners.Source: D. Native Compatibility and Backbone… · p. 7

CONTROLLER STYLES SHOW STABLE EXECUTION AND PLANNING

• The controller-conditioned adaptation yields consistent gains on ResWorld and WorldDrive, indicating it is not tied to a WoTE-specific architecture.Representative examples illustrate more reliable trajectory selection in challenging driving scenarios.Source: CONTROLLER STYLES SHOW STABLE… · p. 7

E. Ablation and Sensitivity Analysis

• Latent adaptation without controller conditioning yields only modest improvement, while conditioning final candidate scores on ec helps but remains below ExecutionBridge, especially on the low-score subset.The comparison supports using controller context to adapt the imagined future before frozen reward evaluation.Source: E. Ablation and Sensitivity Analysis · p. 7
• Adding candidate-wise execution residuals performs slightly worse than conditioning on ec alone, suggesting persistent controller characteristics are more useful than direct per-candidate correction.Source: E. Ablation and Sensitivity Analysis · p. 7
• Execution ADE and planning performance remain nearly unchanged as support-set size varies from N = 64 to 1024, suggesting a small set suffices to infer controller characteristics.Source: E. Ablation and Sensitivity Analysis · pp. 7–8
• Overly tight trust-region constraints hinder adaptation, while performance saturates for sufficiently large ϵ; the study uses ϵ = 0.2.Source: E. Ablation and Sensitivity Analysis · p. 8

VI. CONCLUSION

• Deployment controllers can execute the same planned trajectory differently from the plan-to-motion behavior implicitly captured during pretraining.Even exact execution provides only limited benefit when directly substituted for the planned trajectory.Source: §VI · p. 8
• ExecutionBridge consistently improves planning across unseen controllers and multiple frozen world-model planners while retaining the original plan as the conditioning input.It infers controller-specific execution characteristics from planned–executed trajectory pairs and adapts the native ego latent rollout.Source: §VI · p. 8