What Matters for World Modeling in Vision-Language-Action Models?

• Standard VLAs predict actions from current observations, and without modeling scene evolution they struggle with long-horizon tasks, distribution shifts, and dynamic environments.Source: §1 · p. 1
• The study evaluates four integration paradigms across diverse world representations and policy backbones in a unified pipeline with shared backbones and training protocols within comparisons.Source: §1 · p. 2
• World modeling improves performance over action-only policies, with gains particularly notable on long-horizon and dynamic tasks.Source: §1 · p. 2
• No single representation or paradigm is universally optimal; DINOv3 and SigLIP provide consistent cross-paradigm gains, while decoupled MoT is competitive overall, particularly for OOD robustness.Source: §1 · p. 2

• VLA research has explored 3D priors, dual-system architectures, chain-of-thought reasoning, latent actions, and heterogeneous data mixtures.Source: §2 · p. 3
• World-model integration is grouped into Implicit Future Alignment, Explicit Future Query, MoT Future Expert, and External World Predictor according to conditioning and decoupling choices.Source: §2 · p. 3
• The related work identifies a lack of systematic, controlled comparisons of these paradigms and their effects on world modeling in VLAs.Source: §2 · p. 3

3.1 PRELIMINARIES

• The baseline pairs a pretrained VLM with a trainable action expert, jointly trained using a flow-matching action objective and shared future-representation supervision.Source: §3.1 · p. 3

3.2 STRATEGY 1: IMPLICIT FUTURE ALIGNMENT

• Implicit Future Alignment trains intermediate VLM visual representations to align with future world representations, so action tokens can attend to dynamics-enriched representations; its auxiliary predictor is discarded at inference.Source: §3.2 · p. 4

3.3 STRATEGY 2: EXPLICIT FUTURE QUERY

• Explicit Future Query appends learnable future-query tokens that aggregate visual and textual context; a decoder trains them against future representations, then is discarded at inference.Source: §3.3 · pp. 4–5

3.4 STRATEGY 3: MOT FUTURE EXPERT

• MoT Future Expert uses a parallel stream trained to predict future representations, while the action expert interacts with both streams across intermediate layers to absorb evolving dynamics.Source: §3.4 · p. 5

3.5 STRATEGY 4: EXTERNAL WORLD PREDICTOR

• External World Predictor uses a standalone predictor conditioned by future-query representations, and the action expert receives its predicted future representation directly in cross-attention context.Source: §3.5 · p. 5

4.1 EXPERIMENTAL SETUP

• Primary comparisons use a shared Qwen3-VL-2B backbone and training protocol; π0.5 experiments additionally test transfer to a pretrained VLA.Evaluation covers LIBERO, LIBERO-Plus, RoboTwin 2.0, and DOMINO.Source: §4.1 · p. 6

4.2 WORLD REPRESENTATION ANALYSIS ACROSS PARADIGMS

• DINOv3 and SigLIP consistently improve in-domain and OOD averages across paradigms, while representation rankings vary by paradigm; DINOv3 leads OOD average gain and upper bound.V-JEPA 2.1 is strongest OOD with WM-MoT but falls below baseline with WM-Ext; DINOv3 is therefore the default representation for later comparisons.Source: §4.2 · pp. 6–7, Figure 3 · p. 6

4.3 MAIN RESULTS ACROSS MANIPULATION BENCHMARKS

• All four paradigms improve LIBERO and LIBERO-Plus over the action-only baseline; WM-Align reaches 98.6% in-domain, while WM-MoT adds 4.4% overall OOD.The OOD gains span camera shifts, sensor noise, and background variations.Source: Table 1 · p. 7, §4.3 · p. 7
• On RoboTwin Clean2Clean, gains rise from 5.5–8.6% on short tasks to 16.5–26.3% on long tasks; Clean2Random long-task gains are 1.3–3.7%.On Clean2Random short tasks, only WM-Align improves by 1.9%; the other paradigms drop 0.8–2.5%, and future prediction can introduce slight noise under visual shifts.Source: Table 2 · p. 7, §4.3 · pp. 7–8

4.4 PERFORMANCE UNDER DYNAMIC ENVIRONMENTS AND DATA SCARCITY

• On DOMINO Level 1, WM-MoT and WM-Ext reach 15.5% and 15.1% success versus 10.6% for the action-only baseline, exceeding WM-Align and WM-Query.Dedicated predictive modules may capture continuous object motion more flexibly than patch alignment or bottleneck queries.Source: Table 3 · p. 8, §4.4 · p. 8
• Under data scarcity, WM-Ext reaches 96.5% at 50% in-domain data, while WM-MoT leads OOD at 62.4% with 25%; WM-Query degrades sharply.The relative advantages of WM-Ext and WM-MoT differ across domains.Source: Figure 4 · p. 8

4.5 COMPATIBILITY WITH PRE-TRAINED FOUNDATION VLAS

• On π0.5, WM-MoT and WM-Ext improve both RoboTwin settings, while WM-Align drops over 5% in each; WM-Ext leads in-domain and WM-MoT OOD.The authors suggest shared-representation predictive objectives may distort pretrained features and disrupt manipulation priors.Source: §4.5 · pp. 8–9, Table 4 · p. 9

4.6 REAL WORLD EXPERIMENTS

• Across four real-world tasks, world modeling raises average success from 45.0% to 56.3–73.8%, with WM-MoT highest at 73.8%.WM-MoT reaches 75.0% on dynamic picking and 55.0% on tomato placement.Source: §4.6 · p. 9, Figure 5 · p. 9

5 CONCLUSION

• The study finds 2D semantic representations provide reliable cross-paradigm gains, while pixel and 3D representations are more sensitive to integration architecture.Coupled designs offer compact representations for static manipulation, whereas decoupled designs show superior robustness under distribution shifts and dynamic manipulation, and better compatibility with pretrained foundation VLAs.Source: §5 · p. 9

LLM USAGE DISCLOSURE

• Generative AI and LLMs were used solely to aid and polish writing; the authors reviewed and edited the content and take responsibility for the submission.Source: LLM USAGE DISCLOSURE · p. 10

APPENDIX OVERVIEW

• The appendix is organized into experimental setup, controlled ablations and diagnostics, and detailed benchmark results.Source: APPENDIX OVERVIEW · p. 14

A EXPERIMENTAL SETUP AND IMPLEMENTATION DETAILS

• Appendix A specifies benchmarks, representation encoders, policy backbones, training configurations, evaluation protocols, and computational and deployment costs.Source: A EXPERIMENTAL SETUP AND… · p. 14

A.1 BENCHMARKS AND EVALUATION TASKS

• LIBERO and LIBERO-Plus cover multi-task manipulation and seven OOD perturbation dimensions, while LIBERO suites test spatial, object, goal, and long-horizon skills.The perturbations include camera viewpoints, robot initial states, language, lighting, textures, sensor noise, and object layouts.Source: §A.1 · p. 14
• RoboTwin 2.0 evaluates dual-arm tasks across short, medium, and long horizons under clean and domain-randomized settings; DOMINO tests moving-target manipulation.RoboTwin uses 50 clean demonstrations per task and evaluates Clean2Clean and Clean2Random conditions; DOMINO uses the clean dynamic Level 1 regime.Source: §A.1 · pp. 14–15
• Physical experiments use a Franka Research 3 on four static and dynamic tasks, with 50 training demonstrations and 40 evaluation trials per task.The dynamic tasks are Pick Tomato and Place Tomato on Plate.Source: §A.1 · p. 15

A.2 TARGET WORLD REPRESENTATION ENCODERS

• Six frozen encoders represent pixel-level, 2D semantic, 3D geometric, and spatiotemporal targets, sampled at four shared future timesteps.Image encoders process the future frames independently, while video encoders encode them jointly.Source: §A.2 · p. 15
• In decoupled architectures, the chosen encoder supplies both current-observation features and future targets, so representation rankings reflect the full paired system.This differs from WM-Align and WM-Query, where the encoder interacts with standard VLM visual tokens through the future auxiliary loss alone.Source: §A.2 · p. 15

A.3 POLICY BACKBONES AND BASELINE IMPLEMENTATION

• The primary testbed pairs Qwen3-VL-2B with a cross-DiT action expert, with π0.5 testing transfer to a pretrained foundation VLA.WM-Align, WM-Query, WM-MoT, and WM-Ext differ in how future supervision and predictive representations interact with the policy.Source: §A.3 · pp. 15–16

A.4 HYPERPARAMETER AND TRAINING SPECIFICATIONS

• World-supervised variants share the loss formulation, loss weight, future-frame sampling, and optimization schedule within each controlled comparison.The protocol is intended to compare architecture compatibility under unified training rather than disparate tuning schedules.Source: §A.4 · p. 16, Table A.1 · p. 17

A.5 EVALUATION PROTOCOL AND METRICS

• Evaluation uses matched checkpoints and closed-loop rollouts, reporting success rates across LIBERO, LIBERO-Plus, RoboTwin 2.0, DOMINO, and real-world tasks.DOMINO additionally reports a manipulation score for intermediate task progression, including when a rollout ends in failure.Source: §A.5 · p. 17

A.6 COMPUTE AND DEPLOYMENT COST

• Deployment costs differ by design: WM-Align returns to baseline parameter count and latency, WM-Query remains near-baseline, while WM-MoT and WM-Ext retain predictive modules online.WM-MoT and WM-Ext also require the frozen 85.7M-parameter DINOv3 encoder online in the reported comparison.Source: §A.6 · p. 18, Table A.2 · p. 18
• A within-architecture WM-MoT ablation attributes an explicit performance gain to future-state supervision rather than capacity expansion alone.The control fixes architecture and current-feature inputs while comparing no auxiliary, current-feature, and future-feature supervision.Source: §A.6 · p. 18

B CONTROLLED ABLATIONS, ROBUSTNESS, AND DIAGNOSTIC ANALYSES

• Targeted analyses include multi-seed replications, supervision ablations, factorial encoder isolation, and hyperparameter sensitivity sweeps.The section frames these experiments as checks of empirical reliability, factor isolation, and optimization behavior.Source: B CONTROLLED ABLATIONS, ROBUSTNESS,… · p. 19

B.1 MULTI-SEED TRAINING VERIFICATION AND REPLICATIONS

• Across independent runs, standard deviations range from 0.3% to 1.5% on multi-task benchmarks and reach 2.0% in the catastrophic-collapse regime.The replication used three seeds and full benchmark rollouts after 50k-step optimization.Source: §B.1 · p. 19
• +5.0% OOD gain: WM-MoT reaches 76.4±1.0% versus Baseline 71.4±0.4% across seeds.The three-seed study reports mean and standard deviation across independent training runs.Source: §B.1 · p. 19, Table B.1 · p. 19
• DINOv3 exceeds SigLIP by +2.2% in WM-MoT (76.4±1.0% vs. 74.2±0.4%) and +0.7% in WM-Align (74.2±0.6% vs. 73.5±0.3%).The reported ranking is stable across seeds, with a larger representation gap in WM-MoT than WM-Align.Source: §B.1 · p. 19
• On RoboTwin 2.0, WM-Align falls below the π0.5 baseline on Clean2Clean (63.5±1.5% vs. 69.0±0.8%) and Clean2Random (40.8±1.4% vs. 46.6±0.7%).WM-MoT instead reports gains of +2.4% on C2C and +3.3% on C2R.Source: §B.1 · pp. 19–20
• WM-Ext with V-JEPA scores 70.6±0.8% versus 71.4±0.4% baseline on LIBERO-Plus, while in-domain scores are 97.0±0.4% versus 96.8±0.3%.The passage reports this OOD negative transfer across independent runs.Source: §B.1 · p. 20

B.2 CONTROLLED EVALUATION OF AUXILIARY DINOV3 CURRENT-FRAME INPUT

• Fusing current-frame DINOv3 features raises the baseline from 71.8% to 72.5% (+0.7%), while WM-MoT with future supervision reaches 76.2%, 2.6% above its λ = 0 variant.WM-Ext reaches 71.1% without future supervision and 74.7% with it; the passage attributes the former result to bottleneck modulation drift.Source: §B.2 · p. 20
• With equalized current-frame DINOv3 access, WM-MoT reaches 76.2% (+2.6% over λ = 0) and leads WM-Ext, WM-Align, and WM-Query by 1.5%, 1.7%, and 2.5%.The comparison is on LIBERO-Plus OOD; Table B.2 reports results across seven perturbation dimensions and on average.Source: Table B.2 · p. 20, §B.2 · p. 20

B.3 FACTOR ISOLATION: DISENTANGLING CURRENT INPUT FROM FUTURE SUPERVISORY TARGET

• Homogeneous V-JEPA peaks in WM-MoT at 77.6% versus 76.4% for DINOv3, while WM-Ext favors DINOv3 at 74.6% versus 70.6% for V-JEPA.Cross-encoder pairings reduce WM-MoT performance relative to homogeneous V-JEPA, and WM-Ext remains near or below the action-only baseline whenever its supervisory target is V-JEPA.Source: Table B.3 · p. 21, §B.3 · p. 21

B.4 HYPERPARAMETER SENSITIVITY ANALYSES

• On RoboTwin 2.0, lowering WM-Align’s λworld to 0.01 raises C2C from 63.6% to 67.8%, but results remain below baseline across tested settings.Earlier or all-layer supervision further lowers C2C to 60.8% or 58.7%, respectively.Source: Table B.4 · p. 21, §B.4 · p. 22
• On 25% LIBERO demonstrations, reducing WM-Query to Nq = 4 raises Goal success from 28.8% to 56.2%, still below the action-only baseline’s 77.6% Goal success.The passage also reports the action-only baseline at 84.9% Suite success; the bottleneck adjustment only partially mitigates degradation.Source: Table B.5 · p. 22, §B.4 · p. 22

B.5 SCOPE OF CLAIMS AND LIMITATIONS

• The study defines world modeling as task-conditioned prediction of future visual representations under demonstration trajectories to condition action generation, and warns that backbone differences cannot be attributed solely to robotics pretraining.Model capacity, vision encoders, and action-generation heads also differ; no architectural winner is universal, since the preferred choice depends on horizon, visual shift, and demonstration volume.Source: §B.5 · p. 22

C.1 DETAILED LIBERO RESULTS FOR REPRESENTATION ANALYSIS

• Tables C.1 and C.2 provide suite- and perturbation-level LIBERO success rates underlying Figure 3.The appendix describes these tables as detailed benchmark breakdowns supporting aggregate findings.Source: §C.1 · p. 23, C DETAILED BENCHMARK RESULTS AND… · p. 23

C.2 FEW-SHOT DATA SCALING RESULTS

• LIBERO data-scaling tables report results at 25%, 50%, and 100% of training demonstrations, using the same demonstration subset across methods at each fraction.The tables underlie Figure 4.Source: §C.2 · p. 23

C.3 DETAILED ROBOTWIN RESULTS

• Table C.5 reports task-level success rates for baseline and world-modeling variants underlying Table 2, while Table C.6 reports task-level success rates underlying Table 4.Source: §C.3 · p. 23

C.5 DETAILED DOMINO RESULTS

• LIBERO data-scaling tables give detailed success rates at each training-data fraction, including seven-perturbation LIBERO-Plus results; bold and underlining mark the best and second-best results.Within each fraction, the reported methods use the same training-demonstration subset.Source: Table C.3 · p. 25, Table C.4 · p. 25
• RoboTwin task-level tables report success rates for baseline and world-model variants, including π0.5 results under Clean2Clean and Clean2Random evaluation.Group and overall averages correspond to the respective aggregate tables.Source: Table C.5 · p. 26, Table C.6 · p. 27
• DOMINO Level 1 results cover the clean dynamic setting and report both success rate and manipulation score.The detailed results support Table 3.Source: §C.5 · p. 24, Table C.7 · p. 28

C.6 DETAILED REAL-ROBOT RESULTS

• Table C.8 reports real-world manipulation success rates (%) across static and dynamic tasks on the physical Franka Research 3, with each task evaluated over 40 trials per method.Figures 6–9 show representative physical-robot rollouts for all four tasks; each row contains six uniformly sampled frames from one trajectory.Source: Table C.8 · p. 29, §C.6 · p. 29, §C.7 · p. 30

C.7 QUALITATIVE REAL-ROBOT ROLLOUTS

• The qualitative rollout figures identify each selected trajectory’s method and outcome, with six uniformly sampled frames per trajectory.The figures cover the four physical-robot tasks listed in their captions.Source: §C.7 · p. 30, Figure 6 · p. 31, Figure 7 · p. 31