PaperlayerView exact revision on arXiv

Source-linked AI summary

DRILL: Training World Models to Improve Policies, Not Predict Pixels

Zhaolu Kang, Tailong Luo, Lei Wei, Guangyuan Dong, Sihan Liu, Yingjie He, Siheng Wang, Shuaibo Li, Rongchao Zhang, Zhen Tian, Richeng Xuan, Zhichao Hu

arXiv:2610.30814v1cs.CV

TL;DR

World models for VLA post-training are often optimized for next-frame pixels even though imagined rollouts are used to produce policy-learning signals. DRILL trains the world model for downstream return after imagined GRPO, and DRILL-IMG improves success over prior methods despite higher pixel MSE.

  • Problem

    VLA world models are commonly trained for one-step pixel prediction although their post-training role is to generate imagined rollouts that guide policy updates.

    Source: §1 · p. 1

  • Method

    DRILL updates a world model to maximize target-environment return after an inner GRPO update on imagined rollouts, using either a no-extra-rollout surrogate or a simulator-grounded meta-gradient.

    Source: §1 · p. 2

  • Results

    Across five simulators and two VLA backbones, DRILL-IMG raises mean success by 13.1 points over WMPO and 7.9 points over RLVR-World despite 17% higher one-step pixel MSE.

    Source: §1 · p. 2, §5.2 · p. 7

  • Takeaways & Limitations

    Across the evaluated simulators and VLA backbones, DRILL improves policy success even when one-step pixel MSE worsens.

    Source: §6 · p. 9

    Evaluation is simulation-only, leaving sensitivity to real-world perception noise, actuation delay, safety constraints, and sim-to-real shift unresolved.

    Source: §6 · p. 9

Abstract

from arXiv · show

World models are increasingly used to reduce environment interaction when post-training vision-language-action (VLA) policies with reinforcement learning. Yet they are typically trained to predict pixels one step ahead, even though their role here is not merely to render visually faithful videos but to produce learning signals from imagined rollouts that improve the policy. We show that this mismatch is substantial: lower pixel MSE can produce worse policy gradients, and a world model trained for policy utility can yield better downstream policies despite 17% higher pixel MSE on the next frame. We propose the Downstream-Return Imagination Learning Loop (DRILL), a bilevel framework that updates the world model to maximize the policy's return after an inner GRPO update on imagined rollouts. DRILL has two instantiations. DRILL-VWI is a closed-form surrogate, requiring no additional simulator rollouts, that reweights prediction errors by policy visitation, the group-relative GRPO advantage, and model confidence. DRILL-IMG is a full meta-gradient method that differentiates through the inner update using truncated Hessian-vector products. We show that DRILL-VWI recovers the local one-step component of the DRILL-IMG meta-gradient under score-compatibility and small-residual assumptions. Across five manipulation simulators and two VLA backbones, DRILL-IMG raises mean success rate by 13.1 points over WMPO and 7.9 points over RLVR-World, and complements RLVR pretraining rather than competing with it. More broadly, we believe video world models for VLA reinforcement learning can be productively trained by the policy gradients they produce, not only by pixel-level prediction.

1 Introduction

The paper argues that next-frame pixel accuracy is a poor proxy for whether imagined rollouts produce useful VLA policy updates. DRILL trains world models for downstream return after imagined GRPO, with a lightweight surrogate and a more direct meta-gradient variant.

1 Introduction

  • World models used for VLA post-training generate imagined policy-learning signals, so their role is not limited to predicting the next frame.The paper frames this as a mismatch between one-step pixel-prediction training and policy-gradient utility.Source: §1 · pp. 1–2
  • Pixel MSE weights errors uniformly, whereas policy updates emphasize visited states and actions with large advantages, allowing pixel fidelity and policy utility to diverge.A model can predict frames well yet provide weak learning signals, or have worse average prediction error while supporting policy improvement.Source: §1 · p. 1
  • DRILL trains the world model to maximize target-environment return after an inner GRPO update on imagined rollouts.The objective focuses on the downstream policy rather than pixel accuracy under the model's own rollouts.Source: §1 · p. 2
  • DRILL-VWI is a no-extra-rollout signed surrogate, while DRILL-IMG differentiates through the inner update and uses a small simulator batch.The VWI surrogate is a first-order approximation to the IMG meta-gradient under local assumptions.Source: §1 · p. 2
  • DRILL-IMG raises mean policy success by 13.1 points over WMPO and 7.9 points over RLVR-World across five simulators and two VLA backbones, despite 17% higher one-step pixel MSE.The gains remain complementary to prediction-correctness pretraining, adding 3.2 points on average to an RLVR-World-pretrained model.Source: §1 · p. 2
  • The paper introduces PUR to measure the angular gap between pixel-MSE gradients and policy improvement, and reports that lower PUR tracks higher downstream success.The diagnostic is presented as method-agnostic across the evaluated objectives.Source: §1 · p. 2

2 Related Work

Prior VLA world-model approaches use imagined rollouts for policy updates but primarily train the model as a next-frame predictor. DRILL instead optimizes for the post-update policy's downstream return and brings policy-aware learning to VLA post-training.

2 Related Work

  • VLA post-training methods use demonstration-trained world models for imagined policy updates, but primarily optimize them as next-frame predictors.Policy-Aware WM and ESNR diagnose a prediction-quality/policy-competence gap; RLVR-World and WorldCompass use prediction-correctness rewards, while DRILL targets downstream return.Source: §2 · p. 3
  • DRILL applies policy-aware world-model learning to VLA post-training using online GRPO advantages as a per-batch surrogate for downstream utility.Its signed advantage weighting approximates the meta-gradient under a local score-compatibility condition.Source: §2 · p. 3
  • DRILL-IMG uses a short unroll of the policy update rather than solving the inner optimization to convergence, with truncation bias analyzed as a function of K.The method applies meta-gradients to foundation-scale VLAs and video world models using a small simulator-rollout batch.Source: §2 · p. 3

3 The DRILL Framework

DRILL couples an imagined GRPO policy update with a world-model update guided by the post-update policy's target-environment return. Its two estimators trade computational and simulator cost for how directly they estimate that return objective.

3 The DRILL Framework

  • DRILL uses an imagined GRPO inner update and assigns outer-update credit to the world model by evaluating the resulting policy in the target environment.The two variants differ in how much of this outer gradient they estimate.Source: §3 · p. 3, Figure 1 · p. 4

3.1 Problem setup

  • The setup distinguishes target-environment state visitation from imagined rollouts generated by the policy under the video world model.The target environment in the experiments is the high-fidelity simulator defining task return.Source: §3.1 · p. 3

3.2 The DRILL objective

  • DRILL optimizes the return of the policy produced by one imagined GRPO step, with the expectation taken over target-environment trajectories rather than the world model's own rollouts.This replaces the pixel-fitting update with one aligned to the model's downstream policy-learning role.Source: §3.1 · p. 3, §3.2 · p. 4

3.3 DRILL-VWI: Visitation-Weighted Imagination

  • DRILL-VWI replaces pixel MSE with a signed local surrogate weighted by policy visitation, group-relative advantage, and model confidence, without additional environment interaction over WMPO.Its stopped and clipped weights make it a local update direction rather than a globally normalized likelihood objective.Source: §3.3 · p. 4

3.4 DRILL-IMG: Implicit Meta-Gradient

  • DRILL-IMG estimates the meta-gradient by differentiating through imagined GRPO with a Hessian-vector product and K-step truncated BPTT, using a small simulator batch for the outer return gradient.The main experiments use K=2 and 30 simulator rollouts per outer step; this estimator costs additional simulator interaction and BPTT memory.Source: §3.4 · p. 5

4 Theoretical Analysis

The analysis connects DRILL-VWI to a local component of the DRILL-IMG meta-gradient, characterizes truncated-BPTT convergence, and defines PUR as an alignment diagnostic. The local approximation depends on compatibility and small residuals, while the convergence guarantee is limited to a fixed-policy outer subproblem.

4 Theoretical Analysis

  • The analysis serves three roles: establishing the VWI surrogate connection, bounding DRILL-IMG under truncated BPTT, and defining the policy-improvement alignment diagnostic.These results address the relationship between the estimators, the behavior of truncation, and comparison across world-model objectives.Source: §4 · p. 5

4.1 First-order meta-gradient and the DRILL-VWI surrogate

  • DRILL-VWI recovers the local one-step component of the DRILL-IMG meta-gradient when score compatibility holds approximately and trajectory-locality and demonstration-anchor residuals are small.Positive-advantage transitions are sharpened, while negative-advantage transitions may be deemphasized rather than fitted more accurately as pixels.Source: §4.1 · p. 5

4.2 Convergence rate under truncated BPTT

  • The truncated-BPTT guarantee gives convergence to a neighborhood of a stationary point, with the truncation-bias contribution decaying geometrically in K.The result is a per-outer-subproblem guarantee, not global convergence for the jointly evolving policy and world-model parameters; practical K=2 is chosen empirically.Source: §4.2 · pp. 5–6

4.3 Policy-Utility Residual

  • PUR compares the pixel-MSE gradient with the policy-improvement direction and is estimated by simulator finite differences, independently of any particular training method.The paper tests whether lower PUR tracks higher downstream success across evaluated methods.Source: §4.3 · p. 6

5 Experiments

Across five manipulation simulators and two VLA backbones, DRILL improves policy success despite higher one-step pixel error, with the IMG–VWI trade-off depending on horizon. The experiments also examine robustness, diagnostics, and complementary pretraining.

5.1 Setup

  • The evaluation spans five manipulation simulators and two VLA backbones, uses five seeds for headline results, and includes no real-robot results.DRILL-IMG uses a small simulator-rollout budget for its outer gradient.Source: §5 · p. 6, §5.1 · pp. 6–7

5.2 Main Results

  • DRILL-IMG raises mean success rate by 13.1 points over WMPO and 7.9 points over RLVR-World across five benchmarks, despite 17% higher one-step pixel MSE.On π0-3B, the gains are largest on long-horizon tasks; on SimplerEnv, DRILL-VWI and DRILL-IMG are statistically tied at 64.0±1.9 versus 63.1±2.6.Source: Table 1 · p. 7, §5.2 · p. 7
  • On OpenVLA-7B, DRILL-IMG reaches 68.9 mean SR versus 53.8 for WMPO, while DRILL-IMG and DRILL-VWI are statistically tied on RoboCasa.LIBERO-Long and CALVIN preserve the ranking DRILL-IMG > DRILL-VWI > RLVR-World > WMPO; applying DRILL-IMG after RLVR-World pretraining adds roughly 3 mean SR points.Source: §5.2 · pp. 7–8
  • Figure 3’s diagnostics show DRILL variants improve faster on LIBERO-Long, reduce PUR during training, and separate lower pixel MSE from higher policy success.Extra WMPO updates move LIBERO-Long SR from 64.2±1.6 to at most 65.1±1.9, below DRILL-IMG’s 81.3±1.3.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 Ablations and Scaling

  • Across LIBERO, CALVIN, and RoboCasa, combining visitation, signed advantage, and confidence gives DRILL-VWI 69.2 mean SR, versus 58.2 for WMPO.The exponent sweep peaks at α=1; α=2 reaches 67.9, while VA-WM reaches 62.4 mean SR.Source: Objective ablations · p. 8
  • On LIBERO-Long, DRILL-IMG’s margin over WMPO grows with world-model size from 7.9 points at 300M parameters to 16.4 at 7B.Freezing VWI coaching multipliers from a GRPO-warmstart policy reaches 64.9±1.5 mean SR, 4.3 points below DRILL-VWI.Source: Objective ablations · p. 8

5.4 When DRILL-IMG Pays Off over DRILL-VWI

  • At H=50, DRILL-IMG gains 5.2 points over DRILL-VWI on LIBERO-Long, while the IMG–VWI gap grows from 1.8 points at H=20 to 8.1 at H=200.DRILL-IMG peaks at 83.4 SR at H=100, then drops to 78.6 at H=200 as imagined trajectories become too noisy; DRILL-VWI can be preferable at very short horizons or without simulator rollouts.Source: §5.4 · pp. 8–9

5.5 Diagnostics and Robustness

  • Under LIBERO zero-shot perturbations, DRILL-IMG has an 18.6% average relative SR drop versus 30.2% for WMPO and 25.6% for RLVR-World.Across PUR-reporting headline rows, PUR correlates with downstream SR at ρS = −0.89; on LIBERO-Long, DRILL-IMG reaches 0.81±0.03 TRPS despite higher MSE than WMPO.Source: §5.5 · p. 9

6 Conclusion

The conclusion presents DRILL as a way to train world models for downstream policy improvement rather than next-frame prediction alone. The accompanying appendices collect notation, assumptions, and the GRPO surrogate used in the analysis.

6 Conclusion

  • DRILL trains video world models for policy improvement, and its two variants trade a lightweight local surrogate against a more direct downstream-return meta-gradient.Across five manipulation simulators and two VLA backbones, DRILL improves success even when one-step pixel MSE worsens; its gains combine with RLVR-World pretraining.Source: §6 · p. 9

A Notation and Preliminaries

  • Appendix A collects symbols, assumptions, and intermediate definitions used throughout the paper to make Appendices B–C readable without retracing the main text.Source: A Notation and Preliminaries · p. 11

A.1 Symbol glossary

  • The glossary distinguishes the GRPO surrogate and inner-update policy from the VWI coaching surrogate, IMG target meta-gradient, and named residual terms.It also defines H as rollout horizon, K as truncated-BPTT depth, and Breal as the simulator-rollout batch.Source: §A.1 · p. 12

A.2 Assumptions

  • Appendix A.2 assigns assumptions A1–A3 to Proposition 1 and A4–A6 to Theorem 1; the truncation-bias decay assumption requires ρ<1.The authors note that when ρ approaches 1, the convergence guarantee is no longer informative in K.Source: §A.2 · p. 14

A.3 The GRPO surrogate, restated

  • The restated GRPO surrogate uses imagined trajectories and group-standardized returns to form advantages, which are treated as constants during differentiation.The appendix assigns clipping, advantage-recomputation, visitation-drift, and demo-transfer effects to local approximation residuals.Source: §A.3 · p. 14

B Proof of Proposition 1: First-order meta-gradient and the DRILL-VWI surrogate

This appendix establishes when DRILL-VWI’s local signed surrogate approximates the first-order meta-gradient underlying DRILL-IMG, and identifies residuals that limit the match. It also states the assumptions and estimator components used in the analysis.

B Proof of Proposition 1: First-order meta-gradient and the DRILL-VWI surrogate

B.1 Setup and notation

  • The analysis treats the reward model as frozen, uses one imagined-rollout GRPO step, and assumes a differentiable Gaussian-head world model for the closed-form MSE identity.The Gaussian-head assumption is specific to that identity; analogous score- or flow-matching derivations are noted for other world-model heads.Source: §B.1 · p. 15

B.2 Main proposition

  • Under local score compatibility and small residuals, the target meta-gradient expands into a one-step term plus trajectory-locality and demo-anchor residuals.The proposition assumes a smooth outer objective and bounds the omitted H-step trajectory-score contribution.Source: §B.1 · p. 15, §B.2 · p. 16

B.3 Proof

  • The proof linearizes the outer objective, retains the local policy-score contraction, and uses the Gaussian-head identity to connect the world-model likelihood score to the MSE gradient.In the idealized case, gradient ascent on return is first-order equivalent to gradient descent on the signed DRILL-VWI surrogate up to a scalar; general deviations are controlled by the stated residuals and Taylor remainder.Source: §B.3 · pp. 16–17

B.4 Corollaries

  • With score compatibility and demo-anchor matching, DRILL-VWI at α = 1 matches DRILL-IMG’s local one-step direction; the positive stability factor rescales the surrogate without flipping its sign.The remaining nonlocal mismatch includes the trajectory-locality residual and higher-order Taylor remainder.Source: §B.4 · p. 18

B.5 Discussion

  • DRILL-VWI’s accuracy depends on compatibility, trajectory-locality, demo-anchor, and higher-order Taylor residuals; DRILL-IMG is most useful when omitted trajectory-locality effects are large.The horizon sweep is evidence for an increasing multi-step locality residual, not a direct isolation of a universal O(η^2_θH^2) term.Source: §B.5 · p. 18, §B.4 · p. 18

C.1 Setup

  • The convergence analysis estimates the outer gradient from simulator rollouts and a K-step truncated HVP through imagined rollouts, decomposing the estimator into gradient, bias, and zero-mean noise.The appendix analyzes a fixed-policy conditional objective under its stated assumptions.Source: §C.1 · p. 19

C.2 Theorem statement (with explicit constants)

The appendix gives an explicit-constant convergence bound for idealized stochastic ascent on a fixed conditional objective, then relates truncation depth and the DRILL-VWI bias to practical estimator choices. It also describes the training loops and validates the random-projection PUR estimator.

C.2 Theorem statement (with explicit constants)

  • Under Assumptions A4–A6, the idealized stochastic-ascent iterates on the fixed conditional objective satisfy the stated convergence bound.The result uses a constant step size and a finite objective gap; the supplied passages do not state the bound’s full expression.Source: §C.2 · p. 19

C.3 Proof

  • The proof controls ascent using smoothness, estimator bias, and second moments, then telescopes over outer iterations to balance step-size-dependent terms against truncation bias.The truncation-bias term is independent of the outer step size.Source: §C.3 · pp. 19–20

C.4 Corollary and DRILL-VWI interpretation

  • At K =2, truncated-HVP cosine exceeds 0.7 on gridworld and Meta-World MT10, while downstream success rate plateaus; K =3 adds about 1.5× wall-clock cost for at most 1 point of success rate change.The authors use K =2 as an empirical return-per-compute choice; the bound itself is loose in this regime.Source: §C.4 · pp. 20–21

D.1 DRILL-VWI full algorithm

  • DRILL-VWI samples demonstration anchors, imagines continuations, computes GRPO advantages and visitation/confidence weights, then updates the world model with a signed weighted prediction loss.Its stopped coefficients choose the update direction and magnitude; negative coefficients can move the model away from transitions judged harmful by the imagined policy-gradient signal.Source: §D.1 · p. 21

D.2 DRILL-IMG full algorithm

  • DRILL-IMG performs an imagined GRPO update, evaluates the updated policy using target-simulator rollouts, and sends credit to the world model through a K-step truncated HVP.The implementation describes gradient clipping, float32 HVP accumulation, and frozen normalization statistics as stability controls.Source: §D.2 · p. 22

D.3 PUR estimator

  • PUR estimates the outer world-model gradient with random-subspace finite differences, and its 128-direction estimate has cosine similarity 0.83±0.04 with the true gradient on gridworld and Meta-World MT10.The reported measurement uses 7,680 simulator rollouts and is logged every 500 outer iterations.Source: §D.3 · p. 22

D.4 Exact-vs-truncated HVP on low-dimensional environments

The appendix checks truncated meta-gradients against tractable exact gradients and documents the TRPS diagnostic, visitation estimator, training settings, simulator-budget control, and compute costs.

D.4 Exact-vs-truncated HVP on low-dimensional environments

  • At K=2, the truncated HVP estimator exceeds 0.7 cosine similarity with the exact meta-gradient on both test environments, with further gains below 0.07 for K≥3.The audit uses a 10×10 gridworld and Meta-World MT10, comparing K=1–5 across 1,000 states and 10 seeds.Source: §D.4 · pp. 22–23, Figure 4 · p. 23

D.5 Task-relevant prediction score (TRPS)

  • TRPS measures the fraction of rollout steps whose imagined and simulator reward-head progress labels match, complementing one-step pixel MSE.It uses a frozen VLM reward head, three labels, and 500 held-out transitions per checkpoint averaged over five seeds.Source: §D.5 · p. 23

D.6 Visitation estimator

  • The visitation estimator uses nearest neighbors in the frozen world model’s latent space to estimate model-rollout visitation.Latents are L2-normalized; the index uses k=16 neighbors and is refreshed online.Source: §D.6 · p. 24

D.7 Hyperparameters

  • Shared training settings include GRPO group size 8, AdamW for the outer loop, momentum SGD for the inner loop, and five headline versus three ablation seeds.The table also specifies gradient clipping, mixed precision, and effective batch construction.Source: Table 3 · p. 24, §D.7 · p. 24

D.8 Simulator-budget-matched Sim-PPO baseline

  • The Sim-PPO control matches DRILL-IMG’s simulator episodes at 30 per outer step, but not wall-clock time or gradient updates.Sim-PPO shares the VLA checkpoint and offline-demonstration warm start, then runs PPO without imagined rollouts.Source: §D.8 · pp. 24–25

D.9 Compute budget

  • The compute accounting covers five π0-3B and three OpenVLA-7B benchmarks with five seeds each; DRILL-IMG’s marginal cost over WMPO is the outer-loop phase alone.The runs used a 24-GPU H20 cluster, and the outer loop accounts for approximately 3.8k H20-hours.Source: Table 4 · p. 25, §D.9 · p. 25

E.1 Reproducibility statement

The appendix documents materials and settings for reproducing the experiments, identifies the benchmarks and pretrained-model licenses, and limits the reported evaluation to simulation.

E.1 Reproducibility statement

  • The reproducibility package includes training and analysis code, headline checkpoints, per-benchmark hyperparameters, compute details, seeds, and human-evaluation procedures.The code and trained checkpoints are described as forthcoming releases; the pretrained VLA backbones are not redistributed.Source: §E.1 · pp. 25–26

E.2 Datasets and benchmarks

  • The benchmarks include LIBERO-Long, CALVIN ABC→D, RoboCasa atomic skills, Meta-World MT50 for the exact-HVP audit, and SimplerEnv subsets.The five benchmarks are described as publicly released and used unmodified under their authors’ licenses.Source: §E.2 · p. 26

E.3 Pretrained-model licenses

  • The appendix identifies the public π0-3B and OpenVLA-7B checkpoints and Ctrl-World world model, while noting that the paper does not redistribute the VLA weights.Source: §E.1 · p. 25, §E.3 · p. 26

E.4 Scope of evaluation

  • All quantitative results are from simulation, and the paper makes no claim about real-robot deployment or bridging the simulator-to-real gap.Source: §E.4 · p. 26

F Sensitivity, Failure Modes, and Additional Ablations

F.1 Auxiliary main-result checks

Auxiliary checks examine whether DRILL’s gain is explained by extra updates, which parts of the signed coaching objective matter, and how rollout horizon affects the variants.

F.1 Auxiliary main-result checks

  • Extending WMPO’s outer updates produces marginal, non-monotone changes, so the reported compute-matched control does not explain DRILL’s gain as extra updates or ordinary convergence speed.Source: §F.1 · p. 26, Table 5 · p. 27

F.2 Objective ablations

  • The ablations treat visitation, signed advantage, and confidence as complementary parts of the coaching surrogate: they select where to improve, the update direction, and low-support regions to avoid amplifying.Source: §F.2 · p. 27, Table 6 · p. 27

F.3 Horizon sweep and short-horizon checks

  • DRILL-IMG gains more over DRILL-VWI as LIBERO-Long horizon increases from 20 to 200, but both variants eventually suffer with noisy imagined rollouts.On LIBERO-Long, the gap is 0.6 points at H=10 and −0.2±1.1 at H=5.Source: §F.3 · p. 27, Table 7 · p. 27

F.4 Training diagnostics

The diagnostics separate pixel fidelity from task-relevant rollout quality and probe sensitivity, failure conditions, backbone replication, and reward-model variation.

F.4 Training diagnostics

  • WMPO has lower one-step pixel MSE, while DRILL-IMG has higher TRPS, matching the success-rate gap in the main text.Figure 5 presents rollout quality as a phase-space trajectory over time.Source: §F.4 · pp. 27–28, Figure 5 · p. 28

F.5 Hyperparameter sensitivity

  • On LIBERO-Long (1.2B WM, 5 seeds), DRILL-IMG reaches 81.3±1.3 SR at K=2; scores are 80.7±1.5 at K=3 and 80.2±1.6 at K=4, with wall-clock per outer step growing roughly linearly in K.Collapsed or uniform visitation weights recover WMPO-like behavior within 1 pt.Source: §F.5 · pp. 28–29

F.6 Failure modes

  • Below approximately 300M world-model parameters, DRILL-IMG’s +7.9±1.6-point gain is not significant on RoboCasa with five seeds; other tested settings also expose instability or failed alternatives.Large outer step sizes can cause HVP-related loss spikes, while imagined-rollout estimates of the outer policy gradient remove the reported +13.1-point gain.Source: §F.6 · p. 29

F.7 Cross-backbone replication on OpenVLA-7B

  • On OpenVLA-7B, the method ranking is preserved on LIBERO-Long and CALVIN, while DRILL-IMG and DRILL-VWI are statistically tied on RoboCasa.Pixel MSE again moves in the opposite direction from policy success.Source: §F.7 · p. 29, Table 9 · p. 30

F.8 Reward-model swap

  • Across three reward models on LIBERO-Long, the DRILL-IMG-versus-WMPO gap ranges from 15.6 to 18.4 points and is not quantitatively invariant.LLaVA-CoT-7B gives the largest gap and InternVL-reward-4B the smallest.Source: §F.8 · p. 30, Table 10 · p. 30

F.9 Out-of-distribution robustness

DRILL-IMG leads on zero-shot robustness in three of four LIBERO perturbation conditions, while DRILL-VWI marginally leads under pose perturbation.

F.9 Out-of-distribution robustness

  • The reported robustness metric is relative success-rate drop on LIBERO zero-shot perturbations, with lower values indicating better robustness.Source: §F.9 · p. 30, Table 11 · p. 30
  • Pose perturbation is the sole consistent exception to DRILL-IMG’s lead, matching the paper’s shorter-effective-horizon interpretation.Source: §F.9 · p. 30
  • DRILL-IMG has the highest robustness in three of four LIBERO zero-shot perturbation conditions; DRILL-VWI marginally leads under pose perturbation.The measure is relative success-rate drop, where lower is better.Source: Table 11 · p. 30, §F.9 · p. 30

F.10 Human evaluation

In human evaluation, raters did not significantly prefer DRILL-IMG’s visual quality, but preferred it when task completion was revealed. The checklist reports simulation-only results and documents experimental and human-study procedures.

F.10 Human evaluation

  • Raters strongly preferred DRILL-IMG when shown task completion, although visual-quality preferences against WMPO and RLVR-World were not significant at p<0.05.The task-completion preference was significant at p < 0.001 using McNemar’s test.Source: Table 12 · p. 30, §F.10 · p. 31
  • The human evaluation used six volunteer graduate-student raters, with method identity blinded and rollout pairs presented in random order.Source: §F.10 · p. 31

NeurIPS Paper Checklist

  • The paper reports that all quantitative results are in simulation and leaves sim-to-real transfer to future work.Source: NeurIPS Paper Checklist · p. 32
  • Headline results use five seeds, ablations use three seeds, and statistical claims use Welch’s t-tests and 95% confidence intervals.Source: NeurIPS Paper Checklist · p. 35
  • The checklist reports no IRB approval, exemption, or equivalent review for the internal human evaluation, and says the disclosure should be updated if one exists.Source: NeurIPS Paper Checklist · p. 38

Related papers

Loading 2610.30814v1…