PaperlayerView exact revision on arXiv

Source-linked AI summary

SCOPE: Score-Isolated Agentic Optimization for Video World Models

Yuhua Jiang, Jiaming Wang, Qingbin Liu, Feifei Gao

arXiv:2608.15043v1cs.AI

TL;DR

Inference-time controls for frozen video world models can evolve together, making gains difficult to attribute and risking adaptation to held-out evaluation. SCOPE formalizes bounded, evidence-based control updates and freezes deployment before evaluation; it improves over Frozen Base on Physics-IQ, but gains do not transfer uniformly across backbones and settings.

  • Problem

    Coupled inference controls make it difficult to attribute improvements and prevent held-out outcomes from shaping the deployed policy.

  • Method

    SCOPE uses a typed control state, bounded development-evidence updates, and a fixed deployment route before held-out evaluation.

  • Results

    SCOPE substantially improves over exact Frozen Base on Physics-IQ, while gains are not uniformly transferable across backbones and settings.

  • Takeaways & Limitations

    Reliable inference-time adaptation requires principled selection of which candidate interventions become part of the deployed policy.

  • Takeaways & Limitations

    Control effectiveness is not fully invariant across backbones or metrics, limiting transfer of adaptations learned in one setting.

Abstract

from arXiv · show

Video world models are increasingly used as simulators for planning and embodied decision making, yet improving them at inference time introduces a subtle evaluation problem: prompts, samplers, verifiers, and selectors may evolve together, making it difficult to attribute gains or prevent held-out feedback from shaping the final policy. We introduce \scope (\emph{\scopefullname}), a framework for auditable inference-time adaptation of frozen video world models. \scope represents external controls as a typed state, updates this state only through bounded changes supported by development evidence, and freezes the resulting policy before held-out evaluation. On Physics-IQ benchmark, \scope improves over the exact frozen base by $+14.24$ (95\% CI $[+8.10,+21.23]$). Controlled ablations further identify gains from scene specification, sampling, and learned selection, while the margin over the strongest matched agentic baseline remains unresolved. Cross-backbone and prospective evaluations reveal a complementary result: useful inference-time updates exist, but their benefits do not transfer uniformly across models and settings. Together, these findings suggest that reliable inference-time adaptation requires not only better proposals, but also a principled mechanism for deciding which updates should become part of the deployed system. Code is available at https://github.com/YuhuaJiang2002/SCOPE.

1 INTRODUCTION

SCOPE addresses the inference-control evaluation gap in frozen video world models by making external controls explicit, bounded, evidence-based, and frozen before held-out evaluation. On Physics-IQ, it improves over exact Base by +14.24, while cross-backbone and baseline comparisons leave transferability and the strongest matched baseline margin unresolved.

  • Motivation: Inference-time adaptation improves frozen video world models through external controls, but visually plausible predictions may not be physically consistent and retraining is expensive.Relevant controls include prompts, sampling strategies, verifiers, retrieval, and candidate selection.
  • Problem: Coupled control changes obscure attribution, while reusing held-out outcomes can adapt the system to the evaluation itself.This constitutes the inference-control evaluation gap.
  • Method: SCOPE represents text, sampler, verifier, and reward controls as typed state, accepts only bounded development-supported updates, and fixes the final route with an exact Frozen Base fallback before held-out evaluation.Its score-isolated update rule prevents held-out scores from changing the deployed state.
  • Results: +14.24 over exact Base is achieved by the complete procedure on the 40-scene Physics-IQ common-base protocol, reaching a P-IQ score of 34.94.The confidence interval for the improvement is [+8.10, +21.23].
  • Results: SceneLang conditioning, a sampler modification, and supervised best-of-N selection each improve their matched controls, while the margin over the strongest matched Qwen-Image-Agent-style baseline remains unresolved.The study also reports matched experiments across controls, backbones, and benchmarks.

2 RELATED WORK

SCOPE relates to physics-aware generation, inference-time alignment, agentic tool orchestration, persistent skill adaptation, and adaptive evaluation, while distinguishing itself through evidence-bounded, score-isolated updates to a frozen model.

  • Physics-aware video generation: Physics-aware video-generation methods inject physical supervision, motion priors, geometry rewards, or external knowledge, while related benchmarks stress-test physical reasoning and future-state prediction.These approaches primarily modify training, augment generation, or evaluate physical capabilities.
  • Inference-time alignment and scaling: Inference-time alignment methods rank or steer samples with critics, while iterative refinement and repeated sampling increase test-time compute through different operators.The passage contrasts these mechanisms with SCOPE’s broader control of inference components.
  • Agentic generation and tool orchestration: Agentic generation systems improve visual generation through feedback, planning, interactive world generation, tool calls, and reusable workflow experience.Examples include Qwen-Image-Agent, NEWTON, LingBot-World, GenEvolve, and COMFYCLAW.
  • Persistent skills and self-evolving harnesses: Persistent-skill methods update or select reusable agent skills, while related work identifies failure modes in evolving skill libraries and harnesses.SCOPE similarly maintains persistent state but restricts updates to development evidence and keeps validation and confirmation read-only.
  • Adaptive evaluation and safe updates: Adaptive-evaluation research motivates SCOPE’s score-isolation principle by studying repeated adaptive queries and conservative lower bounds for safe policy deployment.SCOPE prevents direct adaptation to held-out scores, although repeated development on the same ledger can still overfit; prospective evaluation uses disjoint tasks.

3 METHOD

SCOPE adapts a frozen video world model by optimizing an explicit external control state through bounded, development-only edits. The resulting policy is frozen and auditable before held-out evaluation, ensuring score isolation and transition traceability.

  • Control-state formulation: SCOPE keeps the world model parameters frozen and adapts external controls spanning directives, sampling, verification or abstention, and reward-based selection.These controls form an explicit, independently editable persistent state rather than modifying the world model itself.
  • Frozen deployment: The final control state and complete deployment route, including the exact Frozen Base fallback, are fixed before held-out scores are revealed.Learned components and optional memory may be developed beforehand, but cannot be updated using held-out outcomes.
  • Bounded adaptation: Each proposal is a bounded add, delete, or replace operation on one declared control axis, making individual interventions attributable.The proposer may use VLM feedback, retrieval, external tools, or predefined proposal families, but only through the defined state-modification interface.
  • Bounded adaptation: An edit is committed only if it improves the predeclared development contrast and satisfies every guard; otherwise, the incumbent state is retained exactly.Guards include motion, sharpness, identity preservation, and static-video checks.
  • Score isolation: The score-isolation invariant guarantees that changing held-out scores leaves the final control state and every frozen deployment route unchanged before score release.The guarantee is deliberately narrow: it does not establish proposal optimality, unbiased development selection, generalization, or superiority over matched alternatives.
  • Auditable transitions: Provenance binding records each attempted transition’s incumbent state, proposal, development evidence, and gate decision, distinguishing proposed edits from committed changes.Changing development evidence creates a distinct provenance-bound transition that must be evaluated independently, whereas changing held-out scores cannot alter the deployed state.

4 EXPERIMENTS

Across two frozen video backbones and multiple evaluation settings, SCOPE substantially improves over Frozen Base, while gains versus matched alternatives and individual controls are often unresolved or backbone-dependent. Prospective tests further show that development-time component gains do not yet yield robust improvement.

  • Evaluation setup: Experiments evaluate SCOPE with Wan2.2 and CogVideoX on Physics-IQ, P-AI, and OpenS2V-Eval under fixed within-backbone protocols.Physics-IQ uses 40 scenes; P-AI uses robot split6 tasks with the official Qwen2.5-VL-72B judge; OpenS2V-Eval contains 180 items per backbone.
  • Primary results: +2.07 over Qwen-Image-Agent-style feedback remains unresolved, with CI [−0.27, +6.14].SCOPE scores 34.94 versus 32.87 for the strongest non-SCOPE method, and the direct contrast changes only 8/40 scenes.
  • Policy evolution: Physics-IQ rises across policy states from 20.70 to 34.94 on Wan2.2 and from 22.91 to 35.51 on CogVideoX, despite an intermediate CogVideoX decrease.The trajectories describe evolution relative to Base, not comparison with the strongest matched alternative.
  • Cross-backbone analysis: Cross-backbone effects are non-invariant: shared-global changes Overall by +0.80 on Wan2.2 but −2.41 on CogVideoX, while Weighted differences shift from +0.33 to −0.40.All four confidence intervals for the Wan/CogVideoX Mean and Weighted contrasts include zero, and heterogeneous protocols are reported separately rather than pooled.
  • Prospective and component tests: Prospective evaluation finds no clear improvement over Base or the frozen router; typed updates fail to beat matched random updates, degrade 15% of scenes, and PhyGround yields valid proposals on only 4/32 tasks.Grounded routing improves over always-on tool use, while its no-tool and same-budget random contrasts remain unresolved; tool-use gains do not explain the overall Physics-IQ improvement.

5 LIMITATIONS

SCOPE’s main limitations are unreliable deployment selection, non-transferable control effects across backbones and metrics, and dependence on development-task coverage. Frozen Base abstention helps mitigate these issues, but reliable decision-making remains unresolved.

  • Deployment selection: Useful candidate proposals do not ensure reliable deployment decisions because selection errors and insufficient proposal coverage limit performance on unseen tasks.Several controls improve under matched evaluation, yet prospective experiments show that deciding when to apply them is harder than generating candidates.
  • Transferability: Inference-time controls are not fully invariant across backbones or metrics, with some updates producing smaller or reversed effects in other configurations.This limits treating controls learned in one setting as transferable adaptation rules.
  • Development evidence: SCOPE’s deployed policy depends on the diversity and representativeness of development tasks, despite held-out scores being unable to modify the policy directly.More reliable adaptation requires task-disjoint proposal generation, calibrated uncertainty estimates, and selectors that generalize under distribution shift.
  • Fallback and future work: Frozen Base fallback allows abstention from uncertain updates, partially mitigating these limitations, but reliable update decisions remain a central future-work challenge.Abstention avoids forcing every proposal into deployment, yet decision reliability still needs improvement.

6 CONCLUSION … A.5 SELECTIVE GROUNDED ROUTER

SCOPE frames inference-time adaptation of frozen video world models as a controlled, auditable optimization process with a frozen deployment policy. Its evaluations show useful but non-uniform gains, while selective routing avoids always-on harm without resolving superiority over no-tool or random policies.

  • 6 CONCLUSION: SCOPE separates proposal, update, deployment, and evaluation by representing external controls as structured state updated from development evidence before deployment freezing.The control state includes prompts, sampler configurations, verifiers, and reward selectors.
  • 6 CONCLUSION: SCOPE improves over the exact Frozen Base on Physics-IQ, with ablations attributing useful contributions to scene specification, sampling, and learned selection.The strongest matched agentic alternative remains unresolved, and gains do not transfer uniformly across backbones and prospective settings.
  • 6 CONCLUSION: Reliable agentic inference requires deciding which candidate interventions become deployed-policy updates, not merely generating better interventions.SCOPE presents inference-time adaptation as controlled optimization rather than an unstructured sequence of post-hoc improvements.
  • A.1 PROSPECTIVE PAI EVALUATION: The frozen conformal-risk-control stop test uses 96 development tasks and a disjoint 48-task confirmation set, with strict task-LOO proposals and fixed CRC thresholds.At every threshold, the protocol commits the same two tasks, or 2/96.
  • A.2 COMPLETIONGUARD SUCCESSOR EVALUATION: The CompletionGuard-and-ContinuityGuard successor test makes only two non-fallback proposals across 96 fresh development tasks, with both non-harmful, QS-only improvements and zero RO gain.The confirmation set contains 28 separate tasks, and both guards are frozen before generating the three evaluation arms.
  • A.3 PHYT2V EVALUATION: 0.00 selected-minus-incumbent gain was observed on the 40-scene PhyT2V set, although selected-minus-fixed-round2 was +1.37 (CI [−4.04, +7.05]).The GPT-5.4 incumbent and selected P-IQ both scored 33.08, and no eligible round3 proposal was promoted.
  • A.4 PROSPECTIVE INTEGRATED-SYSTEM EVALUATION: +0.096 over Base was achieved by full SCOPE on 32 fresh task-disjoint Physics-IQ tasks, while retirement versus commit-plus-fallback was −0.773 (CI [−1.471, −0.203]).Full SCOPE was −1.019 versus a simple frozen router, with CI [−2.159, +0.146].
  • A.5 SELECTIVE GROUNDED ROUTER: +0.660 fit-group versus always-on was the only preregistered pooled contrast with a confidence interval excluding zero; fit-group versus no-tool and same-budget random remained unresolved.The corresponding contrasts were +0.083 (CI [−0.191, +0.381]) and +0.002 (CI [−0.281, +0.299]).

A.6 PHYGROUND APPLICABILITY ANALYSIS

PhyGround’s preregistered selector failed its pre-score applicability gate because the frozen proxy generated too few non-incumbent proposals. A score-free audit attributes the failure mainly to global task-max conformal deduction and worst-head selection, while showing that post-failure replay cannot justify policy replacement.

  • Applicability gate: The pipeline stopped before reading target scores after producing 4/32 raw non-incumbent proposals, below the required minimum of 8.All 288 frozen target videos and score-blind features were completed.
  • Applicability gate: It produced 0 proposals at confidence ≥−0.25 and 0 at confidence ≥0, although law coverage passed for eight families.The corresponding minimums were 8 and 4, respectively.
  • Failure diagnosis: The audit identified global task-max conformal deduction (58.47, or 0.731 after gain normalization) with worst-head selection as the main driver, and observed 3/64 raw switches in held-out calibration.Motion and sharpness bound on no selected task; at that rate, an eight-of-32 gate has probability 8.9 × 10−5, but replay cannot justify replacing the policy.

B ADDITIONAL DISCUSSION

Component experiments isolate useful and unreliable inference controls, while prospective evaluations show that deployment depends on proposal coverage and selector transfer beyond development data. The findings caution against attributing the integrated Physics-IQ improvement uniformly to every control.

  • Component experiments: Text conditioning and sampler modification improve their matched comparisons, while the supervised reward selector outperforms matched random under scene-grouped OOF evaluation.These component results identify scene specification, sampling, and learned selection as useful parts of the inference-control space.
  • Component experiments: The region-state prior and matched web/SAM tool variants do not show reliable gains.The component experiments therefore distinguish useful controls from controls without reliable improvement.
  • Prospective experiments: Prospective deployment requires sufficient proposal coverage and a selector that transfers beyond its development distribution.The selective router avoids always-on tool-use degradation, but remains unresolved against no-tool and same-budget random routing; PhyGround proposes too few non-incumbent actions.

C REPRODUCIBILITY DETAILS · D EVALUATION PROTOCOLS AND STATISTICAL DETAILS

The paper links every reported result to reproducible scripts and machine-readable outputs while preserving protocol-local inputs. It also makes statistical units, evaluation roles, protocol boundaries, and primary comparisons explicit for each experimental setting.

  • C REPRODUCIBILITY DETAILS: Every reported result is linked to the scripts used to generate its corresponding table or figure.The release connects results directly to their generating scripts.
  • C REPRODUCIBILITY DETAILS: Every reported result is also linked to machine-readable outputs for its corresponding table or figure.These outputs support direct inspection of reported results.
  • C REPRODUCIBILITY DETAILS: The release preserves exact protocol-local inputs for each experiment.Inputs remain tied to the protocol under which each experiment was conducted.
  • C REPRODUCIBILITY DETAILS: Statistics are not aggregated across incompatible evaluation units.This preserves the validity of comparisons across distinct evaluation protocols.
  • D EVALUATION PROTOCOLS AND STATISTICAL DETAILS: Table 8 identifies the independent unit for each experimental setting.The table is designed to clarify the statistical unit underlying each setting.
  • D EVALUATION PROTOCOLS AND STATISTICAL DETAILS: Table 8 records each setting’s evaluation role and primary comparison.These fields make the intended comparison structure explicit.
  • D EVALUATION PROTOCOLS AND STATISTICAL DETAILS: The table makes statistical units and protocol boundaries explicit.Its purpose is to document the boundaries governing each evaluation setting.

E FULL PAI METRIC BREAKDOWN

Table 9 breaks down quality components aggregated by QS in the main PAI panel. Frozen Base is reported in absolute score units, while all other rows are signed differences from that Base and serve only as descriptive diagnostics.

  • E FULL PAI METRIC BREAKDOWN: Table 9 exposes quality components aggregated by QS in the main PAI panel.The breakdown is presented as a component-wise diagnostic.
  • E FULL PAI METRIC BREAKDOWN: Frozen Base is reported in absolute score units.Its values are not expressed as differences from another row.
  • E FULL PAI METRIC BREAKDOWN: Every other row reports a signed difference from the same Frozen Base.These component-wise values are descriptive diagnostics, not separately tested deployment criteria.

F ADDITIONAL QUALITATIVE ANALYSIS

The section presents a disclosed post-hoc paper-submersion intervention as qualitative analysis rather than a native generation result or benchmark comparison. It also documents the official-72B PAI quality-profile table and a web-grounded visualization using a shared Wan2.2 baseline.

  • Qualitative analysis: The paper-submersion visualization is explicitly qualitative and does not constitute a native generation result or benchmark comparison.It illustrates a disclosed post-hoc intervention.
  • PAI quality profiles: Table 9 reports complete official-72B PAI quality profiles, with Frozen Base as absolute values and all other rows as ∆vs Base.SC/BC denote subject/background consistency; MS motion smoothness; AQ/IQ aesthetic/imaging quality; OC overall text–video consistency; and IS/IB I2V subject/background fidelity.
  • Paper submersion intervention: Figure 4 compares two rows using the same strawfree seed-20260785 Wan2.2 baseline and a shared conditioning-frame inpainting step.The complete baseline leaves only the lower paper edge contacting the water, while the intervention compresses the full paper mask onto the dish bottom.
Loading 2608.15043v1…