PaperlayerView exact revision on arXiv

Source-linked AI summary

Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world models

Yuhua Jiang, Junjie Lu, Feifei Gao

arXiv:2609.13257v1cs.CVcs.AI

TL;DR

Test-time scaling can produce better candidates without improving the selected output, because a verifier may misrank candidates or extra actions may not be useful after cost. The paper introduces the Compute-Value Audit to test this decision chain. Across the studied video world models, sampling headroom does not reliably become deployment gain, though the sparse PRM800K control passes the audit.

  • Problem

    Test-time scaling can produce stronger candidates without improving deployed outputs when verifiers misrank pools, difficulty signals miss useful actions, or actions cost more than uniform allocation.

    Source: §1 · p. 1

  • Method

    The Compute-Value Audit treats TTS as four ordered tests—opportunity, state confidence, action value, and entry fee—and identifies the first unsupported link.

    Source: §1 · pp. 1–2

  • Results

    Across studied video world models, wider sampling creates oracle headroom without reliably improving deployment, while the sparse PRM800K population passes the full chain.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Scale compute only when an observable state supports an action whose gain clears total cost; headroom alone is a possibility, not a return.

    Source: §6 · p. 9

    The audit covers fixed pools, adaptive allocation, and one matched-NFE P&P intervention, not every learned policy; its failures do not rule out video TTS.

    Source: §5 · p. 9

Abstract

from arXiv · show

Test-time scaling (TTS) can improve generation only when additional compute produces better candidates and the system can reliably identify them. This distinction is especially important for video world models, where a wider sample pool may contain stronger rollouts without improving the output that is ultimately selected. We introduce the Compute-Value Audit (CVA), a sequential framework that asks whether extra sampling creates opportunity, observable signals provide a reliable state, that state supports a beneficial action, and the resulting gain exceeds the full entry fee of generation and verification. On 192 Physics-IQ scenes, expanding the pool from 4 to 16 candidates increases oracle quality by +9.23 IQ (95% CI [+7.44, +11.14]), but Flow, Cycle, and VideoReward fail to recover this headroom reliably. Across three generators, none of twelve adaptive-depth policies outperforms uniform compute; they recover only 42-69% of the measured entry fee. A matched-60-NFE Predict-and-Perturb intervention on VideoPhy2 is likewise negative across three fresh-seed replicas. These negative results are not universal: anchor-explorer passes all four stages in a sparse PRM800K setting, MMLU-Pro exposes the gap between predictive state and useful action, and a privileged paired future establishes a positive video upper bound. Together, these results show that sampling headroom has deployment value only when it can be converted into a reliable decision whose benefit survives the complete compute charge. Code is available at https://github.com/YuhuaJiang2002/sampling-headroom-is-not-selection-gain.

1 INTRODUCTION

The introduction distinguishes better candidates from better deployed outputs: scaling creates value only if a reliable decision converts opportunity into a gain that clears its compute fee. The Compute-Value Audit (CVA) diagnoses where that conversion fails and shows that failures are not universal.

1 INTRODUCTION

  • Oracle quality and selected quality answer different questions: a verifier can misrank a good pool, a difficulty signal may not identify a useful action, and an action may cost more than uniform allocation.Conflating these outcomes obscures where scaling loses value.Source: §1 · p. 1
  • CVA tests opportunity, state confidence, action value, and entry fee in order, identifying the first unsupported link rather than assigning a score or making a pick.It accounts for every generated and scored candidate and replays cached pools only as deterministic prefixes.Source: §1 · pp. 1–2
  • The paper frames TTS as a falsifiable value account, including laws relating recovered headroom to verifier alignment and memory value to heterogeneity and retrieval regret.Its four tests make information and compute boundaries explicit.Source: §1 · p. 2
  • Across three generators and seven evaluation populations, wider pools create substantial Physics-IQ headroom, but tested verifiers, adaptive-depth rules, and matched-60-NFE P&P do not reliably convert it to gain.The evaluation distinguishes oracle opportunity from deployable recovery.Source: §1 · p. 2
  • A sparse PRM800K population clears the entire chain, while a privileged paired future recovers value in video, showing the tested failures are not universal.These controls demonstrate that both the audit and missing information are consequential.Source: §1 · p. 2

2 RELATED WORK

Related work supplies methods for ranking, steering, and allocating additional inference compute, but their signals and actions must still be tested for net value. The paper situates its audit across video plausibility, verifier design, and test-time memory.

Inference-time search and verifiers

Physical plausibility

  • Video-T1 searches noise trajectories, Stream-T1 allocates compute across streaming chunks, and Proprio ranks completed rollouts with a frozen model’s flow residual.The paper uses Proprio as a fixed-pool baseline and adapts Self-Refining Video Sampling to matched-NFE FlowMatch P&P; OMR requires its own matched-runtime study. Physics-IQ and VideoPhy2 measure complementary physical properties, while VideoReward is an external preference-model baseline.Source: §2 · pp. 2–3

Verifier design

  • Self-consistency and reconstruction energies use a model’s own predictions as label-free scores, but internal consistency need not imply physical correctness.The paper tests held-out alignment and width robustness rather than assuming the cycle residual is valid.Source: §2 · p. 3

Test-time memory

  • Prior test-time memory methods combine rapidly written experience with more slowly consolidated state; this paper instead keeps the world model frozen and uses memory only to rerank a complete pool.PM compresses prior outcome comparisons into semantic state, makes episodic adaptation optional, and is tested against matched shuffle controls.Source: §2 · p. 3

3 THE COMPUTE-VALUE AUDIT

The Compute-Value Audit follows value from candidate generation to deployment, separating available opportunity from recoverable value. Its methods fix the comparison budget, test verifier alignment and memory-based decisions, and account for the full compute fee.

3 THE COMPUTE-VALUE AUDIT

  • CVA fixes a complete candidate budget, separates available from recoverable value, and uses four ordered gates to locate where they diverge.This follows value from generation to deployment.Source: §3 · p. 3

3.1 ONE CANDIDATE BUDGET

  • Every selector scores all B generated candidates and may inspect nothing outside that set, ruling out hidden generation and making pool width the decision set.Fixed seeds make candidates and their original noise replayable; no-TTS is B = 1.Source: §3.1 · p. 3

3.2 CVA PROTOCOL: OPPORTUNITY, STATE, ACTION, AND FEE

  • CVA tests opportunity, state confidence, action value, and fee in order; adaptive allocation must beat matched-random assignment and uniform compute after paying its fee.A predictive state is not a successful action, and oracle quality remains an upper bound rather than a deployable rule.Source: §3.2 · p. 3

3.3 VERIFIER QUALITY PRICES RECOVERABLE HEADROOM

  • A minimal Gaussian model shows that recoverable headroom depends directly on verifier–target alignment; using the within-pool random mean as reference isolates ranking quality from candidate count.The analysis conditions on the selected index and tests empirical alignment with Pearson correlation.Source: §3.3 · pp. 3–4

3.4 FLOW RESIDUAL BASELINE

  • Proprio is a one-step flow residual measuring sensitivity to controlled latent perturbations, and it selects the candidate with the lowest residual.The moving-region weight m(z) emphasizes moving regions.Source: §3.4 · p. 4

3.5 MULTI-STEP CYCLE CONSISTENCY

  • The cycle score re-noises a latent and integrates it back with K Euler steps to measure reconstruction error; lower is better and the first frame stays clamped.Wan2.2-5B and ABot-PhysWorld-14B use native cycle readings, while Cosmos3-Nano uses analogous denoising-x0 error; raw velocity magnitude is not ranked because time reparameterization rescales it.Source: §3.5 · p. 4

3.6 MEMORY-BASED SELECTOR: PROPRIOCEPTIVE MEMORY (PM)

  • PM keeps the model and verifier frozen, uses prior outcomes for a slow correction and condition-local evidence for a fast adapter, and reranks the same candidates.Its common CVA-Select readout combines benchmark-specific global and motion evidence; adapters are checked with shuffle and wrong-task controls.Source: §3.6 · pp. 4–5

3.7 MEMORY VALUE IS HETEROGENEITY MINUS RETRIEVAL REGRET

  • Memory beats the best fixed selector only when retrieval regret stays below contextual heterogeneity; it has decision value only when it changes the optimal action after costs.Memory can fail because little heterogeneity is available or because retrieval regret consumes it.Source: §3.7 · p. 5

4 EXPERIMENTS

The experiments follow the Compute-Value Audit from fixed-pool verification through reasoning and video controls to compute allocation, asking where potential value stops becoming usable value. Across these tests, wider pools create oracle headroom, but tested selectors and allocation rules do not reliably convert it into deployable gains.

4.1 EXPERIMENTAL PROTOCOL

  • The protocol freezes picks before opening target outcomes, uses matched candidate pools where specified, and accounts separately for generation, same-model verification, and external reward-model calls.Physics-IQ retains 192 of 198 scenarios; PAI-Bench-robot uses eight candidates on 174 scenes.Source: §4.1 · pp. 5–6

4.2 VERIFIER RECOVERY FROM FIXED CANDIDATE POOLS

  • CVA-Select’s descriptive point estimate exceeds Native by 3.28/2.44/1.28 IQ across the three generators, while verifier–target correlation explains R2 = 0.629 of recovered-headroom variation.The paired-future bound remains higher everywhere, and CVA-Select was chosen after inspecting component outcomes.Source: Table 1 · p. 6, §4.2 · p. 6

4.3 WHERE THE CHAIN BREAKS ACROSS BENCHMARKS

  • On MMLU-Pro, disagreement predicts difficulty but allocating more compute to it does not beat matched uniform allocation; PRM800K anchor–explorer survives candidate- and token-matched controls.The PRM800K gain is concentrated on a small subset of problems and support is sparse.Source: §4.3 · p. 7, Table 2 · p. 7
  • On the VideoPhy2 ladder, the oracle establishes actionable heterogeneity, but tested deployable states stop at action value, closing only the tested information class.Every deployable state is tested out of fit; the authors do not claim to rule out every possible video state.Source: Table 3 · p. 7, §4.3 · p. 7
  • CVA-Select improves over no-TTS by 0.0104, 0.0036, and 0.0028 on PAI’s 5B, 14B, and Cosmos models, but on OpenS2V its 0.3992 mean is below the best fixed source at 0.4127.The OpenS2V result is post-hoc and descriptive; the RoboTwin retrospective LOEO gain does not reproduce in the forward test.Source: §4.3 · p. 8

4.4 DOES ADDITIONAL COMPUTE PAY FOR ITSELF?

  • From B = 4 to 16, the oracle gains +9.23 IQ, while the frozen control’s +2.75 IQ opportunity is erased by −3.36 IQ alignment; adaptive rules and matched-NFE P&P also fail to beat their controls.The selected CVA-Select score was not evaluated at B = 16; P&P lowers Joint success in all three fresh-seed replicas.Source: Table 5 · p. 8, §4.4 · pp. 8–9

4.5 A COMPLETE SELECTOR-COMPONENT ABLATION

  • Adding G or M changes the average by +0.77 or +0.48 IQ, while N changes it by −0.76; CVA-Select is only +0.05 over G alone.The frozen-row reanalysis motivates the common score without establishing synergy.Source: Table 6 · p. 9, §4.5 · p. 9

5 LIMITATIONS

The audit tests fixed-pool selection, adaptive allocation, and one matched-NFE P&P intervention, not every learned policy. Its positive controls are narrow, and some compute costs remain unmeasured.

5 LIMITATIONS

  • The tested failures localize the evaluated actions but do not rule out video TTS or establish outcomes for every learned policy.The audit covers fixed pools, adaptive allocation, and one matched-NFE P&P intervention.Source: §5 · p. 9
  • Positive controls have limited scope: paired futures are privileged, the PAI grouping renames frozen signals, OpenS2V is post-hoc and negative, RoboTwin fails fresh tests, and PRM800K support is sparse.The authors call for each deployment to remeasure opportunity and alignment.Source: §5 · p. 9
  • CVA-Select was foregrounded after comparing seven frozen B = 4 rows, and confirmation requires a fresh sealed population because its family-level interval and B = 16 picks are absent.All seven B = 4 picks predate IQ, but the selector was selected after their comparison.Source: §5 · p. 9

6 CONCLUSION

Across the studied video world models, wider sampling creates oracle headroom without reliably improving deployment. The audit’s conclusion is to scale compute only when an observable state supports an action whose gain clears total cost.

6 CONCLUSION

  • Across the studied video world models, verifiers leave sampling value unrecovered, adaptive rules do not repay uniform compute, and matched-NFE P&P remains negative.The conclusion is scoped to the studied video world models and tested interventions.Source: §6 · p. 9
  • The sparse PRM800K case is a positive control: CVA certifies a pass when state and action align.Source: §6 · p. 9
  • Treat the proposer, verifier, and allocation policy as one decision system; headroom alone is not a return unless the action’s gain clears total cost.Source: §6 · p. 9

AI USE STATEMENT

The paper discloses AI assistance while locating where different evaluation populations stop in the CVA audit; sparse PRM800K anchor–explorer clears all four gates.

AI USE STATEMENT

  • The authors report using generative AI for research assistance and manuscript drafting, while stating that every reported number came from executable code under a frozen protocol.The authors say they verified all claims and take responsibility for the manuscript.Source: AI USE STATEMENT · p. 10

A MECHANISM-LEVEL EVIDENCE FOR THE AUDIT

  • The gate matrix stops diffuse at opportunity, MMLU-Pro at action value, and video at fee clearance, while PRM800K clears all four with sparse 7/128 support.The matrix presents row-local effects and identifies the first stopping gate.Source: Table 9 · p. 16, A MECHANISM-LEVEL EVIDENCE FOR THE AUDIT · p. 16
  • For Qwen3-VL-4B on 128 disjoint PRM800K problems, K8 consensus/oracle are 0.7891/0.8438.Uniform matches mean depth, and random matches exact depths.Source: Table 11 · p. 17

B DOES ADAPTIVE STOPPING REPAY ITS COMPUTE?

Across the tested adaptive-depth rules, none beats uniform compute at its own mean depth; the corrected comparisons show little evidence of gains beyond noise.

B DOES ADAPTIVE STOPPING REPAY ITS COMPUTE?

  • No adaptive-depth rule clears uniform compute at its own mean depth across the three generators; the table shows the best of 12 rules per generator at B = 8.Five of six reversed statistics beat random on each generator, but no rule beats uniform.Source: B DOES ADAPTIVE STOPPING REPAY ITS… · p. 17, Table 13 · p. 17
  • Signal–remaining-headroom Spearman correlations range from −0.165 to +0.901 across 18 statistic–pool pairs.The range shows substantial variation across the tested signals and pools.Source: B DOES ADAPTIVE STOPPING REPAY ITS… · p. 17
  • 96% of 132 rule–budget–generator comparisons remain inside noise after Bonferroni correction, with three positive resolutions and a largest gain of +0.183.There is no corrected win for B ≥4; at B = 2, one result survives the global correction.Source: B DOES ADAPTIVE STOPPING REPAY ITS… · p. 17

C COMPUTE-MATCHED TRAJECTORY INTERVENTION

The matched-compute P&P intervention fails its preregistered primary gates and is negative across three replicas. Development evidence further limits the video findings to the tested populations and action-value information class.

C COMPUTE-MATCHED TRAJECTORY INTERVENTION

  • Both P&P and uniform use exactly 60 denoiser evaluations and 120 CFG transformer forwards, with the same prompt seed and initial latent and no target feedback.The frozen evaluator was opened only after all 358 videos were hash-bound.Source: C COMPUTE-MATCHED TRAJECTORY… · p. 18
  • 0/3 P&P replicas are nonnegative, with a mean effect of −1.49 pp; the primary run fails its action-value, uncertainty, sign, PC, SA-noninferiority, and groupwise gates.Table 14 reports paired percentage-point differences across all 179 hard prompts per run.Source: Table 14 · p. 18, C COMPUTE-MATCHED TRAJECTORY… · p. 18

D HOW THE EVIDENCE CHANGED DURING DEVELOPMENT

  • The video transfer diagnostics establish a boundary for video, not a universal law; the authors withdraw their earlier unification and note that the deep-pool effect still shows no gain.The section distinguishes retained evidence from interpretations later narrowed or withdrawn.Source: D HOW THE EVIDENCE CHANGED DURING… · p. 18
  • Anchor–explorer passes its constructive reasoning audit, while MMLU-Pro confirms a gap between state confidence and action value; video fee localization recovers only 41.6–69.0%.The attempted Qwen3.5 transfer is invalid, and the one-point compound prediction is not confirmed.Source: D HOW THE EVIDENCE CHANGED DURING… · p. 18
  • The retrospective cross-regime state–action matrix localizes failed CVA gates but cannot broaden a population claim; the trap cannot arise when no alternative action exists.The matrix is a fixed synthesis, not an additional confirmatory result.Source: D HOW THE EVIDENCE CHANGED DURING… · p. 19
  • The diffuse replication has +5.86 points of oracle headroom but only 12/256 positive contributions and a top-10%-trimmed effect of −0.42 points; it supports neither diffuse nor cross-source generality.The authors characterize the replication as invalid.Source: D HOW THE EVIDENCE CHANGED DURING… · p. 19
  • The direct structured set predictor fails the video action-value gate before fresh evaluation, with a +0.27-point advantage (p = .572) versus +3.07 points for target-oracle routing.The tested representation ladder and candidate-specific preview likewise did not justify fresh video generation.Source: D HOW THE EVIDENCE CHANGED DURING… · p. 19

E PM IMPLEMENTATION DETAILS AND EVIDENCE BOUNDARY

The appendix separates deployable memory and routing claims from privileged references, then tests prospective memory and seed-screening routes under explicit controls. The reported evidence leaves several proxy gains non-deployable or unconfirmed, while the tested pretrained memory does not beat ridge.

E PM IMPLEMENTATION DETAILS AND EVIDENCE BOUNDARY

  • The paired-future reference exceeds Native by +4.68/+4.12/+3.25 IQ, but family-held-out retrieval is worse than train-global history by −1.87 IQ.The positive reference uses an exactly paired real future; it does not establish held-out memory generalization.Source: E PM IMPLEMENTATION DETAILS AND… · p. 20
  • PAI and RoboTwin reward-tier memory fail their gates, while the E28 transition-score gain is diagnostic rather than label-sealed fresh evidence.The E28 result fails wrong-task, global-history, strict-win, and official7-noninferiority gates; 72/112 candidate hashes matched an earlier pool.Source: E PM IMPLEMENTATION DETAILS AND… · pp. 20–21
  • The strengthened stopping and residual variants retain unresolved gates: the post-outcome oracle is not deployable, and the repaired task-residual CI versus wrong-task crosses zero.The task-residual development effect is +0.001314 (CI [−0.000230, +0.003191]), while its wrong-task comparison is +0.000874 (CI [−0.000753, +0.002456]).Source: E PM IMPLEMENTATION DETAILS AND… · p. 21
  • Episode 29 confirms success-region proxy alignment, but its official7 loss makes that proxy-target value non-deployable.It beats wrong-task memory and task derangements, yet is indistinguishable from global success memory and reduces official7 versus Ordinary by −0.007112.Source: E PM IMPLEMENTATION DETAILS AND… · pp. 19, 21
  • On 5B, Semantic Slow PM and ordinary RM are identical; on 14B, PM–RM is −0.000705 (CI [−0.002508, +0.000639]).The 14B PM–Native comparison is +0.001086 (CI [−0.001728, +0.004211]), but the matched-RM comparison is the final control.Source: E PM IMPLEMENTATION DETAILS AND… · pp. 20–21
  • None of six frozen routing keys passes its familywise gate; the oracle improves task routing by +0.001093, but pilot memory makes zero swaps.The conclusion is limited to the frozen signals tested; a future fresh early-denoise uncertainty probe remains untested.Source: E PM IMPLEMENTATION DETAILS AND… · p. 21

F PROSPECTIVE PRETRAINED GLOBAL CAPABILITY MEMORY

  • Pretrained Global Capability Memory uses a frozen shared public bank without target examples or labels, but both source estimands yield +0.00 IQ versus ridge.Private PAI labels remain open, and confirmation requires a new private split of at least 100 scenarios.Source: F PROSPECTIVE PRETRAINED GLOBAL… · pp. 22–23

G MEMORY-CONDITIONED SEED SCREENING AND VELOCITY DIAGNOSTICS

  • Memory-conditioned seed screening preserves independent production-prior noise, but Equation (17) is prospective rather than a reported gain, and no positive raw-velocity claim is made.The protocol matches ordinary-preview ranking and includes history-shuffle and motion, sharpness, and diversity checks.Source: G MEMORY-CONDITIONED SEED SCREENING… · pp. 23–24

H REPRODUCIBILITY DETAILS

The appendix binds the paper’s headline comparisons to frozen artifacts, declared uncertainty procedures, and independent audit trails. It documents how the analyses preserve information boundaries and reconstruct reported results.

H REPRODUCIBILITY DETAILS

  • The full-factorial reanalysis uses seven already-frozen Physics-IQ B = 4 rows and marks CVA-Select descriptive, while the separate paired-future diagnostic is privileged.The records also specify held-out retrieval controls, a post-hoc calibration split, and a frozen PAI eight-dimensional artifact.Source: H REPRODUCIBILITY DETAILS · pp. 24–25
  • Headline width and PAI comparisons are reconstructed from frozen candidate identities, with 50,000 paired event-family draws for Physics-IQ and paired scene draws for PAI.The implementation record also specifies the relevant intersections, hashes, and audit procedures.Source: H REPRODUCIBILITY DETAILS · p. 25
  • The verifier-law test uses five deterministic folds over 66 event families, with family-resampled bootstrap uncertainty retaining each sampled family’s full set of evaluation points.Motion-strata uncertainty resamples 33 held-out families; calibration and fixed-compute analyses have separately specified splits and bootstrap procedures.Source: H REPRODUCIBILITY DETAILS · p. 25
  • The task-local confirmation freezes six arms and 256 wrong-task assignments before evaluation scores are created, then bootstraps tasks rather than scenes.Its analyzer and runner outputs are byte-identical, and the implementation record gives frozen-pick, result, and independent-audit hashes.Source: H REPRODUCIBILITY DETAILS · p. 26

I DETAILED INTERFACES AND COMPUTE ACCOUNTING

This appendix details the common selector interface and compute accounting, then reports complete PAI dimensions and an OpenS2V matched-pool reanalysis. In OpenS2V, the unified selector’s mean is below the best fixed source.

I DETAILED INTERFACES AND COMPUTE ACCOUNTING

  • The cross-benchmark interface uses the same within-pool sum while allowing benchmark-specific definitions of global evidence G and oriented motion evidence M.The interface is not a claim that the benchmarks’ measurements are interchangeable.Source: Table 22 · p. 26, I DETAILED INTERFACES AND COMPUTE… · p. 26
  • Per-scene compute accounting separates world-model DiT forwards from external reward-model calls; cycle verification uses 10 evaluations per PIQ candidate and 12 per PAI/RT candidate.PM distances are CPU-only.Source: Table 23 · p. 26

J COMPLETE PAI DIMENSIONS

  • The PAI dimension table reports official means over 174 scenes at B = 8, while the component ablation is a post-hoc exploratory audit with 50,000 scene-bootstrap draws.The component table’s picks are frozen without official labels, and no-TTS at B = 1 anchors its score scale.Source: J COMPLETE PAI DIMENSIONS · p. 26, Table 24 · p. 27, Table 25 · p. 27

K OPENS2V-EVAL MATCHED-POOL CVA-SELECT REANALYSIS

  • CVA-Select’s OpenS2V-Eval mean was 0.3992 versus 0.4127 for the best fixed source, with paired category-stratified difference −0.0197 (95% interval [−0.0301, −0.0096]).The matched pool covers 180 Open-Domain tasks at a fresh generation seed; this is a within-pool evaluation, not a public leaderboard submission.Source: K OPENS2V-EVAL MATCHED-POOL CVA-SELECT… · pp. 27–28, Table 26 · p. 28

L WHY THE PAI MEMORY VARIANTS WERE NOT PROMOTED

The PAI memory variants and related controls do not establish a robust memory-based deployment advantage. The appendix also states complete-budget accounting rules, reports held-out PAI ordering, and explains why generation and verification counts are not separated.

L WHY THE PAI MEMORY VARIANTS WERE NOT PROMOTED

  • Model-specific PAI fitting has scene-clustered mean effect −0.000024 (CI [−0.000208, +0.000155]), while absolute-residual PM’s gated 8D delta is −0.000169 (CI [−0.000446, +0.000038]).Only 1/5 training gates passes for absolute-residual PM.Source: L WHY THE PAI MEMORY VARIANTS WERE NOT… · p. 28
  • Pairwise repair changes 43/174 PAI picks and yields seven-dimensional delta +0.000399 (CI [−0.000272, +0.001126]); only 1/5 certification folds passes.A later risk diagnostic was proposed after the outer result and is not independent confirmation.Source: L WHY THE PAI MEMORY VARIANTS WERE NOT… · p. 28
  • On task-disjoint RoboTwin tasks, DEQM and Candidate-DEQM fail their matched shuffled-history specificity gates, so their gains cannot be assigned independently to memory.The later fresh-episode result supersedes the relocation diagnostic for independent-PM inference; PM does not beat ordinary RM on 14B.Source: L WHY THE PAI MEMORY VARIANTS WERE NOT… · pp. 28–29
  • Four prior outcomes expose task identity, but task-local residual reranking does not yield a robust or uniformly safe deployment rule on the 14 evaluation tasks.Its mean improvement over the global model has a task-bootstrap interval crossing zero, and it scores −0.001191 versus the parent pairwise proposer without current history.Source: L WHY THE PAI MEMORY VARIANTS WERE NOT… · p. 29

M COMPLETE-BUDGET ACCOUNTING CHECKLIST

  • The accounting checklist requires best-of-B to generate and score B candidates, and treats an oracle on the same B candidates as evaluation-only.No reported method ranks a generated set larger than B before scoring a subset.Source: M COMPLETE-BUDGET ACCOUNTING CHECKLIST · p. 29

N ADDITIONAL PAI HELD-OUT RESULTS

  • On the last 94 sorted PAI scenes, held-out unified-B = 8 ordering matches the all-scene result, with the complete zero-memory ensemble above every baseline.The five reported means are 0.7530/0.7551/0.7566/0.7607/0.7615 for no-TTS, Proprio, VideoReward, the zero-memory ensemble without i2v, and the complete zero-memory ensemble.Source: N ADDITIONAL PAI HELD-OUT RESULTS · p. 29

O WHY SEPARATE VERIFICATION COUNTS ARE OMITTED

  • When video generation dominates cost, reducing scored candidates while holding generated candidates fixed is not meaningful end-to-end scaling; the protocol sets G = V = B.Larger generated pools followed by scoring a subset are excluded from the reported method and headline tables.Source: O WHY SEPARATE VERIFICATION COUNTS ARE… · p. 29
  • The paired-future audit reports +3.58 IQ for paired-future minus held-out PIQ, while the PAI5B difference is +0.000069 and nonsignificant.The paired future is a privileged diagnostic.Source: O WHY SEPARATE VERIFICATION COUNTS ARE… · p. 31, Table 17 · p. 31
Loading 2609.13257v1…