PaperlayerView exact revision on arXiv

Source-linked AI summary

Budget-Constrained Embodied Perception: Four Resource Walls and a Pre-Registered Evaluation of Access-Structured Perception on Open Models at less than 31B

Defu Lin, Wenhui Chen, Ziyao Lin, Jianlin Chen, Peiji Long, Chi Man Vong

arXiv:2608.22975v1cs.AI

TL;DR

Embodied agents must answer from growing observation streams while every decision has a fixed token budget, raising whether access structure matters beyond model representation and context size. The paper formalizes four resource walls and wraps frozen multimodal models with ASP, combining structured state, verbatim retrieval, and query-conditioned allocation. On synthetic SEW-Bench, query-conditioned access strongly outperforms query-independent alternatives, but the full three-component architecture fails to validate channel duality.

  • Problem

    The paper asks whether the access-structure account transfers from symbolic streams to embodied multimodal streams under fixed per-decision budgets.

  • Method

    ASP is a training-free wrapper combining a compressive structured state, a verbatim episodic index, and a query-conditioned router for frozen multimodal models.

  • Results

    Budgeted query-conditioned access beats equal-budget query-independent alternatives by 17–66 points on every backbone, while ASP does not beat the verbatim-only baseline.

  • Takeaways & Limitations

    Under a fixed deadline, where the model looks matters more than simply increasing how much it samples, within the evaluated synthetic setting.

  • Takeaways & Limitations

    Natural-video benchmarks were not run because their frames require dataset agreements, so the evidence does not establish natural-scene perception performance.

Abstract

from arXiv · show

Embodied multimodal agents must answer from growing observation streams under a fixed per-decision token budget. We formalize this constraint through four resource walls: a perceptual Shannon wall for bounded state, a horizon wall for query-independent frame selection, a round wall for non-adaptive retrieval, and a conditional composition wall for fixed-depth inference. We introduce ASP, a training-free wrapper for frozen multimodal models that combines a capped structured state, a verbatim episodic index, and query-conditioned budget allocation with iterative access. Following a pre-registered protocol, we evaluate seven open-weight models from 3B to 31B on SEW-Bench, a license-free synthetic long-horizon walkthrough benchmark constructed to instantiate these walls. The registered natural-video benchmarks were not run because their frames require dataset agreements; our evidence therefore concerns access mechanisms, not natural-scene perception. Under a 4,096-token decision budget, ASP reaches 75 to 94% episodic retrieval accuracy, compared with 3 to 19% for equal-budget query-independent sampling, and budget reallocation outperforms quadrupling the sampling budget on every backbone. However, the full three-component architecture does not validate channel duality: removing the compressive state raises the flagship mean from 35.4 to 58.0, ASP does not outperform the verbatim-only baseline on any backbone, and two of four pre-registered falsification criteria fire. These results show that query-conditioned access, rather than parameter count or context growth alone, is decisive under a fixed budget, while prompted online compression does not earn its cost in this setting.

I. INTRODUCTION

The paper argues that embodied multimodal capability is constrained less by frame representation than by access across unbounded streams under fixed decision-time budgets. It formalizes four resource walls and evaluates ASP, finding strong benefits from query-conditioned access but not from its full channel-dual architecture.

  • Motivation: Fixed per-decision token budgets make selecting which observations to read unavoidable as egocentric streams grow beyond any finite context window.The paper charges prompts, retrieved pixels, and generated reasoning to one budget B and treats access structure as the bottleneck.
  • Motivation: The transfer from symbolic to embodied streams is nontrivial because frames have far higher bit density, temporal redundancy, episodic and spatial queries, and coarser decision-time budgets.These differences motivate testing whether the access-structure account survives in embodied multimodal settings.
  • Theory: The paper formalizes four walls: perceptual Shannon, horizon, round, and conditional composition, each ruling out a corresponding single-channel access strategy.The round wall concerns non-adaptive retrieval, while the composition wall concerns fixed-depth inference and is conditional on TC0 ≠ NC1.
  • Method: ASP combines a compressive structured state, a lossless verbatim episodic index, and a query-conditioned router that allocates one decision budget across wall prices.The wrapper is training-free and is designed to be access-complete for a stated witness family under stated assumptions.
  • Results: Budgeted query-conditioned retrieval beats equal-budget query-independent baselines by 17–66 points on every backbone, with all comparisons surviving Holm correction.The reported result attributes the strongest measured benefit to restructuring access rather than simply increasing model scale or context.
  • Scope: The registered natural-video benchmarks were not run because their frames require dataset agreements, so the evidence concerns access mechanisms on synthetic SEW-Bench rather than natural-scene perception.The paper explicitly marks those predictions as not evaluated.

III. THEORY: THE FOUR WALLS OF BUDGETED EMBODIED PERCEPTION

The paper defines budgeted embodied agents and shows that compressive memory, query-independent selection, non-adaptive retrieval, and fixed-depth inference each face distinct resource walls. The proposed access structure separates compressed state from verbatim storage and supports query-conditioned, iterative access to cross-frame information.

  • Setting: A budgeted embodied agent updates cross-frame state online and must answer each query using at most B backbone tokens.The access structure separates an O(1)-size compressive channel from a lossless verbatim channel.
  • Shannon wall: An o(Nb)-bit state forces episodic-query error toward 1 − 1/K as N grows, so constant-size memories cannot recover arbitrary past keyframes.Here b = d log2 K, and the bound applies to semantic summaries such as scene graphs, captions, embeddings, and learned latents.
  • Shannon wall: A compressive state stores at most m bits of the Nd log2 K queried stream bits, yielding the Shannon-wall bound through Fano’s inequality.The tightness construction stores floor(m/log2 K) attribute values verbatim and guesses elsewhere.
  • Horizon wall: Any query-independent context of fixed width w becomes a vanishing-accuracy strategy on streams whose mission length continues to grow.The horizon wall covers uniform sampling, recency windows, and salience-based top-w selection; query-conditioned retrieval escapes by making selection depend on q.
  • Composition wall: Under TC0 ≠ NC1, fixed-depth attention cannot track growing composition chains in one pass, while R iterative rounds simulate depth Θ(R · dmodel).The paper distinguishes this conditional composition result from the unconditional information-theoretic walls.
  • Round wall: A single non-adaptive retrieval round cannot solve depth-2 pointer chasing, whereas sequential access rounds can extend effective composition depth.The round separation is unconditional, and the paper proves the R=1 case instantiated by its baselines.

C. Access-completeness and super-additivity

The paper formalizes access-completeness through resource-wall bounds and shows that channel composition can provide strict dominance or super-additivity under explicit assumptions.

  • Horizon wall: At N=300, a query-independent window of w=12 frames under B=4k has a 16.0% horizon ceiling.For any fixed window, the ceiling decays toward chance as stream length grows.
  • Resource-wall assumptions: Three walls are unconditional, while the composition wall additionally assumes TC0̸ = NC1 and naturalistic items realizing the S5 encoding.The access-loop case remains supported without the composition-wall assumption.
  • Access-completeness: ASP attains CapB ≥1 −(ϵ0 + δr) on Tret, ≥1 −Nϵu on Ttrk, and ≥1 −dqϵ1 on Tcmp, while single channels remain bounded on at least one family.The single-channel bounds include CapB(Cs) ≤κs and CapB(Cv) ≤1/K.
  • Super-additivity: Strict super-additivity holds on the conjunctive witness, where the hybrid exceeds the single-channel bounds because both channels are required.The uniform mixture instead supports strict dominance, since its additive upper bound can exceed 1.
  • Mixture dominance: At SEW-Bench parameters, κs=0.547 is just below 1/2(1 + 1/K)=0.5625, so the first branch of the mixture result applies.The applicable branch depends on corpus parameters rather than on the method itself.
  • Convergence conditions: The edge-capability convergence bound is narrow only when backbone error margins satisfy ¯ϵ ≪1/N, and shared retrieval-error assumptions are idealized.Because the index uses backbone-generated captions, weaker backbones can degrade retrieval as well as reading.

IV. METHOD: ACCESS-STRUCTURED PERCEPTION (ASP)

ASP is a training-free wrapper that implements a structured state, a lossless episodic index, and an access policy around a frozen multimodal backbone.

  • ASP overview: ASP is a training-free wrapper around a frozen multimodal backbone implementing A=(Cs,Cv,ρ) with four components.The wrapper is accessed through an OpenAI-compatible completion endpoint, but its design does not depend on the endpoint.

A. Keyframe gate

ASP gates incoming frames into keyframes, updates a capped typed scene state, and retrieves losslessly stored frames through a hybrid caption-and-embedding index.

  • Keyframe gate: A novelty gate emits keyframes when 1−cos(e(x),e(xlast))>τg, controlling N while remaining shared across baselines.The shared gate isolates access-structure differences in comparisons.
  • Structured state: The scene state is a typed, capped record updated by st=Mupd(st−1,xt) and deterministic salience eviction enforcing |st|≤Ls.Its fields include rooms, objects, agent trace, and event log.
  • Episodic storage: Each keyframe is stored losslessly off-GPU with its image metadata, one-line caption, embedding, timestamp, and pose.The stored tuple is (xi,ĉi,e(xi),ti,posei).
  • Verbatim index: Retrieval ranks frames by α cos(etxt(q′),e(xi))+(1−α)BM25(q′,ĉi) and returns top-k images within Br tokens.Embedding similarity handles paraphrase, while caption overlap disambiguates instances but misses synonyms.
  • Index evidence: On the evaluated corpus, α=1 gives recall@8=0.005, whereas α=0 gives 0.936, making captions rather than visual similarity the effective index cue.This supports a lossless verbatim channel but not the visual half on natural frames.

D. Router ρ and iterative access loop

ASP routes each query among capped state, verbatim retrieval, and computation budgets, then uses iterative access rounds under a fixed token cap. The protocol evaluates this design on seven open-weight models through an API serving arm and on SEW-Bench’s synthetic, access-structured tasks.

  • Router and allocation: ASP classifies each query as RET, TRK, CMP, or NOW, then water-fills the remaining budget across state, retrieval, and computation wall prices.The router uses a Bρ-token classification call and allocates (Bs, Br, Bc) from the profile table.
  • Iterative access: The access loop runs for at most R rounds, choosing RETRIEVE, STATE, or ANSWER and debiting each executed action against the remaining budget.The loop halts upon ANSWER or exhaustion, returning a forced best guess when the budget runs out.
  • Cost model: ASP costs at most B backbone tokens across ≤ R calls, with analytic FLOPs ≈2Pact · B, KV-cache footprint O(B), and an O(N) flash-resident index.The cost model is hardware-agnostic; the index remains outside accelerator memory.
  • Serving protocol: Only the API serving arm was run, leaving provider routing, uninspectable quantization, and silent revision changes as documented design weaknesses.Model fingerprints detect silent revisions after the fact but do not prevent them.
  • Experimental design: The evaluation spans seven open-weight backbones from 3B to 31B and uses SEW-Bench’s 80 questions over four 300-frame synthetic episodes.The ladder includes Ministral, Qwen, and Gemma models; SEW-Bench separates RET, TRK, and CMP subtests tied to the formal walls.
  • Experimental design: SEW-Bench uses 16 objects per frame with d=32 queryable attributes and Nd=9,600 attributes per episode, testing access structure rather than natural-scene perception.Its RET targets occur in exactly one frame, while filler attributes are re-randomized across frames.

B. Registered headline predictions

The registered predictions test whether access restructuring, scale, and budget reallocation produce the expected gains, while separately probing channel-specific failures and corpus validity. Results support query-conditioned access over query-independent sampling, but the corrected ablations expose important limits to the dual-channel claim.

  • Prediction 1: Embodied scissors gap: ≥78% was the registered ASP target on EW-Bench-RET, versus ≤30% for STATE-ONLY and ≤38% for UNIFORM-w across five backbones.The evaluated RET mapping uses SEW-RET, while the original EW-Bench-RET thresholds were registered predictions.
  • Prediction 2: Capability convergence across scale: ρscale was registered to fall from 0.85 under UNIFORM-w to ≤0.40 under ASP, with the OpenEQA 8B–27B gap shrinking from ≥7 to ≤3 points.The supplied result passage states this prediction across five backbones.
  • Prediction 3: Reallocation beats growth: ASP at B=4k was registered to exceed UNIFORM at B=16k on OpenEQA and EW-Bench mean, testing reallocation against quadrupled context.The OpenEQA clause was not evaluated; the SEW-Bench evaluation tests only the available half of this criterion.
  • Measured headline results: On the flagship, ASP exceeded UNIFORM-w by 33.1 points over 80 items, but it had the higher mean than all six equal-budget baselines on only 4 of 7 backbones.No baseline comparison reached the registered α=0.01 criterion on all six baselines for any backbone.
  • Ablation falsification test: Removing Cv was predicted to collapse RET, removing Cs TRK, and forcing R=1 CMP, but the registered design treats crossed collapse patterns as evidence against channel duality.The ablation table uses the same access loop with one component disabled.
  • Corpus correction: A degenerate CMP slice let BLIND score 87.5–100 before repair; after exact 4–4 balancing, none of four backbones remained above the 25.0% chance line.The correction did not change the falsification verdict, while RETR-ONLY improved on flagship CMP from 37.5 to 71.9 and ASP beat it on no backbone.

G. Falsification criteria (frozen)

The frozen falsification criteria compare query-independent sampling with ASP, scale dependence, ablation signatures, and budget growth without revising the registered tests. Two criteria fire, while the remaining evaluated criteria do not, with one growth comparison weakened by endpoint limits and an unrun benchmark.

  • F1: RET gap: F1 does not fire: UNIFORM-w does not come within 10 points of ASP on SEW-RET for the three evaluated ≥27B backbones.The criterion is evaluated on SEW-RET as the registered EW-Bench-RET mapping.
  • F2: Scale dependence: F2 fires: ρscale falls from −0.58 under UNIFORM-w to +0.00 under ASP, a drop of −0.58.The analysis notes that the criterion is separately recorded as uninformative because the baseline correlation is negative.
  • F3: Ablation pattern: F3 fires: the ablation collapse pattern is crossed, and the verdict is unchanged after the degenerate CMP corpus slice is corrected.The pre-registered Figure 9 boxes were not moved to follow the data.
  • F4: Reallocation versus growth: F4 does not fire on the SEW mean, but its OpenEQA half was not evaluated, so the criterion is weaker than registered.UNIFORM@16k was also endpoint-limited for Ministral 3B, Gemma 3 12B, and Ministral 14B.

H. Analysis: reading the measured numbers

The measured results largely support query-conditioned retrieval and distinguish access failures by subtest, but they do not validate the complete channel-duality architecture. In particular, prompted compression fails on cumulative tracking, while retrieval remains well explained by index recall and per-access reading competence.

  • Scale dependence: −0.58 under UNIFORM-w versus +0.00 under ASP is the observed ρscale comparison, although the negative baseline makes F2’s interpretation limited.At fixed budget, higher tokens-per-frame cost can buy fewer frames, complicating scale interpretation.
  • Shannon and horizon walls: 54.7% is the SEW-RET ceiling for STATE-ONLY, while measured STATE-ONLY scores are 0.0–6.2% across seven backbones.The denominator uses the attribute count Nd; replacing it with N would make the bound vacuous.
  • Access-structure results: 14.8–16.9% is the RET horizon ceiling at realised w ∈[8,15], against measured 3.1–18.8%; verbatim-only tracking is bounded at 12.5% when its budget is below N frames.The two channels fail on different subtests, while ASP beats the better single channel by −20.5 to +8.0 points on the mixture.
  • RET mechanism check: 0.94/0.94 through 1.00/0.91 are representative predicted/measured ASP RET accuracies, with mean absolute residual 0.07.The result matches Proposition 1’s retrieval-plus-reading account, but the unique-key RET construction limits how broadly to interpret the gap.
  • Index competence: 0.75–1.00 retrieval recall@8 implies δr ∈[0.00,0.25], while vision-only recall@8 is 0.005 and captions-only recall@8 is 0.936.The corpus therefore primarily demonstrates caption-based text indexing plus budgeted frame reads, not strong vision-only retrieval.
  • Channel-duality test: ASP beats RETR-ONLY on 0 of 7 backbones; removing Cs raises the flagship mean from 35.4 to 58.0 and CMP from 20.8 to 83.3.The flagship −Cv RET result still holds decisively, from 93.8 to 9.4.
  • Tracking limitation: The prompted accumulator requires per-update error below 1/300 for additive tracking, but the evaluated implementation does not meet that premise at N=300.The Nϵu factor is described as essentially tight because a missed increment is never recovered.
  • Scope and validity: The natural-video benchmarks were not run because their frames require dataset agreements, so the evidence concerns access mechanisms rather than natural-scene perception.The registered clauses remain marked not evaluated rather than passed.

VI. LIMITATIONS AND CONCLUSION

The paper’s access-structure result holds on a synthetic benchmark, but its three-component architecture does not, and several scope and evaluation constraints limit interpretation.

  • Limitations: Caption-only retrieval reaches recall@8 0.936, whereas vision-only retrieval reaches 0.005 on the schematic corpus.The demonstrated retrieval mechanism is therefore a textual index over model-written captions plus a budgeted frame read.
  • Limitations: The measurements use synthetic 2-D schematic renders rather than natural egocentric video, so they establish access-structure behavior under a token budget, not natural-scene perception.Registered natural-video benchmarks were unavailable because their source frames require dataset agreements.
  • Limitations: The study covers 80 questions across four episodes and seven backbones, supporting headline orderings but not one- or two-point differences.Per-subtest cells move in three- to four-point increments.
  • Limitations: The paper makes no hardware claims, studies passive streams only, and reports API serving rather than the registered controlled local serving arm.Provider-side routing, quantisation, and revision factors therefore remain inside the measurement.
  • Limitations: The compressive channel’s prompted error compounds across 300 updates, making its cost exceed its measured return.The scene-state schema is hand-designed, and adversarially dense streams may exceed any practical budget.
  • Future work: Future work prioritises non-compounding compression, measuring ϵu across corpus seeds, evaluating conjunctive items, and rerunning the grid on natural video.The paper specifically identifies pixel-based retrieval transfer as unresolved because vision-only retrieval is at chance on SEW-Bench.
  • Conclusion: Removing the compressive channel raises the flagship mean from 35.4 to 58.0, while ASP never beats the verbatim-only baseline.Two of four frozen falsification criteria fired, localising the failure to the prompted compressive component.

APPENDIX

The appendix formalizes the model, its access assumptions, and the limits of the resulting walls, while separating unconditional claims from conditional ones.

  • Formal setup: The cross-frame state is S = σ(c) ∈ {0,1}^m, with Nd independent attribute values and b = d log2 K salient bits per keyframe.The setup permits arbitrary, possibly randomized and unbounded-compute state maps.
  • Formal setup: The read-only retrieval convention restricts an access to returned contents, excluding aggregate information about unretrieved items.This convention matches ASP’s off-GPU ranking and top-k context reads.
  • Shannon wall: The Shannon bound is information-theoretic and applies regardless of backbone scale or semantic sophistication.Its ceiling becomes vacuous when the state is large enough to hold roughly one bit per queryable attribute.
  • Dependency structure: The figure maps tools to resource walls and then to composite claims through Assumptions 1–2.Only Theorem 3 depends on TC0 ≠ NC1; Theorem 4(i) retains its architectural conclusion unconditionally.
  • Horizon wall: Query-independent selection of at most w frames succeeds on a uniformly queried attribute with probability at most w/N.The per-frame attribute count cancels, distinguishing the horizon wall’s frame scale from the Shannon wall’s attribute scale.
  • Horizon wall: Query-conditioned retrieval breaks the selection independence that drives the horizon wall, but pays a token cost for that access.Salience, novelty, and tree-search selection remain bounded when they are query-independent.

D. Proof of Theorem 3 (composition wall)

The composition-wall proof embeds group-product chains into embodied streams, conditionally rules out single-pass fixed-depth attention, and shows how iterative rounds restore effective depth.

  • Embedding: The S5 embedding maps each generator word to a keyframe stream whose answer is YES exactly when the product equals the identity.Each keyframe depicts one scripted rearrangement event from a fixed two-generator set.
  • Negative direction: Under TC0 ≠ NC1, no fixed-depth polynomial-width constant-precision attention stack can solve growing composition chains in one forward pass.The reduction routes the problem through Barrington’s NC1 characterization and fixed-depth transformer simulation in TC0.
  • Positive direction: R sequential rounds partition the chain into constant-size chunks and maintain a seven-bit S5 partial product.The resulting effective depth is Θ(R·dmodel), with success at least 1 − dqϵ1.
  • Scope: The negative result is conditional on both TC0 ≠ NC1 and naturalistic items realizing the S5 encoding rather than admitting shortcuts.Theorem 4 supplies the unconditional architectural conclusion used by the empirical baselines.
  • Round wall: A single non-adaptive retrieval can depend on the query and state but not on retrieved content, so it cannot localize a depth-2 pointer chain reliably.The proof uses list-Fano bounds on the unretrieved pointer.
  • Correction: The earlier universal read-groundedness claim was false because an m-bit state can tabulate selected pointer-attribute pairs.At N=300, k=8, K=8, tabulation reaches 0.708 at m=600 bits and 1.000 at m=900 bits.
  • Corrected result: The corrected unconditional results retain the no-state and access-localisation claims, while adaptive rounds can resolve the chain with O(1) retrievals per round.The round-based construction succeeds with probability at least 1 − dqϵ1.

F. The verbatim channel cannot track (replacing a deferred reduction)

Without a compressive channel, even adaptive verbatim retrieval cannot track an additive reduction when it reads fewer than all keyframes.

  • Theorem: A verbatim-only agent reading fewer than N keyframes has tracking success exactly 1/K, regardless of adaptive retrieval strategy.The result assumes no cross-frame state and a total read count ktot < N.
  • Proof: Conditioned on the full transcript, every unread frame attribute remains independently uniform.Read-only access preserves the prior distribution of not-yet-revealed contents at every adaptive round.
  • Proof: The unread attributes contribute a uniform group sum, making the target uniformly distributed even after all retrieved contents are known.The agent’s output is transcript-measurable, so its conditional success probability is 1/K pointwise.
  • System relevance: At B = 4k, measured frame costs of 259–432 tokens limit retrieval to 9–15 frames against N = 300.Thus ktot < N is enforced by the operating budget rather than assumed about the agent.

G. Proof of Proposition 1 (access-completeness)

Proposition 1 derives upper bounds for single channels and lower bounds for ASP across RET, TRK, and CMP families. The claimed hybrid advantage is restricted to conjunction tasks; on the mixture, arithmetic conditions determine whether super-additivity is available.

  • Single-channel bounds: Single-channel state capacity is bounded by κs on Tret and Tcmp, while the verbatim channel is bounded by 1/K on Ttrk.The Tcmp bound follows because each hop requires a correct episodic lookup.
  • Conjunction tasks: On conjunction tasks, CapB(Cs) ≤κs and CapB(Cv) ≤1/K because correctness requires both RET and TRK components.The blind-agent baseline is CapB(∅) = 1/K2.
  • Hybrid bounds: ASP achieves CapB(Cs+Cv+ρ) ≥1 −∆, with strict super-additivity when ∆< 1 −κs −1/K + 1/K2.At SEW-Bench parameters, κs = 0.547 and K = 8, so the condition becomes ∆< 0.344.
  • Mixture limitation: For the mixed task distribution, additive super-additivity is available only if κs < 1/K, which fails at the operating point κs = 0.547 and 1/K = 0.125.The paper therefore states Proposition 2 on Tconj and uses a dominance statement for the mixture.
  • Mixture limitation: The hybrid mixture gap is positive whenever ∆T < min{1 −1/K, 2(1 −κs)}, but the operative branch must be checked against measured κs.At K=8, the relevant comparison depends on whether κs is below 0.5625.

I. What the theory does and does not license

The theory licenses wall-specific conclusions only on witness families designed to isolate access modes. SEW-Bench instantiates synthetic access structure, but its shallow chains and schematic renders limit which theorems and perception claims it can support.

  • Theory scope: Table VII identifies which theoretical statements are testable at the chosen parameters, including two vacuous rows.Theorem 3 is uninformative at deepest chain depth dq=3, while Theorem 4(ii) is vacuous because m/N = 20.5 bits per keyframe can name any pointer.
  • Theory scope: Theorems 1 and 2 and Lemma 3 are unconditional, Theorem 4 is proved here only for R=1, and Theorem 3 is conditional on TC0̸ = NC1 and naturalistic S5 items.The bounds apply to witness families rather than directly to mixed benchmarks such as OpenEQA or VSI-Bench.
  • Unrun benchmark: The registered EW-Bench was not built because rendering requires habitat-sim and HM3D/ScanNet licences unavailable through a licence-free path.Its retained design covers 900 items over 60 rendered walkthroughs.
  • SEW-Bench design: SEW-Bench contains four synthetic episodes of 300 frames and 80 questions split into 32 RET, 24 TRK, and 24 CMP items.The corpus is programmatically generated with a fixed seed and released with the code.
  • Corpus construction: The generator re-randomizes object slots and attributes across frames, making compression genuinely lossy rather than allowing a state to memorize fixed room layouts.The load-bearing attribute count is d=32 and Nd=9,600 per episode.
  • SEW-Bench limits: SEW-Bench uses schematic 2-D renders and tests access structure under a token budget, not natural-scene perception, egocentric motion, occlusion, or lighting.Its results should not be read as OpenEQA-style results.

K. EW-Bench: the registered design, not built

EW-Bench was registered as a natural-video diagnostic but was not run because its required scene-rendering and dataset licences were unavailable. The registration preserves its predictions, provenance, accounting rules, and planned access traces without presenting them as results.

  • Access trace: ASP’s registered access trace allocates (Bs, Br, Bc) = (1014, 1419, 1622) from B=4096 after Bρ=41, then uses STATE, RETRIEVE, and STATE operations before answering.The worked example spends 118, 742, and 96 tokens across three rounds.
  • Prediction construction: Registered predictions combine published anchors, a generation adjustment, transported ASP structure gains, and wall ceilings rather than measured outcomes.Each value is computed as anchor + generation + structure, capped by the applicable wall ceiling.
  • Limitations: The paper records three registered limitations: anchor staleness, transport error, and unobserved serving-resource differences.The transport step assumes CCH gains transfer through the redundancy factor alone.
  • Auditability: The protocol logs (z, Bs, Br, Bc, Rused) and realized action sequences so allocations can be audited against the registered design.Figure 12 represents equal budget as the same total partitioned differently.
  • Unrun results: Table X contains three natural-video benchmark columns that were registered but not run, and none of their values is claimed as a result.The same status applies to the unrun registered EW-Bench predictions.

S. Cost accounting

The cost-accounting framework treats tokens as the exactly observable per-decision resource and compares methods at equal total token budgets. FLOPs, cache bytes, and index storage remain unmeasured on API arms, so the registered iso-compute test is outstanding.

  • Token accounting: Every prompt, image, and completion token is debited against B per decision, making Table III an equal-budget comparison by construction.The accounting uses the provider’s usage fields rather than post-hoc normalization.
  • Unmeasured resources: FLOPs, KV-cache bytes, and index storage per decision are unobservable on API arms, so token counts do not proxy for those resources.Two methods can spend the same tokens at different FLOPs when their image/text mixes differ.
  • Unmeasured resources: The registered iso-compute check requires ≤3% FLOPs spread across methods at fixed B, but remains unevaluated without the local-serving arm.Per-image token costs are measured per backbone, making each method’s image/text mix visible in the ledger.
  • Registered follow-ups: The planned S1 sweep varies K ∈{4, 8, 16} to test whether STATE-ONLY RET accuracy remains below the Theorem 1 ceiling.A value above the ceiling would refute applicability of the bound, not the bound itself.
  • Registered follow-ups: The planned S2 sweep varies N ∈{100, 200, 400, 800} at w=16 to test horizon-wall decay for UNIFORM-w against flat ASP accuracy.The registered expectation is monotonic decay toward 1/K for UNIFORM-w while ASP stays within 3 points.
  • Registered follow-ups: The S3 depth-versus-rounds sweep tests whether CMP accuracy follows R≥dq rather than token budget.Cells with R<dq are registered to remain near the Theorem 4 single-round bound regardless of added budget.
Loading 2608.22975v1…