The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale

Wenhui Chen; Jianlin Chen; Ziyao Lin; Chi Man Vong

Source-linked AI summary

The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale

Wenhui Chen, Jianlin Chen, Ziyao Lin, Chi Man Vong

arXiv:2607.14144v2Jul 14, 2026cs.AIcs.ITmath.IT

TL;DR

The paper asks whether representational convergence also yields capability under a fixed inference budget, distinguishing representation from access to information at query time. It defines an access-complete architecture and develops a witness-based separation, while treating field-level convergence toward that architecture as a conjecture.

Problem
Models may align representationally yet diverge in computation, so the paper asks whether representation alone suffices for capability under a fixed inference budget.
Source: §1.2 · p. 3
Method
The paper defines access-complete architectures by a compressive state channel and scalable verbatim-index channel, then combines formal bounds with observational and controlled interventional evidence.
Source: §1.3 · p. 3, §1.4 · p. 4
Results
On the stated witness family and regime, the access-complete class has a capability closure that strictly contains the union of the pure-family closures.
Source: §1.3 · p. 3
Takeaways & Limitations
The paper’s supported claim is that query-time access structure can be decisive on its witness, not that hybrids dominate every task or workload.
Source: §1.3 · p. 4
The field-level convergence trend is an economics-motivated conjecture rather than a measured phenomenon, and the tests are confined to Architectural CCH.
Source: §1.3 · pp. 3–4, §1.4 · p. 4

Abstract

from arXiv · show

The Platonic Representation Hypothesis (PRH) holds that as models scale, representations of heterogeneous networks converge toward a shared model of reality. We propose its sequel and boundary, the Capability Convergence Hypothesis (CCH): under a fixed per-token inference budget, representational convergence does not entail capability convergence. Capability instead converges toward a class, the access-complete hybrid: any architecture holding both a compressive O(1)-state channel and a scalable verbatim-index channel. We anchor it on a witness task, the Newton's-apple problem in an infinite stream, and name three resource walls: a Shannon wall barring any o(Nb)-state architecture, a horizon wall barring any fixed window, and a circuit wall barring fixed-depth attention-only composition (conditional on TC0 != NC1). Under an explicit separability assumption a hybrid crosses all three by paying each wall's price, so capability is strictly super-additive under composition. We separate what we prove from what we conjecture: the access-completeness principle rests on information-theoretic lower bounds and pre-registered experiments, while the field-level convergence trend is an economics-motivated conjecture. We report the first pre-registered small-scale tests under criteria frozen before the data: the predicted scissors gap is measured (exact-retrieval error 0.994 vs. 0.000 once a 64-scalar state gains one global-attention layer), the state-tracking bifurcation lands at the registered boundary, and a conjunction witness shows an irreducibly two-channel solution; one prediction failed with its direction reversed and is reported as such. Representational convergence is given freely by scale; capability convergence must be purchased by access structure.

1 Introduction: Two Convergences

PRH [21] describes scale-driven representational convergence, while this paper argues that fixed-budget capability depends on access structure. It presents the access-completeness principle as testable and the field-level convergence trend as conjectural.

1.1 The first convergence

PRH [21] frames scale as driving representational convergence, with subsequent evidence including cross-modal translation and state-space/linear-attention duality.The paper characterizes representational convergence as a robust empirical regularity.Source: §1.1 · pp. 2–3

1.2 A crack

Usama and Chang [54] report aligned representations diverging at answer commitment, motivating a distinction between representation and fixed-budget computational access.The passage reports pre-decision CKA of 0.875 and post-decision CKA of 0.274.Source: §1.2 · p. 3

1.3 The second convergence

The access-complete architecture pairs an O(1)-update compressive state channel with a scalable verbatim-index channel and strictly separates from both pure families on a witness family.This claim is under a fixed per-token inference budget and the stated regime; the capability closure strictly contains the union of the pure families.Source: §1.3 · p. 3
The paper confines its tested claim to Architectural CCH; Systems CCH, including tool- or external-store-supplied index access, is a discussion-level extension.The convergence conjecture has consistency support rather than demonstration, unlike the access-completeness principle’s falsifiable core.Source: §1.3 · pp. 3–4
The paper’s contribution is a capacity-and-complexity framing of query-time access structure, not a claim that hybrids dominate every task or workload.The composite recall-plus-state-tracking primitive is attributed to Rawat et al. [49], while the paper adds pre-registered measurement and an explicit witness regime.Source: §1.3 · p. 4

1.4 A thought experiment first; then its first measurements

The evidence combines observational measurements, formal lower bounds, and pre-registered small-scale tests with criteria frozen before data and misses reported.The tests address Predictions P1–P3 and include a dissociation experiment; the witness is the Newton’s-apple problem in an infinite stream.Source: §1.4 · p. 4

2 Why Representation Does Not Buy Capability: One Channel, Three Walls

The section compares Transformers and SSMs as history-to-query channels with different fidelity, using a shared channel-capacity framing. It defines the query-time accessible state and introduces the data-processing inequality as a basis for CCH.

2 Why Representation Does Not Buy Capability: One Channel, Three Walls

Transformer attention is framed as high-fidelity indexed access and SSMs as structured, compressed mixing, compared as history-to-query channels.The shared matrix-mixer coordinate builds on Dao and Gu [8] without erasing dynamical differences between the families.Source: §2 · p. 5

2.1 Setup and notation

At query time t, the accessible state Σ_t is what the model may consult to compute its output, and its capacity is written B := H(Σ_t).Examples include an SSM hidden state and a Transformer’s servable KV cache.Source: §2.1 · p. 5

2.2 The state-channel postulate

CCH’s state-channel postulate is introduced as a direct application of the data-processing inequality.Source: §2.2 · p. 5
The section lists a hybrid witness-existence construction conditional on Assumption 1 and separately identifies single-axis separations learnable by gradient descent.Source: §2.2 · p. 6

Appendix A

Appendix A examines how channel capacity and access structure constrain capability under fixed inference resources, and how architectural composition can enable capabilities unavailable to either component alone. It presents a conditional access-complete hybrid for the witness task at a reduced servable-memory coefficient, while treating capability convergence as conjectural and keeping the witness claims within their stated scope.

2.3 Making B precise: finite-precision regularization

For d_state coordinates stored at p bits each, effective state capacity is bounded by B := d_state p; Transformer capacity grows as Θ(L), while SSM capacity is O(1) in L.The paper frames fixed per-token inference budget as a resource profile that includes memory coefficient, read bandwidth, precision, and fixed depth.Source: §2.3 · pp. 6–7, §2.4 · pp. 7–8

2.5 The unified statement: capability lives at the union

The task-family witness combines long-range direction with exact binding retrieval, because the pure Transformer’s window cliff and the compressive SSM’s binding-loss slope are distinct failures.“Infinite stream” denotes a family whose permitted resources grow with instance length, not a stream processed on fixed absolute hardware.Source: §2.5 · p. 8, §2.6 · p. 8

2.7 Wall I: the Shannon wall (unconditional)

If x = Nb/B exceeds 1, the Shannon wall forces at least b(1 − 1/x) bits of target uncertainty for exact retrieval from a finite state.The bound assumes independent, uniformly targeted bindings; compressible or structured bindings fall outside its premise.Source: §2.7 · p. 9

2.8 Wall II: the horizon wall (unconditional)

A fixed attention window with capacity o(Nb) loses out-of-window bindings, and query-dependent top-k selection cannot recover information absent from its state.Escaping the horizon wall requires servable memory to grow linearly, at some coefficient.Source: §2.8 · p. 9

2.9 Wall III: the circuit wall (conditional on TC0̸ = NC1)

Conditioned on TC0 ≠ NC1, fixed-depth attention and ordinary diagonal SSMs cannot length-generalize S5 composition, even when attention has Θ(L) memory.The stated regime fixes depth, uses p = O(log L) precision, models softmax attention within the cited TC0 bound, and excludes chain-of-thought.Source: §2.9 · pp. 9, 11

Appendix A

In Experiment B, pure-state exact-retrieval error rose to the chance ceiling as load increased, while the same 64-scalar state plus one global-attention layer stayed near zero.Appendix A labels channel complementarity partial and alignment–capability dissociation observational; the conjunction result is training-reliability evidence only, with existence-level status open.Source: Appendix A · p. 6, Figure 4 · p. 10

2.10 How the hybrid crosses, under a separability assumption

A hybrid composes an O(1)-update state that tracks the referent with a Θ(ρL) index that retrieves bindings, crossing the walls only under write-time code separability.The construction is an existence result; it does not establish that natural language supplies such anchors or that gradient descent finds the construction.Source: §2.10 · pp. 11–12

3 What Capability Converges To: The Access-Complete Class

The access-complete class is presented through a witness separation under scaling, while industry convergence is conjectural and the census is only consistent with it.The reported industry scope is surface-form level and mixed; embedding-level and incidence claims remain open.Source: Appendix A · p. 6, §3 · p. 12, §3.1 · p. 13

3.1 The core proposition: super-additivity under composition

The compositional join solves a witness task beyond the task closures of either indexed or compressive architectures alone.Under Proposition 3’s conditions, Newton’s apple requires both an Ω(Nb)-indexable channel and an O(1)-composable state; exclusion from the compressive class is unconditional, while exclusion from the indexed class depends on TC0 != NC1. This is strict super-additivity on the witness set, not a claim that hybrids outperform pure architectures on every task.Source: §3.1 · p. 13

3.2 The convergence conjecture (motivating, not demonstrated)

Capability convergence toward access-complete hybrids is an economics-motivated conjecture, not demonstrated by this section.The proposed pattern has multiple implementations, including Mamba/attention [5] [35] and other hybrids [41],, [27], while fixed-window forms [2] [10] lack a global index; shared capability closure remains a hypothesis. The proposed driver is inference economics: Θ(L) KV-cache serving pressure [31] combines with pure SSM limits on exact retrieval [1] [23].Source: §3.2 · pp. 13–14

3.3 The relation to Shannon: the price of the second convergence

The hybrid meets Newton’s apple’s index and composition requirements while reducing the coefficient of its Θ(L) servable-memory channel.It preserves the growth class required for exact retrieval, with per-token servable KV coefficient µ→ρµ for ρ ≪1, and uses an O(1)-update state channel; this is not an asymptotic-order separation or a claim of coefficient optimality. Shannon’s bound still applies equally under strictly equal B: the hybrid pays for the second convergence rather than bypassing the capacity ceiling.Source: §3.3 · p. 14

4 Evidence: Is Capability Converging?

The paper separates consistency evidence for an industry trend from pre-registered tests of whether access-complete architectures have capability advantages. Production architectures cluster by access structure, while controlled small-scale tests support several predicted boundaries and disclose a reversed prediction and scope limits.

4.1 Four regularities in production architectures

Among 61 genuine hybrids, 57% fell in the [1/12, 1/4] global-attention-share band, a 3.4× enrichment over uniform; the concentration is stronger evidence than a nonsignificant narrowing trend.The dispersion trend had p ≈0.06.Source: §4.1 · p. 15, Figure 5 · p. 15
Production comparisons associate access-complete architectures with stronger long-range retrieval, while representation alignment alone does not predict the observed capability gap.The reported pattern includes retrieval collapses after removing attention and recovery when some attention heads are restored.Source: §4.1 · p. 15, Figure 6 · p. 16
Published comparisons show two-sided failures across pure families, while diverse hybrid implementations retain the same two-channel access structure; integration topology still matters within that class.Access-completeness is presented as necessary, not as guaranteeing capability equivalence among implementations.Source: §4.1 · p. 16

5 Discussion: Alternative Explanations

The production regularities remain compatible with an economics-only account, and the authors identify controlled predictions—not the census—as the tests distinguishing capability advantages from lower cost.They also flag CoT/tools and natural-workload incidence as open comparisons or measurements.Source: §4.1 · p. 17, §5 · p. 19

4.2 Pre-registered small-scale tests

Five of six bifurcation predictions held outright: one reflection sufficed for swap extrapolation, while general-stream solving required nh=4 and was not reliably trainable.Only 3/8 seeds solved the general stream at T=256, so the authors claim existence, not reliability.Source: §4.2 · p. 17
Exact-retrieval error is 0.994 vs. 0.000 at N=128 when the same 64-scalar state gains an index, supporting the scissors-gap prediction.The comparison holds the recurrent state fixed; its advantage is purchased by adding index capacity.Source: §4.2 · pp. 17–18
The matched-scale dissociation found retrieval stratified by access structure in all ten cells, while the commensurability prediction failed with its direction reversed.The dissociation was observational; P3′ was a distinct follow-up, not a retroactive rescue.Source: §4.2 · p. 18
At the conjunction witness’s registered cell, the hybrid passed on every seed while no other arm met criterion, but one pure-recurrence seed also solved it.At N=64, the hybrid’s worst seed scored 0.706 versus 0.654 for pure recurrence; ablations indicate both channels were necessary for the learned solution.Source: §4.2 · p. 18, Figure 8 · p. 18

6 Predictions, Limitations, and Implications

The paper states falsifiable predictions, records a post-registration frontier release, and identifies limits on the scope of its claims. Its practical proposal is to report access structure explicitly, while its construction shows how a hybrid can pay for a scalable index alongside a compact state.

6.1 Falsifiable predictions

P1 predicts pure-architecture error will rise with load while access-complete hybrids remain near zero; a low-error o(Nb)-state model without a scalable index would falsify the claim.The prediction is framed in nominal state terms to prevent refitting an effective capacity.Source: §6.1 · p. 20
The registered state-tracking grid supports an expressivity boundary: β ∈(0, 1) lacks reflection, whereas β ∈(0, 2) permits it; P3’s alignment prediction failed with its direction reversed.P3′ is a separate partial re-operationalization, not a retroactive rescue.Source: §6.1 · p. 20
P4 predicts a stable global-attention-share band, while P5 predicts narrowing capability fingerprints for frontier hybrids; the latter remains schematic and untested.The paper says a long-term stable mainstream model outside the band would falsify P4’s economic account.Source: §6.1 · p. 20

6.2 Postscript (added in revision): two frontier releases after registration

Kimi-K3’s released configuration pairs 69 linear-state layers with 24 global-index layers at ρ=0.258, and its addition tightens the 2026 cohort dispersion from 0.067 to 0.062.The second postscript flagship, Qwen3.8-Max, had not disclosed its attention layout, so the authors registered a forward prediction rather than reporting a measurement.Source: §6.2 · p. 21

6.3 Limitations

CCH is scoped to fixed-budget, no-CoT inference; its circuit wall is conditional, and the naturalness of the worst-case witness and separable keys in natural workloads remains uncertain.Surface-form measurements find entity-like keys largely separable, but numeric IDs and code identifiers less so.Source: §6.3 · p. 21
The construction establishes existence, not learnability, and small-model tests do not settle scale-universality or whether frontier models can approximate a state channel through depth or routing.The authors leave frontier-scale reproduction and the emergence of separable codes under particular data mixes open.Source: §6.3 · p. 22

6.4 Implications for measurement and reporting

The authors recommend reporting access class, ρ, and servable-memory coefficient µ, and evaluating fixed-budget conjunction tasks that combine retrieval with state tracking.They also call for controlled same-data comparisons at increasing scale and measurements of witness-type workloads.Source: §6.4 · pp. 22–23

7 Conclusion

The conclusion frames access-complete architectures as a design-pattern class whose capability requires paying for access structure, and presents field-level capability convergence as a conjecture.The paper distinguishes that conjecture from its small-scale pre-registered tests, which it describes as mechanism-level evidence rather than frontier confirmation.Source: §7 · pp. 23–24

B Galois Closure Axioms, Solv Semantics, and Deferred Proofs

This section formalizes which tasks architecture families can solve under a fixed resource regime, then uses that relation to define capability closure. It also sketches an information bound on compressive state.

B.1 The capability closure, via a Galois connection

The solvability relation asks whether a family can reach accuracy ≥2/3 on a task with length generalization under the fixed regime R.The regime fixes the serving profile as a class, not constant query-time index-read compute; indexed reads scale as Θ(ρL).Source: §B.1 · pp. 25–26
The maps F and G relate task sets and architecture sets through universal solvability, and Cap=F∘G is an extensive, monotone, idempotent closure operator.C∞(a) denotes the tasks eventually solved by one family under unbounded scaling within R.Source: §B.1 · p. 26
The deferred proof bounds the information about independent bindings stored in a B-bit state, yielding an average log-loss and error floor for retrieval.The proof uses data processing and conditional entropy; its stated bound is average-case, with a separate worst-case strengthening via Index.Source: §B.1 · p. 26

C KV-Memory Accounting for Figure 11

The section estimates per-token KV memory from the number of global-attention layers and their head configuration. Across the census, memory spans about 160×, largely reflecting global-layer counts and MLA compression.

C KV-Memory Accounting for Figure 11

KV memory is calculated from global-attention layers × key/value heads × head dimension × bytes per scalar.Linear/SSM layers contribute O(1), not Θ(L); bf16 uses 2 bytes per scalar, and hybrid ρ is the global-attention-layer share.Source: C KV-Memory Accounting for Figure 11 · p. 26
≈160× separates Llama-3.1-70B (305 GiB) from Jet-Nemotron-2B (1.9 GiB) in KV memory per 1M tokens.Five headline figures were re-derived from primary configurations to within 2.4% of originating-lab values.Source: Figure 11 · p. 27, C KV-Memory Accounting for Figure 11 · p. 27
Dense GQA caches at every layer, while layer-wise hybrids cache only at global-attention layers; pure SSM/linear models have no Θ(L) memory bar.The figure uses a log scale and colors dense GQA red, MLA amber, and layer-wise hybrids green.Source: Figure 11 · p. 27

D Architecture Census

The census finds a concentration of hybrid architectures in a limited range of global-attention-layer shares, with robustness checks and stated measurement conventions qualifying that pattern. Its architecture-space visualization places access-complete hybrids between pure-compressive and pure-index families.

D Architecture Census

The census covers 140 released architectures from 2023-07 to 2026-05, with ρ read from layer maps and most values verified against primary configurations or papers.There were 147 raw records, with seven duplicates merged; lower-confidence sources are flagged.Source: D Architecture Census · p. 27
61 hybrids have mean ρ = 0.207 and median ρ = 0.167; 57% fall in [1/12, 1/4], a 3.4× enrichment over uniform.The band boundaries were selected after inspecting the data, so the enrichment is descriptive rather than a significance test.Source: D Architecture Census · p. 27
At exactly ρ = 1/4, the narrowing is not statistically significant (p ≈0.06), so cross-sectional concentration is treated as the regularity and narrowing as suggestive.Source: D Architecture Census · p. 27
The ρ convention excludes sliding-window-only models from the hybrid class and places MLA-at-ρ=1 outside access-complete hybrids because it lacks an O(1)-state channel.Parallel-hybrid and hybrid-head models are also not directly comparable to layer-wise ρ; shared-cache designs use a stored-cache convention.Source: D Architecture Census · p. 27
Under the stated frontier rule, 45 models remain after excluding distillation conversions and LFM2 edge models; 62% lie in-band, versus 57% of the full hybrid set.The frontier median ρ is 0.125, with σ = 0.109 versus 0.135 in the full set.Source: D Architecture Census · p. 28
The concentration survives design-menu nulls but weakens under a scale-invariant null, where enrichment is 1.45× with CI [1.1, 1.7].The figure’s operationalizations correlate at ρs = 0.90; the uniform-null enrichment bootstrap CI is [2.7, 4.1], while the conservative cluster-plus-scale-invariant interval is [1.03, 1.87].Source: D Architecture Census · p. 28
The PCA’s PC1 explains 46% of variance and correlates with ρ alone at only 0.66, placing pure-compressive and pure-index families around access-complete hybrids.The embedding uses seven standardized architectural properties; the paper calls it a visualization, not independent evidence, because the features encode access structure.Source: D Architecture Census · p. 29, Figure 12 · p. 29

E Long-Context Capability Data

The appendix cautions that published long-context retrieval measures are heterogeneous and that missing scores are not negative measurements. The accompanying robustness figure assesses the concentration pattern through alternative operationalizations, bootstrapping, and mechanism-family comparisons.

E Long-Context Capability Data

RULER scores, passkey/needle lengths, and “none published” are not strictly commensurable and must not be read as one ranking.Blank cells in no-index classes reflect reporting practice, not negative measurements; they may reflect target use cases or benchmark timing.Source: E Long-Context Capability Data · p. 29
The two index-reliance conventions correlate at 0.90, the uniform-null enrichment CI is [2.7, 4.1], and the in-band fraction holds across mechanism families aside from the LFM2 edge-model outlier.The figure summarizes operationalization agreement, bootstrap enrichment, and within-family fractions.Source: Figure 13 · p. 30

F Demarcation from Prior Work

The paper distinguishes its synthesis from prior work on representation, resource limits, and hybrid design. It then describes pre-registered, controlled mechanism tests and reports the limits of their scale and interpretation.

F Demarcation from Prior Work

CCH uses representational convergence as a premise, but its lower bounds do not depend on PRH holding; prior work found aligned representations alongside divergent computation [54].The paper distinguishes its evidence portfolio of observational results, lower bounds, and pre-registered interventions from PRH’s observational lineage.Source: F Demarcation from Prior Work · p. 30
CCH synthesizes previously scattered limits on retrieval, recall, state tracking, expressivity, and attention circuits rather than claiming a new resource wall [23] [1] [39] [50] [38].The cited work supplies individual components; CCH connects them to an economic regularity and directional hypothesis.Source: F Demarcation from Prior Work · p. 30
Prior hybrid engineering studies efficiency–accuracy trade-offs, whereas CCH asks why capability converges to hybrids and identifies the attractor class [2] [5] [10] [14] [15] [27] [35].Rawat et al. [49] had already proposed composing recall and state tracking; the paper assigns its own experiment the capacity/complexity framing and registered measurements.Source: F Demarcation from Prior Work · pp. 30–31
The access-spectrum visualization places access-complete hybrids between pure-compressive and pure-index corners, with PC1 explaining 46% of variance and correlating with ρ at 0.66.The seven-property PCA is a visualization, not independent evidence, because its features encode access structure.Source: Figure 14 · p. 31
CCH presents its novelty as a unifying framework, access-structure account, and witness, while treating the component ideas and empirical bricks as prior work.Table 5 locates the framework against adjacent lines of work.Source: F Demarcation from Prior Work · pp. 30–31, Table 5 · p. 33

G Experimental Details and Full Verdicts

The controlled tests operate inside R—fixed depth, no chain-of-thought, and no tools—and use small trained models as mechanism-level tests, not frontier demonstrations.The comparisons hold data, optimizer, and budget fixed while varying the sequence-mixing operator.Source: G Experimental Details and Full Verdicts · p. 31
The protocol froze criteria before corresponding data existed, specified falsification conditions, and reports all completed cells without post-hoc exclusions.Thirteen timestamped amendments added controls, follow-up phases, and redesigns before their data; the experimental bundle is publicly archived.Source: G Experimental Details and Full Verdicts · pp. 32–33
In the fixed-base Mamba2InLlama distillation staircase, capability rises with retained attention fraction on every metric, but attention, inherited ability, and trainable parameters co-vary.AlpacaEval-LC at ρ=1/2 is 26.78 versus 22.90 for the dense teacher; the table cautions that this is not a clean intervention on access alone.Source: Table 4 · p. 33

G.1 P2, measured: the state-tracking bifurcation (Experiment C)

The S5 tracking tests locate a configuration-specific capacity boundary: only the nh=4 DeltaProduct tracks the general stream at T=256, though learning is seed-brittle. Nonlinear recurrence can compose the task, while fixed-depth Transformers and diagonal SSMs fail to extrapolate.

G.1 P2, measured: the state-tracking bifurcation (Experiment C)

The task uses running products over 120-way S5 labels, with swap and arbitrary-element streams trained through T≤40 and evaluated through T=512.Models have two layers and dmodel=128; the critical nh=4 general-stream cell was expanded to eight seeds.Source: §G.1 · p. 33
The starred DeltaNet β∈(0,1) stays near chance (≈0.008) even at training length after a 5× budget extension.It cannot fit composed S5 without reflections; adding one reflection (β∈(0,2), nh=1) is sufficient for swap, with seed accuracies 1.0/1.0/0.998 at T=256.Source: §G.1 · p. 34
At T=256, nh=4 tracks the general S5 stream in 3 of 8 seeds, while nh≤2 remains at chance across six seeds.The frozen three-seed cell passed its criterion, but the expanded run showed the result is a capacity boundary, not reliable trainability, for this two-layer configuration.Source: §G.1 · p. 34
The nh transition is a step rather than a staircase: only nh=4 tracks composed S5 at 6.4× training length in the tested configuration.Deeper stacks may realize S5 with smaller per-layer nh, and reliable learning at the tested budget remains open.Source: §G.1 · p. 34
Accuracy 1.000 at every length through T=512 on both streams lets the LSTM pass, unlike the diagonal SSM and both Transformer controls.The 8-layer Transformer reaches 1.0 in-length on swap but falls to 0.03 at T=64; all controls received at least the predicted-pass training budget.Source: §G.1 · p. 34
Both Transformers fail extrapolation, while the LSTM passes both streams through T=512, supporting an axis distinction rather than a general weakness of SSMs.Expressive recurrence buys composition; the LSTM still lacks the index capacity tested in Experiment B.Source: §G.1 · p. 34

G.2 P1, measured: the information floor and the scissors gap (Experiment B)

The retrieval experiment measures a sharp gap between a fixed compressive state and a global index channel, alongside capacity and horizon limits. The results are qualified by effective-capacity estimates, optimization failures, and a design-specific window bound.

G.2 P1, measured: the information floor and the scissors gap (Experiment B)

The setup plants N key–value bindings with 8-bit values in a length-512 stream and compares pure states, windowed access, a global-attention hybrid, and full attention.The pure-state arms use 64, 256, or 1024 scalars; comparisons use identical budgets within each comparison.Source: §G.2 · p. 34
At N=128, pure-state errors are 0.994, 0.977, and 0.854 for m=64, 256, and 1024 scalars, respectively, approaching the 0.996 chance floor.With frozen peff=0.39 bits/scalar, 50%-crossings fall at x=Nb/(m peff)∈[0.82,1.03]; the ±0.1 criterion is met by the two larger states but missed by m=64.Source: §G.2 · p. 34
At N=128, exact-retrieval error is 0.994 for the pure m=64 state and 0.000 for the same state plus one global-attention layer.The registered-budget hybrid mean was 0.060; after the logged budget doubling, mean error was ≤0.0003 at every N, yielding the measured 0.994 vs. 0.000 scissors.Source: §G.2 · p. 34
A window-limited layer does not rescue the same state: its error remains ≥0.91 at every N, while out-of-window sliding-window retrieval stays at chance (0.99+).Within-window retrieval is good at small N (error 0.07 at N=2), but the design cannot fit more than eight bindings in the window.Source: §G.2 · p. 35
With block-duplicated values at c=16, the 50%-crossing shifts to N≈27, near the effective-entropy prediction ≈25 rather than the nominal ≈12.At c=4, crossings track the nominal prediction; the block design also permits a retrieval-free shortcut at N≤c, making this axis mixed evidence.Source: §G.2 · p. 35
The nominal-capacity floor is not triggered at these settings, so the quantitative support for pure-state failures is at effective rather than nominal capacity.At m=64, nominal x≈1 while xeff≈16; the full-attention anomaly and m=64 convergence failures are reported as optimization observations without a capacity claim.Source: §G.2 · p. 35
Semantic-overlap ordering shows no detectable effect at matched weights (0.610 vs. 0.609), and 2-layer full attention reaches only 0.91 at N=128.The authors describe the latter as an optimization failure at this scale; 3 of 6 pure m=64 runs never learned even N=2.Source: §G.2 · p. 35
The collision control removes the hybrid’s scissors advantage: hybrid error lands near the Bayes floor c/(c+1) at every load.Worst-cell deviations from the c=1/3/7 error floors 0.500/0.750/0.875 are 0.023/0.025/0.005; the authors interpret this as evidence that the advantage depends on write-time separability.Source: §G.2 · p. 35

G.3 Representational convergence without capability convergence (Experiment A)

Across matched-scale models, representational alignment grows with scale while retrieval capability stratifies by access structure. A separate pre-registered channel-alignment prediction failed in the opposite direction.

G.3 Representational convergence without capability convergence (Experiment A)

At matched ∼2.8B scale, retrieval accuracy is Pythia 0.345, Mamba 0.05, and RWKV 0.00; the ordering holds in all ten matched-scale cells.The flagship 32.0pp margin over Mamba and 36.5pp over RWKV clears the frozen threshold after the six deciding cells were reevaluated at n=1000.Source: §G.3 · pp. 35–36, Figure 15 · p. 36
Cross-family mutual k-NN alignment rises with scale across all three model-family pairings, replicating the PRH trend.The figure reports that this representational convergence does not predict the capability gap; the measured Spearman correlation is positive, not the preregistered negative direction.Source: §G.3 · pp. 35–36, Figure 15 · p. 36
Across 74 cross-family pairs, alignment correlates positively with the absolute capability gap (Spearman +0.48, p<10^-4), surviving control for model scale.The partial Spearman correlation is +0.30 (p=0.009).Source: §G.3 · p. 36
The measured dissociation supports the section’s claim that representational convergence does not purchase capability convergence; the capability boundary tracks access structure.Source: §G.3 · p. 36

G.4 P3 miniature: a failed prediction, reported as such (Experiment D)

The pre-registered channel-commensurability prediction failed with its direction reversed: solving hybrids align below random initialization.Mutual k-NN means order random 0.576, undertrained 0.527, failing finals 0.376, and solving finals 0.284; the schematic is retained as a record of the original claim.Source: §G.4 · p. 36, Figure 16 · p. 37
Random initialization is not a low-alignment floor: the two random channels echo the same input geometry, while training differentiates channels in solving hybrids.The result is reported as a failed CCH prediction, not a retroactive confirmation.Source: §G.4 · p. 36

G.5 The conjunction witness (Experiment E) and channel complementarity (D-ii)

The conjunction witness separates reliable long-extrapolation performance from expressivity and reports evidence that its learned solution uses both channels. A post-hoc corpus measure finds write-time separability varies by key type and remains a proxy.

G.5 The conjunction witness (Experiment E) and channel complementarity (D-ii)

The witness writes permutation-address-to-8-bit-value bindings, composes a swap stream through S5, then queries the value at the final referent.Arms include a recurrent-front hybrid, pure recurrence, an 8-layer Transformer, a β∈(0,1)-front hybrid, and a window-32 hybrid.Source: §G.5 · p. 37
At T=256, N=16, the hybrid scores 0.966/0.979/0.964, and no other arm meets the 2/3-seed pass criterion.One pure-recurrence seed scores 0.995, so the cell separates training reliability rather than expressivity; N=16, or 128 bits, is inside pure-state capacity.Source: §G.5 · p. 37
At T=256, hybrid-minus-best-other margins are +0.275, +0.422, and +0.532 for N=4, 16, and 64, respectively.At N=64, the hybrid’s worst seed is 0.706 versus the pure-recurrence best seed at 0.654, with both below the pass line.Source: §G.5 · p. 37
Bypassing either channel drops every tested cell to ≤0.03 in every seed, while the answer is linearly decodable from the index mixer at 1.0 versus 0.004 from the state mixer.Evaluation-time lesioning shows the learned solution uses both channels, but the authors caution that lesioning is a distribution shift and does not establish that the task requires both modules.Source: §G.5 · p. 38

G.6 A first corpus measurement of write-time key separability (exploratory)

The exploratory surface-form measure finds separability for entity-like keys (87.5% at document scale), numeric IDs (50.7%), and code identifiers (56.2%).The proxy may miss semantic aliasing and surface near-neighbors that do not collide in learned embeddings; an embedding-level measurement remains open.Source: §G.6 · p. 38

G.7 Deviations, and what these results do not establish

At major-prediction granularity, the scorecard reports partial support for P1 and P2, a failed P3, suggestive-only P4, and untested P5. The experiments establish mechanisms at small scale but do not establish frontier magnitudes.

G.7 Deviations, and what these results do not establish

Protocol departures included w=32 rather than registered w=64, L ∈{896, 1900} rather than 2048, and completing the β ∈(0, 1) arm later; none was chosen after affected results.The first pass used 100 instances per cell, later brought to the registered ≥200 with unchanged conclusions.Source: §G.7 · p. 38
The frozen first-phase scorecard recorded 11 supported, 7 partial, and 1 failed prediction, but the authors caution that split-dependent counts are not summary evidence.They identify per-prediction detail as the canonical statement.Source: §G.7 · p. 38
At major-prediction granularity, P1 and P2 were partial, P3 failed, P4 was suggestive only (p ≈ 0.06), and P5 was untested.No pre-registered falsification condition for P1 or P2 was triggered.Source: §G.7 · pp. 38–39
The tests establish predicted mechanisms at small scale under gradient descent, but not frontier magnitudes; nh=4 trainability remains open (3/8).P3 lacks positive evidence, and the separability assumption is a measurable hinge: removing it puts the hybrid on the exact information-theoretic floor.Source: §G.7 · p. 39
The Pythia–Mamba margin and the ±0.1 collapse band at m=64 missed their frozen thresholds and are claimed only directionally.The window-width deviation still left the receptive field far smaller than the binding–query distance required by the horizon-wall logic.Source: §G.7 · pp. 38–39