PaperlayerView exact revision on arXiv

Source-linked AI summary

Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing

Adam Deng, Shicheng Liu, Yada Zhu

arXiv:2610.22026v1cs.LG

TL;DR

Text is a lossy channel for information already computed by frozen agents, and prior latent channels often frame communication as sender-specific transfer. This paper tests a shared-state approach organized around how the receiver consumes each carrier, finding that KV caches favor augmentation while activations favor replacement. The positive results are bounded by controlled settings and, for activation replacement, oracle alignment.

  • Problem

    Text is a lossy projection of agents’ richer internal state, while prior internal-channel mechanisms can be tuned to known sender-receiver pairs rather than a shared pool.

    Source: §1 · p. 1

  • Method

    The paper evaluates a shared-pool framing with a 2x2 comparison of KV-cache and activation carriers under substitution and augmentation.

    Source: §1.1 · p. 2

  • Results

    Query-side-adapted KV augmentation scores 0.695 versus 0.665 for the native reference, a difference the authors report as not statistically distinguishable from a tie; activation replacement transfers with oracle alignment.

    Source: §1.1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    The supported rule is to augment the KV cache and replace the activation, because consumability depends on where the carrier enters the receiver’s computation.

    Source: §5 · pp. 9–10

    The conclusions are restricted to controlled latent fact transfer: positive results use one model family, so the paper does not establish that the rule generalizes across architectures, scales, or attention shapes.

    Source: §5 · p. 9

Abstract

from arXiv · show

Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that must communicate needs its own transfer bridge, and every agent added to a system needs a new bridge fitted against each previous agent. We reframe this problem as sharing: an agent publishes its internal state once into a pool, and every peer reads that pool through a single mechanism fitted to no sender in particular, so adding an agent adds a publisher and no new bridge. Whereas the carrier of the internal state and the fusion mechanism are usually chosen independently in the literature, our main finding is that these two choices are coupled; the deciding factor is where the carrier enters the receiver's computation. Consumability is a property of the receiving computation rather than of the representation. We find that the key-value (KV) cache must be augmented rather than swapped in: the receiver cross-attends over a peer's cache while keeping its own, and with only a small query-side adapter trained, matches the accuracy it gets from reading the source document. Activations invert this. They are best written straight into the receiver, with no bridge, no adapter and no training, which beats exposing them alongside the receiver's own, though that advantage rests on knowing where in the receiver the state belongs. The rule is augment the KV cache, replace the activation, established under one protocol with a shared control ladder. One adapter, trained once, also serves publishers it never trained on, and a pool of several at once, but only those whose internal state lies close to the receiver's own.

1 Introduction and Related Work

Text communicates only a lossy projection of agents’ internal state, while prior latent-channel work often treats the carrier and fusion mechanism as largely independent. This paper frames sharing around receiver-side consumption and evaluates how carrier and mechanism interact.

1 Introduction and Related Work

  • Text is the shared interface between language-model agents, but it is a lossy projection of richer internal state.Internal carriers considered here are the KV cache and residual-stream activations, both computed during context processing and neither a model output.Source: §1 · p. 1
  • Prior internal-channel work can tune pairwise mechanisms to known senders, whereas the paper’s shared-pool setting requires a mechanism not fitted to a particular sender.The appendix tests holders whose answer-bearing state is unknown and senders not used in training.Source: §1 · p. 1
  • Prior work studies cache exchange, cache reuse, activation steering, and representational correspondence; the paper distinguishes its content-transfer question from behavior portability and notes that similarity need not imply interchangeability.The related-work discussion also contrasts its framing with the survey conclusion that carrier and fusion can be chosen largely independently (Liu, 2026).Source: §1 · pp. 1–2

1.1 This paper

  • The receiver-centric frame separates substitution, which replaces the receiver’s state, from augmentation, which exposes source state alongside it.The paper argues this distinction makes the KV-cache and activation results commensurable.Source: §1.1 · p. 2
  • The paper attributes KV substitution’s failure at high fidelity to the different consumption points: caches enter competitive softmax attention, while activations enter a near-unit-gain pre-softmax path.Its measured readout gain falsifies the proposed conditioning-cliff explanation, according to the paper.Source: §1.1 · p. 2
  • A 2x2 evaluation finds that KV-cache augmentation clears the floor before training, while activation replacement transfers without training but requires oracle position and layer alignment.The positive results are established in the same-base setting; the paper also reports that tested cross-family sharing configurations collapse to the floor.Source: §1.1 · p. 2

2 A receiver-centric account of consumability

The section defines consumability through the receiver’s readout and distinguishes replacing its state from exposing source state alongside it. It argues that the point where each carrier enters computation explains why caches and activations favor opposite mechanisms.

2 A receiver-centric account of consumability

  • The receiver-centric account predicts an asymmetry: augment the KV cache, but replace the activation.The figure states that KV substitution fails even at the annotated fidelity, while activation replacement works training-free.Source: Figure 1 · p. 3, §2.4 · pp. 4–5

2.1 Receivers, carriers, and readout maps

  • A carrier is the transported object; caches derive from activations through producer-specific projections, while the receiver’s readout map converts a cache into its next-token distribution.The receiver’s native cache on the true context defines its native ceiling.Source: §2.1 · p. 3

2.2 Two regimes: substitution and augmentation

  • Substitution places source state in the receiver’s own slots, whereas augmentation preserves the receiver’s state and exposes source state additionally.Substitution succeeds only if its output falls within the receiver’s tolerable set; augmentation is not bounded by that condition.Source: §2.2 · pp. 3–4

2.3 Why the carriers differ: the consumption point

  • The carriers differ because activations enter a near-unit-gain residual path, while caches are consumed by competitive softmax attention that can change token rankings.The paper reports that fidelity does not guarantee consumability and that measured readout gain is near unity, so the cache wall is attributed to the readout rather than amplification or depth.Source: §2.3 · p. 4
  • The analytic bound links per-slot sensitivity to grouped-query attention shape; the reported constant is about 3.46 and is too small to explain the wall by amplification.The constant is described as belonging to attention shape rather than model weights.Source: §2.3 · p. 4

2.4 The Asymmetry Conjecture

  • The asymmetry conjecture predicts that cache substitution remains near the transfer floor while augmentation can approach or exceed the native ceiling, and that activation replacement transfers better than activation augmentation.The proposed falsifiers include successful cache substitution above the floor or activation augmentation succeeding where replacement fails.Source: §2.4 · p. 5

3 Methodology

The methodology tests whether a frozen receiver can answer a fact question using state from a holder that saw the unmasked source passage. It compares carrier-mechanism combinations under explicit anchors, transfer checks, and controls.

3 Methodology

  • The split-context probe pairs a SQuAD passage and question with full and answer-masked versions, giving the full passage to the holder and the masked passage to the frozen receiver.The receiver then receives an object derived from the holder, and evaluation uses validation items while training uses train items.Source: §3 · p. 5
  • Measurements are bounded by a masked-context floor and a full-context native reference, with KV and activation probes reported against separate anchors.The reported floor and ceiling are 0.18 and 0.665 for KV runs, and 0.20 and 0.742 for activation runs.Source: §3 · p. 5
  • The study scores normalized substring match and credits transfer only when shared accuracy clears the floor and conditional accuracy exceeds span-copy rate by a wide margin.Controls include receiver-state no-op, fact-free-prefix position, mismatched-source, and answer-disjoint splits; KV augmentation cross-attends over the holder cache while retaining the receiver cache, whereas activation replacement overwrites the fact-region residual state.Source: §3 · p. 5
  • Oracle substitution interpolates the receiver’s key toward its own k-th nearest key, but no neighbor rank recovers graded tolerance.The table caption identifies Qwen2.5-1.5B, n=200, floor 0.18, and ceiling 0.665.Source: Table 1 · p. 6
  • Before results are read, each run must reproduce native greedy decoding exactly and rebuild the native cache within tolerance.Source: §3 · p. 6

4 Experimental Results and Analysis

The experiments show that whether shared state is usable depends on its carrier and how it enters the receiver: KV caches benefit from augmentation, while activations benefit from replacement.

4.1 The KV cache: substitution is floor-bounded

  • 0.000 transfer: substituting the nearest genuine receiver-produced key yields nothing, while the identity arm reproduces the 0.665 ceiling.The nearest-key tolerance remains ≤0.01 across neighbor ranks k=1–16; the candidate key has cosine 0.826 and attention-logit correlation 0.892 to the true key.Source: §4.1 · p. 6

4.2 The KV cache: augmentation clears the ceiling

  • Query-side-adapted cache augmentation reaches the 0.665 native reference, but the observed 0.695 versus 0.665 difference is not statistically distinguishable from a tie.The receiver keeps its own cache and cross-attends over the holder’s verbatim cache; only query-side parameters adapt.Source: §4.2 · p. 6, Table 2 · p. 7
  • A trained query-side bridge retains positive control-subtracted lift with up to eight holders and transfers to untrained publishers close to the receiver’s own state.Raw attention falls below floor early in the holder-pool sweep; the publisher sweep reports cosine 0.9994 to the receiver’s cache and indistinguishability from control by 0.9933.Source: §4.2 · p. 7

4.3 Activations: replacement transfers, and beats augmenting

  • Training-free activation replacement at the fact region clears floor at every probed depth and reaches the native reference numerically at L=12.The clean operating band is L∈{4, 8, 12}; the mismatch control rises above floor at L≥16, so deeper-layer results are suggestive only.Source: §4.3 · p. 7, Table 3 · p. 8
  • Activation replacement outperforms activation augmentation: the latter peaks at 0.438, and the carrier ordering is activations over values over keys.The value carrier’s partial transfer appears only in the suffix regime; at a single layer, keys, values, and their combination have lift ≤+0.025. The measured activation carrier has roughly 20× more geometric headroom than the key carrier.Source: §4.3 · pp. 7–8

4.4 Summary of the two regimes

  • The comparison is near-mirror: KV caches work under augmentation rather than substitution, while activations work better under replacement than augmentation.The table reports each carrier-mechanism cell relative to its own probe anchors, not as an absolute cross-carrier comparison.Source: Table 4 · p. 9, §4.4 · p. 9

5 Conclusion and Further Work

The conclusion argues that carrier and consumption mechanism are coupled: KV caches should be augmented, while activations should be replaced. It limits the rule to a compatibility regime and outlines alignment and evaluation questions for further work.

5 Conclusion and Further Work

  • The carrier-mechanism choice depends on where state enters the receiver: augment KV caches, but replace activations.The conclusion frames consumability as a property of the receiving computation, not the representation.Source: §5 · p. 9
  • Positive results use one model family, while cross-family transfer collapses and held-out senders work only within a narrow band around the receiver.Only the substitution bound is architecture-general; the rule's positive half is limited to a compatibility regime.Source: §5 · p. 9
  • Cache-side results retain an answer-copying confound; the hardened run shows copying does not explain the result, but covers only one arm.The holder's cache contains the answer verbatim.Source: §5 · p. 9
  • Activation replacement assumes oracle position and layer alignment, while the pool experiment is controlled retrieval rather than a multi-agent workflow.Single-seed runs support reaching, but not exceeding, the native reference; bridge-count accounting does not measure compute, bandwidth, or pool memory.Source: §5 · p. 9
  • Further work targets cross-family alignment, separates alignment method from injection depth, tests attention sharpness independently of depth, and uses multiple seeds.The authors identify activation's wider geometric headroom as making it the more promising substrate.Source: §5 · pp. 9–10

Use of AI tools

  • A language-model assistant helped implement and analyze experiments and edit the manuscript; the authors reviewed the work and take responsibility for its claims and artifacts.The assistant did not generate synthetic data, and the theoretical framing is the authors' own.Source: Use of AI tools · p. 10

A Appendix

The appendix reports conditioning measurements, query-side bridge training details, and a substitution sweep. The measurements show heterogeneous layer gains without a cliff, while substitution methods largely fail to transfer.

Conditioning measurement

  • κoff=0.506 and κon=0.343 (ratio 1.475×); the analytic Jacobian bound predicts measured per-layer gain at Pearson correlation 0.999.Gains are small early and rise to roughly 0.72 by layer 6, but show no cliff.Source: A Appendix · p. 14

Bridge training

  • The rank-16 query-side LoRA bridge trains on 200 items for four epochs, but its 32-item keep-best slice makes checkpoint selection noisy.Weights remain frozen; training minimizes cross-entropy for the gold answer's first token at one reserved decision step.Source: A Appendix · p. 14

Substitution sweep

  • Substitution transfer stays at or below 0.20 for the surveyed mechanisms, while identity reproduces the 0.665 ceiling and best-fidelity codebook selection transfers at 0.00.The codebook's cosine fidelity is 0.880 and attention-logit correlation is 0.928, despite zero transfer.Source: A Appendix · p. 14

A.1 Multi-source sharing: a pool of holders

The pool experiment tests whether a receiver can retrieve task-relevant state from multiple holders using one bridge trained at pool size four. Control-subtracted lift stays positive as the pool grows, though it erodes.

A.1 Multi-source sharing: a pool of holders

  • The receiver cross-attends over caches from N holders, exactly one of which contains the answer-bearing passage; the bridge is trained once at N=4.Evaluation at N=1, 2, and 8 also tests generalization across pool size.Source: §A.1 · p. 14
  • At N=8, the trained query-side bridge reaches 0.395, while raw cross-attention falls below the 0.18 floor by N=4.Raw cross-attention scores 0.555, 0.375, 0.155, and 0.070; the trained bridge scores 0.785, 0.735, 0.550, and 0.395 across N=1, 2, 4, and 8.Source: §A.1 · p. 14, Figure 3 · p. 15
  • +0.640, +0.545, +0.310, and +0.165 lift over the strongest control at N=1, 2, 4, and 8, respectively.The lift remains positive at every pool size but declines as N grows.Source: §A.1 · p. 14, Figure 3 · p. 15

A.2 Held-out senders: how many agents one bridge serves

A bridge trained only on the receiver’s own pools transfers to the nearest held-out sender, but not to more distant senders. The measured sharing range is narrow: one bridge can serve previously unseen publishers only when their cache is close to the receiver’s own.

A.2 Held-out senders: how many agents one bridge serves

  • The sharing test is stronger than varying pools or answer-holder slots: one mechanism must work for publishers it was not fitted to.The bridge is trained on pools built from the receiver’s own weights alone, then evaluated unchanged against held-out senders.Source: §A.2 · p. 15
  • The experiment varies sender identity while holding the receiver’s weights, pool size, and other cell values fixed.The four Qwen2.5-1.5B checkpoints share geometry and tokeniser; the receiver’s RoPE frequency is imposed on each sender.Source: §A.2 · p. 15
  • At cosine 0.9994, the bridge reaches 0.550 against a strongest control of 0.225, a +0.325 lift; at 0.9933 and 0.9810, lift is +0.030 and −0.005.The nearest-sender result matches the trained-on sender, while the two more distant senders are indistinguishable from their controls and below the floor.Source: §A.2 · p. 15, Table 5 · p. 16, Figure 4 · p. 16
  • The sender range is inherited from the receiver’s narrow tolerance for displaced keys; augmentation keeps the receiver’s own cache but does not remove that bound for foreign caches.The experiment measures where the bound falls along the sender axis rather than establishing a new radius.Source: §A.2 · p. 15
  • Within that band, one bridge can replace a bridge per ordered pair; outside it, agents fall back to text or a fitted pair.The supported scope is a population of closely related agents, not a heterogeneous one.Source: §A.2 · p. 16

A.3 Does the wall depend on the architecture?

The key-substitution bound appears across seven receivers from three model families and a range of depths and attention geometries. The sweep tests substitution only, so it does not establish that the paper’s positive augmentation and activation-replacement results generalize across architectures.

A.3 Does the wall depend on the architecture?

  • The experiment re-derives each receiver’s own floor and ceiling before comparing its identity and nearest-key substitution arms.This leaves the substitution bound tested against receiver-specific anchors rather than a shared benchmark anchor.Source: §A.3 · p. 16, Table 6 · p. 17
  • On every receiver, the identity arm recovers its ceiling, while the nearest genuine key transfers at or below 0.008 against floors from 0.000 to 0.350.The result spans seven receivers, six to thirty-six layers, three model families, and multi-head and grouped-query attention up to G=8.Source: §A.3 · pp. 16–17, Table 6 · p. 17
  • Variation in G is incidental, because checkpoints also differ in depth, width, training data, and tokeniser.The clean within-architecture group-size test predicted by the 2G analysis was not run.Source: §A.3 · p. 17
  • The sweep probes oracle key substitution only; it does not test whether KV augmentation or activation replacement generalizes with the bound.Receivers are probed in isolation, so the table also does not establish cross-family sharing.Source: §A.3 · p. 17

A.4 Methodological detail: model geometry and verification gates

The methodology specifies the receiver and cross-family holder geometries, then verifies that manual attention injection reproduces native decoding and cache construction. Trained bridges also undergo an epoch-zero identity check against the untrained arm.

A.4 Methodological detail: model geometry and verification gates

  • The frozen Qwen2.5-1.5B receiver has 28 layers, hidden width 1536, 12 query heads, two key/value heads, and key/value width 256.The GPT-Neo-1.3B holder has 24 layers and key/value width 2048; a frozen per-layer SVD map projects its keys into the receiver’s key space.Source: §A.4 · p. 17
  • A parity gate requires manual attention to reproduce native greedy decoding string-exactly from the same prefix.This verifies the manually implemented injection path against native decoding.Source: §A.4 · p. 17
  • A rebuild gate requires reconstructed prefix keys and values to match the native cache within maximum absolute deviation 10^-4.Trained arms also assert that their zero-initialized bridge matches the untrained arm bit-identically at epoch zero.Source: §A.4 · p. 17

A.5 Mechanism: the wall is at the readout, not the depth

The measured wall is at the readout rather than amplification or depth: activations transfer through the same receiver stack in which caches fail, and tolerate much larger displacement. The positive results remain bounded: cross-family sharing collapses, and trained-router results require controls that expose source-independent answer emission.

A.5 Mechanism: the wall is at the readout, not the depth

  • Readout Jacobian gain is 0.506 off-manifold versus 0.343 on-manifold, a 1.47× ratio that is too small to explain the wall through amplification.Per-layer gains remain O(1), with no single layer exploding.Source: §A.5 · p. 17, Figure 5 · p. 18
  • Activations transfer at 0.72 while caches do not in the identical 28-layer receiver, implicating where each carrier is consumed rather than depth.Both carriers traverse the same stack; only their consumption point differs.Source: §A.5 · p. 17
  • Activations remain usable to cosine 0.90, while genuine nearest-neighbor keys at approximately 0.995 already fail, giving activations about 20× more geometric headroom.Across the matched sweep, activation accuracy stays near 0.70 while the KV carrier is at the floor.Source: §A.5 · pp. 17–18, Figure 5 · p. 18

A.6 Where it breaks: cross-family transfer, and a control that bites

  • Cross-family KV augmentation and activation sharing collapse to their controls or floor; the tested foreign-key identity arm is 0.000.The passage attributes the bottleneck to representational alignment, not receiver adaptability or carrier choice.Source: §A.6 · p. 18, Figure 6 · p. 19
  • The trained activation-router score of 0.677 is substantially contaminated: a mis-source still scores 0.300 against a floor of 0.19.The additive router-only arm gives the defensible controlled result: 0.454 against a mis-source control of 0.069; LoRA-adapted arms fall to 0.115 or less on answer-disjoint evaluation.Source: §A.6 · p. 18, Figure 6 · p. 19
  • A trained sharing mechanism needs both a mis-source arm and an answer-disjoint split to distinguish source use from answer memorisation.The controls test whether the mechanism consumes source content rather than emitting memorised answers.Source: §A.6 · p. 18

Related papers

Loading 2610.22026v1…