PaperlayerView exact revision on arXiv

Source-linked AI summary

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

Simon P. Villani

arXiv:2609.25053v1cs.CLcs.AI

TL;DR

Hybrid models retain recurrent memory beyond attention KV, but whether that persistent state can cross a model boundary usefully is unclear. The paper tests a 4B-to-9B handoff without target prefix replay. Adding GDN state lowers NLL, and a corrected handoff beats continued 4B inference in the tested setting.

  • Problem

    Cross-model KV translation can avoid rereading context in full-attention models, but hybrid models also retain recurrent and convolution state whose usefulness across different model weights is uncertain.

    Source: §1 · p. 1

  • Method

    The study transfers translated attention KV with GDN persistent state between geometry-matched Qwen3.5 Base siblings, then applies component selection and a compact correction without target prefix replay.

    Source: §1 · p. 1

  • Results

    0.747 nats/token: adding the GDN package to translated KV lowers NLL across all 64 test documents, and the corrected handoff significantly beats continued 4B inference.

    Source: §1 · p. 1

  • Takeaways & Limitations

    Direct recurrent and convolution reuse outperforms tested learned GDN maps, consistent with partially shared functional coordinates in the tested sibling pair.

    Source: §1 · p. 1

    Evidence covers one directed, geometry-matched Base-model pair and 4K teacher-forced continuation; free-generation equivalence and a general state interface remain unproven.

    Source: §1 · p. 1, §7 · p. 8

Abstract

from arXiv · show

Can one language model hand its live memory to another without the receiver rereading the context? We demonstrate useful persistent hybrid-state transfer across one architecture-matched Qwen3.5 4B-to-9B sibling pair. To our knowledge, this is the first demonstrated cross-model handoff of persistent recurrent inference state between differently sized hybrid language models without target prefix replay. Translated attention KV alone leaves a large gap; adding the Gated DeltaNet (GDN) persistent-state package lowers teacher-forced negative log-likelihood (NLL), the average next-token log-loss, by 0.747 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]), improving all 64 PG19 documents. Direct recurrent and convolution reuse outperforms the tested learned GDN maps, consistent with partial functional compatibility of persistent-state coordinates. A fresh component factorial selects translated KV with direct recurrent and convolution state. An additional 434,176-parameter correction improves that base on 64 fresh web documents: continuation loss is 0.076 nats/token above native 9B (excess NLL), Jensen-Shannon (JS) divergence is 0.022, and native context recovery (NCR) is 0.918. Corrected 9B significantly beats continued 4B inference while processing zero historical prefix tokens. Evidence covers one direction, one geometry-matched Base-model pair, and 4K teacher-forced continuation; the near-native gate failed, the 16K branch was not run, and free-generation equivalence and a general state interface remain unproven.

1 Introduction

Hybrid models retain recurrent and convolution memory beyond attention KV, raising whether that state can cross a model boundary usefully. The paper tests a 4B-to-9B handoff without historical prefix replay and reports gains from adding GDN state and correcting the selected transfer.

1 Introduction

  • KV-only transfer leaves hybrid persistent memory behind, and matching state shapes alone does not establish that copied state will be useful to the receiver.The source and receiver use different projections and gates.Source: §1 · p. 1
  • Direct recurrent and convolution reuse outperforms tested learned GDN translation in the geometry-matched Qwen3.5 4B-to-9B sibling pair.The result is consistent with partially shared functional coordinates for persistent recurrent memory in this pair.Source: §1 · p. 1
  • 0.747 nats/token: adding the GDN package to translated KV lowers NLL, with all 64 test documents improving.The handoff uses zero historical target-prefix tokens.Source: §1 · p. 1
  • Evidence is limited to one directed, geometry-matched Base-model pair and 4K prefixes; free-generation equivalence and a general state interface remain unproven.The near-native gate failed, and no 16K result was run.Source: §1 · p. 1
  • The selected handoff translates attention KV, reuses GDN recurrent matrices and convolution history, and applies a residual correction before the receiver processes a real bridge token.The receiver processes no historical prefix tokens.Source: Figure 1 · p. 2

A Beyond KV

Figure 2 reports a GDN-package benefit beyond fixed translated KV and a corrected-handoff comparison against continued 4B inference. Both panels use teacher-forced targets after 4K prefixes.

A Beyond KV

  • 0.7473 nats/token: adding the full translated GDN package to fixed translated KV lowers NLL across all 64 paired PG19 documents.The mean per-document reduction of KV-only excess NLL is 79.7%.Source: Figure 2 · p. 2
  • 0.0521 nats/token: on 64 FineWeb-Edu documents, the corrected handoff beats continued 4B inference.The 95% CI for the paired difference is [0.0185, 0.0843].Source: Figure 2 · p. 2
  • The GDN contrast bundles recurrent matrices, convolution history, and initialization semantics, so it does not isolate any one component.Both experiments score 64 teacher-forced targets after a 4K prefix; the panels use different corpora and scales.Source: Figure 2 · p. 2

2 Hybrid State and the Handoff Mechanism

The handoff exploits exactly matching persistent-state geometry across differently sized models, installs target-compatible state without replaying the historical prefix, and selects direct GDN reuse alongside translated KV.

2.1 Matched geometry, different models

  • Persistent-state geometry matches despite different model size and hidden width; exact index correspondence permits direct installation but does not establish functional equivalence.Both models repeat three GDN layers followed by one full-attention layer.Source: §2.1 · pp. 2–3, Table 1 · p. 3

2.2 What is installed and what is replayed

  • After the source reads 4,096 historical tokens, the receiver starts with the real next token and processes 64 scored targets without replaying the prefix.KV-only leaves both GDN components fresh; installing the GDN package also supplies correct initialization semantics.Source: §2.2 · p. 3

2.3 Component translation and direct reuse

  • The component factorial selects TDD: translated KV with directly reused recurrent matrices and convolution history.The tested translation maps K, R, and C; the direct-GDN alternative translates K and copies R and C.Source: §2.3 · p. 3

2.4 An additional behavioral correction

  • The correction trains residual outputs through the receiver while keeping both language models frozen; its 434,176 parameters are additional to the translator and fixed bases.Validation selects rank 4 and identity weight 0.01.Source: §2.4 · p. 3

3 Experimental Design

The experiments use frozen, disjoint data splits and shared continuation schedules to evaluate cross-model handoff. Restoration checks, defined metrics, and document-paired uncertainty procedures establish how the comparisons are measured.

3.1 Data and frozen selection

  • Experiment 1 calibrates on 128 FineWeb-Edu documents, validates on 32, and tests on 64 PG19 books with 4,096-token prefixes and 64 teacher-forced targets.Experiment 2 uses separate fresh documents for its component factorial and primary test; held-out outcomes do not enter fitting or selection.Source: §3.1 · p. 4

3.2 Restoration and execution controls

  • Same-model restoration achieves 100% top-1 agreement and zero maximum absolute logit difference, verifying capture and restoration rather than portability.Each calibration checkpoint uses an independent one-shot prefill because chunked and one-shot prefix processing were unequal; compared conditions share a frozen continuation schedule.Source: §3.2 · p. 4

3.3 Metrics and uncertainty

  • Native context recovery measures the handoff’s recovered continuation benefit from the prefix relative to an empty 9B state; 0.918 denotes 91.8% recovery, not accuracy.NCR is a ratio of aggregate NLLs and can lie outside [0, 1].Source: §3.3 · p. 4
  • Uncertainty uses 10,000 paired document-bootstrap resamples and central 95% percentiles, treating documents—not tokens—as the statistical units.Excess NLL compares handoff NLL with native 9B on the same continuation; JS divergence measures full-distribution differences, while top-1 agreement compares highest-probability tokens.Source: §3.3 · p. 4

4 Results

The results show that GDN state adds useful information beyond translated KV, and that direct recurrent-state reuse with translated KV supports a corrected 9B handoff that beats continued 4B on held-out teacher-forced continuations. The transfer remains bounded to the tested setting, and the stronger near-native gate was not met.

4.1 Does GDN memory contribute beyond KV?

  • Adding translated GDN state lowers NLL by 0.7473 nats/token (95% CI [0.6921, 0.8047]), and all 64 PG19 documents improve.KV-only NLL is 3.115 and falls to 2.368 with the GDN package, removing about 80% of KV-only excess NLL; the package includes recurrent matrices, convolution history, and initialization semantics, so their separate contributions are not isolated.Source: §4.1 · pp. 4–5, Figure 3 · p. 5, Table 3 · p. 5

4.2 Which components need translation?

  • In Experiment 1, direct recurrent and convolution reuse lowers NLL from 2.368 to 2.291, a 0.077-nat improvement with translated KV unchanged.The comparison with learned GDN maps also changes convolution translation and evaluation domain.Source: §4.2 · p. 5

4.3 Can the corrected handoff beat continued 4B?

  • Correction lowers excess NLL from 0.105 to 0.076; corrected excess NLL is 0.076, JS is 0.022, and NCR is 0.918.The result passes the full-state gate but fails all three stronger near-native thresholds.Source: §4.3 · p. 5
  • The 16K branch was not run because the near-native gate failed; the measured gain is teacher-forced fidelity, not free-generation or downstream-task equivalence.The corrected result misses the stated near-native thresholds for excess NLL, NCR, and top-1 agreement.Source: §4.3 · pp. 5–6

4.2 Which components need translation?

  • In the fresh factorial, TDD is selected over TDT under the 0.01-nat tie rule, and its test excess NLL is 0.105 versus 0.134 for TTT.Translating recurrent state raises NLL by 0.0280 on average, while convolution translation has an uncertain small effect.Source: §4.2 · p. 5, Figure 4 · p. 6

4.3 Can the corrected handoff beat continued 4B?

  • Corrected 9B NLL is 1.989 versus 2.042 for continued 4B on held-out continuations, with a paired corrected-minus-source difference of −0.0521 nats/token.The comparison uses 64 fresh FineWeb-Edu documents and 4K prefixes.Source: §4.3 · p. 5, Table 4 · p. 6

4.4 Does the correct source context matter?

  • Correct full state beats shuffled donor state by 0.948 nats/token in Experiment 1 and by 1.188 nats/token in Experiment 2.The shuffle changes the complete donor state, so it does not isolate context specificity to recurrent matrices or identify particular retained facts.Source: §4.4 · p. 6

5 Related Work: Interfaces for Reuse

Prior work addresses reuse through KV translation, hidden-state communication, external memory, activation transfer, and same-model recurrent reuse. The paper distinguishes its contribution as a differently sized hybrid receiver installing another model’s prefix-conditioned persistent state without replaying the target prefix.

5.1 Cross-model KV/cache translation

  • Cross-model KV/cache work transfers or reconstructs attention KV, while the paper’s additional question is whether a hybrid receiver can use persistent recurrent and convolution state.The cited lines distinguish these approaches without asserting that KV transfer itself is new.Source: §5.1 · pp. 6–7

5.2 Hidden-state and latent communication

  • StateBridge communicates through a continuous prefix processed by the receiver, rather than installing state in its built-in persistent inference slots.Its reported multi-agent runs share weights within each run.Source: §5.2 · p. 7

5.3 External memory and reader adaptation

  • Engram-style external memory transfer concerns learned memory and injected reader outputs, whereas this handoff installs prefix-conditioned built-in state without adapting target backbone weights.Prior work includes direct reuse of compatible memory/reader artifacts and target-side reader adaptation.Source: §5.3 · p. 7

5.4 Cross-model activation-state transfer

  • Cross-model activation projection tests activation injection, but its reported effects depend on the model pair and do not install persistent hybrid cache state.The cited work is adjacent evidence about functional compatibility, not evidence that all internal state is portable.Source: §5.4 · p. 7

5.5 Same-model recurrent/hybrid reuse

  • Other systems reuse, compose, recover, or intervene on recurrent state for the same model; this work instead has a different model consume state without historical prefix replay.The cited examples include checkpointing, segment composition, compression, suffix replay, and cache-write interventions.Source: §5.5 · p. 7

5.6 Positioning

  • The qualified novelty claim is cross-model transfer of built-in persistent recurrent inference state between differently sized hybrid LLMs without target prefix replay.The authors explicitly do not claim that cross-model internal-state transfer generally is new.Source: §5.6 · p. 7

6 Discussion

Direct reuse supports useful state transfer in the tested sibling pair, but does not establish identical representations or a general interface. The findings motivate further tests of portability and cross-size state interfaces.

Partial functional state compatibility

  • Direct GDN reuse and wrong-donor degradation are consistent with partially compatible functional coordinates, not representation identity or a formal state ABI.Shared pretraining or a shared training recipe could contribute, but the paper presents that as a hypothesis.Source: §6 · pp. 7–8

Lower reconstruction error does not guarantee better continuation

  • The learned recurrent mapper has lower reconstruction error, 0.5037 versus 0.6732 for direct copying, yet learned GDN translation performs worse behaviorally.The behavioral contrast also changes convolution translation and evaluation domain, preventing an isolated causal conclusion about recurrent-state Euclidean error.Source: §6 · p. 8

Bounded portability beyond KV

  • The result is an existence demonstration under favorable correspondence, not a universal translation rule; geometry mismatch or different update dynamics could break compatibility.Prototype timings do not establish serving speedup.Source: §6 · p. 8

A possible state-interface design direction

  • A stable state ABI across model sizes is a speculative design direction motivated by direct reuse, not something this experiment establishes.The paper proposes testing whether model families can preserve compatible persistent-state geometry and functional coordinates.Source: §6 · p. 8

Decisive next tests

  • The proposed decisive next tests are free generation after handoff and replication on an independent model pair.Mismatched geometry is identified as a subsequent, harder test.Source: §6 · p. 8

7 Limitations

The evidence is limited to a single directed, geometry-matched Base-model transfer and short teacher-forced continuations. Several broader quality, robustness, and deployment questions remain outside the evaluation.

Scope of model transfer

  • The study tests one directed Qwen3.5 Base 4B-to-9B transfer with exactly matched persistent-state geometry; cross-family, Instruct-model, and mismatched-geometry transfer are untested.The translators and correction are specific to this pair.Source: §7 · p. 8

Evaluation setting

  • Evaluation uses 4K prefixes and 64 teacher-forced targets, so free-generation equivalence and downstream-task success do not follow; the 16K branch was not run.The primary contrast also does not isolate recurrent matrices from convolution history and initialization semantics.Source: §7 · p. 8

Uncertainty and deployment

  • Bootstrap uncertainty is conditional on the frozen models, fitted maps, selected correction, and corpus procedure, not training-seed or model-pair variation.Post-verdict ablations reuse test documents, and timing is not an end-to-end production benchmark.Source: §7 · p. 8

8 Conclusion

The conclusion argues that hybrid inference state extends beyond attention KV: in one geometry-matched Qwen3.5 4B-to-9B pair, the receiver uses persistent recurrent state without replaying the historical prefix. The paper presents this as a bounded demonstration that motivates further testing of cross-size compatibility.

8 Conclusion

  • In one geometry-matched Qwen3.5 4B-to-9B pair, the receiver directly uses persistent recurrent inference state without target prefix replay.The authors describe this as the first demonstrated cross-model handoff of persistent recurrent state between differently sized hybrid language models.Source: §8 · p. 8

A Frozen Models and Runtime

  • Both experiments use official Base checkpoints without weight updates, LoRA, quantization, or speculative decoding.The loaded models contain 4,205,751,296 and 8,953,803,264 language-model parameters.Source: A Frozen Models and Runtime · p. 10

B Splits and Deterministic Selection

  • E002 selects 32 fresh documents for factorial validation and 64 further fresh documents for LOCKED evaluation, excluding all E001 split IDs and text hashes.The records retain 4,352 tokens for bridge-plus-255-input state tracking.Source: B Splits and Deterministic Selection · p. 10

C Translator and Correction Details

  • E001 translation hyperparameters are selected by validation tensor error, with separate treatment of KV positions, recurrent checkpoints, and convolution features.The KV position sampler includes endpoints and 128-token boundaries; recurrent fitting uses eight one-shot checkpoint captures per document.Source: C Translator and Correction Details · pp. 10–11
  • The selected E002 correction uses rank 4, identity penalty 0.01, and epoch 3, with 434,176 trainable parameters under a two-million-parameter cap.Selection minimizes validation behavioral KL; the fixed bases are not trainable.Source: C Translator and Correction Details · pp. 10–11, Table 5 · p. 11

D Condition and Metric Semantics

This section specifies the handoff conditions, scoring metrics, and document-level uncertainty procedure. It also records experiment-specific gates and verdicts, including the stronger near-native criterion.

Condition semantics

  • KV-only installs translated attention KV and leaves both GDN components fresh; other conditions install the stated direct or translated components, with shuffled controls using another document’s state.Corrected E002 applies the correction and convolution residual; shuffled donors are rotated among equal-length records without self-donors.Source: D Condition and Metric Semantics · p. 11
  • The receiver first consumes the real next token after the 4K prefix as a bridge; the 64 scored targets follow it, without target historical replay in the handoff path.Native target prefix processing is used for the baseline and offline training supervision.Source: D Condition and Metric Semantics · p. 11

Metric semantics

  • TQR normalizes target-quality recovery by the target’s advantage over the source, while NCR and TQR are ratios of aggregate means rather than accuracy scores.Document-level TQR is undefined for a nonpositive source-minus-native denominator; the aggregate measure uses all documents’ mean NLLs.Source: D Condition and Metric Semantics · p. 11

Uncertainty procedure

  • Confidence intervals use 10,000 paired document-bootstrap resamples, preserving each document’s condition pairing rather than treating tokens as independent replicates.The RGR interval recomputes its ratio of means within each resample.Source: D Condition and Metric Semantics · pp. 11–12

Experiment gates

  • E002’s recorded verdict is FULL_STATE_HANDOFF, while its stronger near-native gate fails and the 16K branch remains unrun.The stronger gate requires excess NLL at most 0.05, NCR at least 0.95, and top-1 agreement at least 0.90.Source: D Condition and Metric Semantics · p. 12

E Full-Precision Continuation Results

The full-precision results section keeps E001 and E002 separate, and the component factorial reports whether pairwise interactions meet its preregistered replication criterion.

E Full-Precision Continuation Results

  • Tables 7–8 retain six-decimal numerical presentation, and results are not pooled across the two corpora.Figure 5 shows Experiment 2 controls on an overview scale and an expanded fidelity scale.Source: E Full-Precision Continuation Results · p. 12

F Complete Component Factorial

  • No pairwise interaction meets the preregistered replication criterion requiring matching signs and a LOCKED estimate of at least 0.02 nats/token in absolute value.The K × R LOCKED interval can exclude zero while still failing the specified materiality criterion.Source: F Complete Component Factorial · p. 12

G Secondary State Diagnostics and Post-Verdict Ablations

Secondary checks track recurrent-state mismatch and correction sensitivity, while the timing and reproduction passages delimit what the reported artifacts establish.

G Secondary State Diagnostics and Post-Verdict Ablations

  • E001 full-translated recurrent normalized error declines from 0.4687 after one new input to 0.4053 after 64; E002 corrected error falls from 0.6239 to 0.3650 through 256 inputs.E002 cosine rises from 0.7778 to 0.9266; these state measurements are not a 16K-context test or free-generation evaluation.Source: G Secondary State Diagnostics and… · p. 12, Figure 6 · p. 14

G.1 Post-verdict component removals

  • Removing recurrent correction has the largest aggregate effect among complete-component removals, while convolution-removal impact has an interval spanning zero.These post-verdict diagnostics use the same LOCKED documents and do not establish the best sparse architecture.Source: §G.1 · p. 12

H Prototype Timing and Its Accounting Boundary

I Evidence Integrity and Paper Reproduction

  • 0.076367 is corrected excess NLL; the base-minus-corrected improvement is 0.029022 nats/token (95% paired document bootstrap CI [0.022828, 0.035413]).Figure 5 markers are means over 64 paired documents, not independent token-level replicates.Source: Figure 5 · p. 13
  • The extraction script reproduces reported aggregates and intervals from frozen records but performs no model evaluation, translator fitting, correction training, or LONG evaluation.The accompanying project includes a claim ledger, evidence map, input hashes, verification outputs, and deterministic figure and table scripts.Source: I Evidence Integrity and Paper… · p. 13

Related papers

Loading 2609.25053v1…