PaperlayerView exact revision on arXiv

Source-linked AI summary

Draft-KV: Learning Useful Latent Communication Between Language Models

Linquan Wu, Shichang Meng, Tianxiang Jiang, Haoyu Yang, Peng Zhong, Fengming Zhu, Xi Peng, Linqi Song, Jacky Keung, Jingyu Zhang

arXiv:2609.34754v1cs.CL

TL;DR

Latent interfaces can improve receiver accuracy without showing that gains depend on correctly paired messages. Draft-KV sends key–value states formed during a sharer’s answer draft and trains a frozen receiver to use them. With a Qwen3-8B sharer, the frozen Qwen2.5-0.5B-Instruct receiver reaches 78.04% on MMLU-Redux, versus 37.45% Receiver-only and 36.40% Deranged.

  • Problem

    Higher receiver accuracy from a latent interface does not establish that the receiver benefits from the message content rather than interface adaptation.

    Source: §1 · pp. 1–2

  • Method

    Draft-KV projects key–value states formed while a frozen sharer drafts an answer into a gated interface read by a frozen receiver.

    Source: §1 · p. 2

  • Results

    With a Qwen3-8B sharer, MMLU-Redux accuracy is 78.04% Matched, 36.40% Deranged, and 37.45% Receiver-only.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across Public settings, Draft-KV beats the receiver alone in all 35 and T2T in 33, while private-context results show collaboration can improve on either model’s local prediction when they hold different evidence.

    Source: §5.2 · p. 7

    The interface is specified through key–value features and mapped receiver layers, though tokenizers, sequence lengths, hidden sizes, and KV-head counts may differ.

    Source: §4 · p. 5

Abstract

from arXiv · show

Latent communication passes internal states between language models instead of decoded text, but higher receiver accuracy does not show that the receiver used the message content. Across five method-dataset pairs, replacing each message with one from an unrelated question changes accuracy by at most 0.60 points, even when communication adds 15.44 points over the receiver alone. Thus the interface can supply the gain while making the sharer dispensable. Draft-KV instead sends the key-value states formed while the sharer drafts an answer to the current question. Linear projections place these states in a side memory read through a gated attention branch, and progressive training moves from message reconstruction to answer supervision under a guard on harm from mismatched messages. Both models remain frozen and the interface trains 1.05M parameters, 348x fewer than C2C. With a Qwen3-8B sharer, a frozen Qwen2.5-0.5B-Instruct receiver reaches 78.04% on MMLU-Redux, versus 37.45% alone and 36.40% with reassigned messages. At fixed interface size, scaling the sharer from 0.6B to 8B raises accuracy from 46.11% to 78.04%; communication also transfers to held-out tasks and can exceed both models when each holds different evidence.

1 INTRODUCTION

Higher receiver accuracy from latent communication does not establish that the receiver used the correctly paired message. Draft-KV sends states formed as the sharer drafts an answer and trains a frozen receiver to use them.

1 INTRODUCTION

  • Existing latent-interface gains can come from interface adaptation, so they do not by themselves show that the receiver benefits from the supplied message content.The proposed contrast holds the interface fixed while changing the message.Source: §1 · p. 1
  • The information-use gap describes content-independent gains that make the sharer dispensable and prevent stronger partners from conveying their advantage.The paper distinguishes system gain from pairing gain: overall improvement versus the value of correctly paired content.Source: §1 · p. 2
  • Draft-KV sends key–value states formed during the sharer’s answer draft through a gated interface while both models remain frozen.Progressive training first establishes cross-model readability, then supervises answers while limiting harm from mismatched messages.Source: §1 · p. 2
  • With a Qwen3-8B sharer, MMLU-Redux accuracy is 78.04% Matched, 36.40% Deranged, and 37.45% Receiver-only.The 1.05M-parameter interface is 348× smaller than C2C and 36× smaller than DLC.Source: §1 · p. 2

2 RELATED WORK

Prior work studies model collaboration through text, continuous representations, cache reuse, heterogeneous cache alignment, and latent reasoning or memory compression. Draft-KV focuses on whether the receiver benefits from the cache formed during the sharer’s answer attempt.

2 RELATED WORK

  • Text-based collaboration methods coordinate models through conversations, specialized roles, debate, response aggregation, or learned communication policies.This paper instead examines what one model must send for another to benefit from its computation.Source: §2 · p. 2
  • Continuous-state approaches communicate embeddings, hidden activations, trajectories, or organized latent factors; Draft-KV sends layerwise key–value states from an answer attempt.These approaches make continuous representations an explicit communication medium.Source: §2 · p. 2
  • Figure 1 reports that message replacement changes accuracy by at most 0.60 pp across five method–dataset pairs, despite system gains reaching 15.44 pp.The figure also reports 93–96% retention of Matched accuracy with mismatched messages and a 29.85 pp sharer-only increase versus a 3.54 pp system increase across C2C sharers.Source: Figure 1 · p. 3
  • Cache-sharing and relay methods reuse, select, organize, or compress KV caches, while Draft-KV asks whether receiver accuracy depends on which cache arrives.LatentMAS combines autoregressive latent-thought generation with shared KV memory for training-free collaboration.Source: §2 · p. 3
  • Heterogeneous cache-alignment methods transmit prompt caches, whereas Draft-KV sends states formed while answering and reads them as separate memory.C2C fuses projected prompt cache into the receiver cache; DLC uses positional disentanglement, structured head transformations, reconstruction, and generation training.Source: §2 · p. 3
  • Latent reasoning and compression methods feed hidden states back as embeddings or learn continuous memories; Draft-KV reconstructs text across two models before downstream use.SoftCoT projects instance-specific soft thoughts into input embeddings rather than layerwise attention memory.Source: §2 · p. 3

3 THE INFORMATION-USE GAP IN LATENT COMMUNICATION

The section tests whether existing latent-interface gains depend on correctly paired messages, and whether stronger sharers transfer more of their capability. Accuracy often remains high after message replacement, while sharer scaling yields little system improvement.

3 THE INFORMATION-USE GAP IN LATENT COMMUNICATION

  • The evaluation compares Matched, Deranged, and Receiver-only conditions on the same questions, with Figure 2 assigning another question’s message to each Deranged row.The two reported contrasts are system gain and pairing gain, in percentage points.Source: §3 · pp. 3–4
  • Across five method–dataset pairs, pairing gain never exceeds 0.60 pp although system gain reaches 15.44 pp.For C2C, MMLU-Redux has 12.29 pp system gain and −0.47 pp pairing gain; OpenBookQA and ARC-Challenge have 13.40 pp and 15.44 pp system gains, respectively.Source: §3 · p. 4
  • Adapter-only C2C retains 96.06% of Matched accuracy on MMLU-Redux, 95.93% on ARC-Challenge, and 93.16% on OpenBookQA.Matched exceeds adapter-only by 1.69, 2.22, and 3.60 pp, respectively.Source: §3 · p. 4
  • Across C2C’s 0.6B–8B sharers, standalone sharer accuracy rises 29.85 pp while system accuracy rises 3.54 pp.With a Qwen3-4B receiver, increasing the sharer from 8B to 14B raises standalone sharer accuracy by 4.48 pp but system accuracy by 0.77 pp.Source: §3 · p. 4
  • The reported results show that existing interfaces can improve receiver accuracy with little benefit from correctly paired content, while stronger sharers do not consistently improve the system.The authors interpret the fixed interface contribution as leaving much of a stronger partner’s advantage on its own side of the interface.Source: §3 · p. 4

4 DRAFT-KV

Draft-KV sends answer-draft key–value states from a frozen sharer to a frozen receiver through a projected, gated memory branch. Progressive training moves from message readability to answer use, with a guard against harm from mismatched messages.

4 DRAFT-KV

  • Draft-KV sends the key–value states formed while the sharer drafts an answer, rather than relying on prompt-position states that do not incorporate the answer.Under causal attention, draft-position states incorporate the question and preceding answer tokens.Source: §4 · p. 4, Figure 3 · p. 5

4.1 DRAFT-DERIVED KEY–VALUE MESSAGES

  • The sharer greedily drafts without seeing the dataset answer, then its frozen model processes the prompt and draft to produce message key–value tensors visible at draft positions.A visibility mask exposes only draft positions.Source: §4.1 · p. 5

4.2 CROSS-MODEL PROJECTION AND GATED INJECTION

  • Bias-free key and value projections reshape sharer states into the receiver’s KV-head layout, which a gated attention branch reads as separate memory.The branch reuses the receiver’s frozen projections, normalization, and head layout; communication parameters include the projection weights and per-head gates.Source: §4.2 · p. 5

4.3 PROGRESSIVE TRAINING

  • Stage 1 trains the receiver to reconstruct a sample-specific message payload from its packet alone, using a fixed decoding instruction rather than the question.The sharer encodes the payload with a per-sample transmission key.Source: §4.3 · p. 6
  • Stage 2 uses the sharer’s draft packet to supervise the dataset reference answer, with reconstruction replay every fifth update.Because the draft may be wrong and the reference is not given to the sharer, the receiver learns to answer from draft states rather than copy them.Source: §4.3 · p. 6
  • Stage 3 trains on full gold answer text and uses a one-sided guard that reduces damage only when a mismatched packet raises gold-answer cost beyond τ.At inference, the packet is fixed and available throughout receiver decoding.Source: §4.3 · p. 6

5 EXPERIMENTS

Across Public and Private evaluations, Draft-KV improves receiver performance and often depends on correctly paired messages. Its gains scale with sharer capability, while message-source and training-stage ablations show the importance of draft-position states and progressive training.

5.1 EXPERIMENTAL SETUP

  • The Public protocol shares questions across five benchmarks; Private splits gold evidence between models, and both models remain frozen during separately trained pair-specific runs.Training uses OpenHermes in Stages 1–2 and ARC training data in Stage 3; evaluation benchmarks are held out from Stage 3 training and checkpoint selection.Source: §5.1 · pp. 6–7

5.3 MESSAGE INTERVENTION ANALYSIS

  • Under adapter-only donor averaging, C2C retains 93.16–116.42% of Matched accuracy; Draft-KV retains 42.28–51.05% and falls below Receiver-only.Restoring the correct donor is worth 26.96 and 29.49 pp for Draft-KV with the 1.7B and 4B sharers.Source: Figure 4 · p. 7, §5.3 · p. 9

5.4 SHARER SCALING

  • With the receiver fixed, Draft-KV’s mean system gain rises 32.6 pp from 0.6B to 8B, versus 30.2 pp for T2T and 1.4 pp for C2C.Draft-KV beats the sharer-only reference in 19 of 20 settings and stays more accurate than T2T at every size.Source: Figure 5 · p. 7, §5.4 · p. 9

5.2 MAIN RESULTS

  • Across 35 Public settings, Draft-KV beats Receiver-only in all, is most accurate in 30, and exceeds T2T in 33, with a 5.85-point average gain.The main-results table covers multiple model pairs, benchmarks, and both evaluation protocols.Source: §5.2 · p. 7, Table 1 · p. 8
  • Across 14 Private settings, Draft-KV beats T2T in all, averages 6.41 points higher, and has positive pairing gain in 13.Where Draft-KV ranks first, it also exceeds both models answering independently.Source: §5.2 · p. 7, Table 1 · p. 8

5.3 MESSAGE INTERVENTION ANALYSIS

  • Draft-KV’s 24.24-pp system gain and 25.65-pp pairing gain contrast with C2C’s 1.31-pp pairing gain and negative system gain in 29 of 35 settings.Draft-KV’s mismatch harm has a 1.19-pp median and stays within 2.5 pp in 33 settings.Source: §5.3 · pp. 8–9

5.5 ABLATION STUDY

  • Replacing draft-position KV with prompt-position KV lowers Matched accuracy by 33.70 and 21.48 pp and nearly eliminates pairing gain; deleting either training stage also lowers both metrics.The ablation uses a Qwen3-1.7B sharer and Qwen2.5-0.5B-Instruct receiver on ARC-C and MMLU-Redux.Source: Table 2 · p. 9, §5.5 · p. 9

6 CONCLUSION

The conclusion treats message pairing, rather than accuracy alone, as evidence that latent communication carries useful content. Draft-KV’s gains grow with sharer capability and disappear under message reassignment.

6 CONCLUSION

  • Draft-KV sends draft-formed key–value states, and its gains grow with sharer capability and disappear under message reassignment, making pairing gain evidence of useful content.Source: §6 · p. 9
  • The authors report assistant help with editing, formatting, prose, and code, while stating that research design and reported numbers were produced without generative AI.They state that they reviewed and reran the code and take responsibility for the final content.Source: AI USE STATEMENT · p. 10
  • The study reports no ethical concerns; experiments use public models and benchmarks and involve no human subjects or sensitive personal data.Source: ETHICS STATEMENT · p. 10

Appendix of Draft-KV

  • Reproducibility materials document the interface, training objectives, interventions, models, data, and evaluation; an anonymized artifact provides implementation, weights, and scripts.Source: REPRODUCIBILITY STATEMENT · p. 10

A DETAILED METHOD AND TRAINING PROCEDURE

Draft-KV extracts the sharer’s draft-conditioned key–value states and makes them available to a frozen receiver through projected, gated attention. Training progresses from message reconstruction to answer supervision, with reconstruction replay and a per-example guard against harm from mismatched messages.

A.1 SHAPES AND MESSAGE EXTRACTION

  • The message and receiver sequences need no token alignment because each model keeps its own tokenizer and their lengths are independent.Dense, layer-specific projections can mix information across sharer KV heads before forming the receiver’s head layout.Source: §A.1 · p. 14
  • The sharer drafts an answer, then its fixed-token forward pass supplies contextualized key–value states at selected layers, with only draft positions transmitted.Pre-rotary keys remove position information from transmitted tensors, while the sharer’s own computation retains native rotary encoding and its causal mask.Source: §A.1 · p. 14

A.2 EXTERNAL ATTENTION AND GATED INJECTION

  • Projected message keys and values enter a receiver-conditioned external-attention branch, whose per-KV-head gates control injection into the frozen receiver.The receiver reuses its frozen query projection and normalization; the packet remains separate from the native cache, which grows during decoding.Source: §A.2 · pp. 14–15

A.3 INTERFACE SIZE

  • Interface parameter count depends on the models’ flattened KV widths and communication-layer count, not their depth, parameter count, or message length.Reshaping, masking, layer assignment, and head mapping add no parameters.Source: §A.3 · p. 15

A.4 STAGE OBJECTIVES AND REPLAY

  • Stage 1 trains the receiver to reconstruct a supplied payload; Stage 2 trains it to answer from the sharer’s draft; Stage 3 adds mismatched-message guarding.Stages 2 and 3 alternate four task updates with one reconstruction update, carrying the communication parameters forward without optimizing either backbone.Source: §A.4 · p. 16
  • Stage 3 alternates four task updates with one reconstruction update; on task updates, its one-sided guard, when active, lowers gold-answer NLL under a mismatched packet and cannot raise it.Source: §A.4 · p. 16

B.1 AUDIT SETTINGS FOR EXISTING LATENT COMMUNICATION METHODS

  • Existing-interface audits compare Matched, Deranged, and Receiver-only conditions to separate correct message pairing from total system improvement.Deranged replaces the message with one from another question while holding the receiver input and interface fixed.Source: §B.1 · p. 16

B.2 MESSAGE INTERVENTIONS IN OUR EXPERIMENTS

  • Training and evaluation derange messages within benchmark-specific donor pools using a bijection with no self-pairings.Matched and Deranged packets share tensor width, with padding hidden by the packet attention mask.Source: §B.2 · p. 17

B.3 ADAPTER-ONLY EVALUATION

Adapter-only evaluation measures accuracy when a trained interface receives averaged outputs from other questions’ messages. Its results show that retained accuracy varies by method and configuration, while a self-donor diagnostic helps distinguish donor content from query convention.

B.3 ADAPTER-ONLY EVALUATION

  • Adapter-only averages interface outputs from five donor messages without training new parameters, retaining a receiver-conditioned component rather than isolating a source-free constant.For Draft-KV, it also fixes external queries to the no-communication trajectory, unlike Matched’s communication-conditioned receiver states.Source: §B.3 · pp. 17, 19
  • C2C exceeds 100% of Matched accuracy in five of 12 settings, while the weakest Draft-KV configurations retain 77.50–86.90%.These are aggregate point estimates and do not imply paired statistical significance.Source: §B.3 · p. 19, Table 4 · p. 19
  • Restoring the correct donor raises accuracy by 26.96 and 29.49 pp for the 1.7B and 4B sharers on the first 512 MMLU-Redux examples.For the 1.7B sharer, self-donor accuracy is 60.55% versus 60.35% for standard Matched, suggesting the adapter-only drop follows donor content on this subset.Source: §B.3 · p. 19, Table 5 · p. 20

C.1 MODELS AND INTERFACE CONFIGURATIONS

  • The four communication layers use pair-specific assignments, and interface sizes depend on model cache widths even when layer assignments match.Against the Qwen2.5-0.5B-Instruct receiver, four Qwen3 sharers and the Llama-3.2-3B sharer share a 1,048,584-parameter interface.Source: §C.1 · p. 20, Table 6 · p. 20

C.2 TRAINING DATA AND STAGE CONFIGURATION

The training setup combines reconstruction and answer-alignment data with staged training and held-out checkpoint selection. Evaluation uses distinct prompt and scoring procedures for public multiple-choice tasks and private evidence-splitting tasks.

C.2 TRAINING DATA AND STAGE CONFIGURATION

  • Stages 1 and 2 draw on the first 500,000 OpenHermes 2.5 conversations, and Stage 3 checkpoint selection uses calibration and gate-validation examples.Eligible snapshots must preserve reconstruction performance, and selection uses calibration-set matched answer NLL; test data is excluded.Source: §C.2 · pp. 20–22
  • Stage 3 lowers matched task NLL from 0.90 to 0.46 while replay stays within 0.51–0.74, below the 0.81 reconstruction bound.The reconstruction gap remains above its required 2.67 nats, leaving every evaluated snapshot eligible; Figure 6 plots these held-out training dynamics.Source: §C.2 · p. 22, Figure 6 · p. 22

C.3 PROMPTS AND DECODING

  • Multiple-choice evaluation selects the option from receiver logits at a fixed prompt position, whereas training supervises answer text.Sharer drafts are decoded greedily, and Stage 3 evaluation uses at most 128 generated tokens for the private protocol.Source: §C.3 · pp. 23–24

C.4 EVALUATION DATA AND SCORING

  • The public protocol includes five benchmark splits, while the private protocol evaluates HotpotQA and 2WikiMultihopQA with exact match.For private tasks, annotated gold evidence is randomly divided between sharer and receiver so each holds evidence absent from the other’s input.Source: §C.4 · p. 24

C.5 COMPUTE AND SOFTWARE

  • Draft-KV inference cost includes sharer drafting and state extraction as well as receiver processing; Text-to-Text is the medium-isolating baseline.Text-to-Text prefills the draft through every receiver layer, while Draft-KV reads message positions at four communication layers.Source: §C.5 · p. 24, Table 8 · p. 25

D.1 CORE ABLATIONS: RESULTS AND PROTOCOL

The core ablations compare message source and removal of training stages under a shared configuration. Their reported mismatch-harm changes differ across ARC-Challenge and MMLU-Redux.

D.1 CORE ABLATIONS: RESULTS AND PROTOCOL

  • The ablations evaluate ARC-Challenge and MMLU-Redux, with test data excluded from Stage 3 training and checkpoint selection.Table 9 reports three accuracies alongside system and pairing gains; one question corresponds to 0.085 pp on ARC-Challenge and 0.018 pp on MMLU-Redux.Source: §D.1 · pp. 25–26, Table 9 · p. 26
  • The Prompt KV control retrains the interface to use prompt-position states, so its comparison with Draft-KV tests message source together with learning to use that source.Receiver, architecture, Stage 1 reconstruction, and replay remain unchanged; prompt states are available at prefill, while draft states require decoding and extraction.Source: §D.1 · p. 26
  • The no-pretraining and no-answer-alignment variants retain reconstruction replay wherever applicable, with Stages 2 and 3 alternating four task updates and one reconstruction update.Both variants use the same initialization choices described for the standard procedure.Source: §D.1 · pp. 25–26
  • Removing either reconstruction pretraining or answer alignment reduces pairing gain but changes mismatch harm in opposite directions across the two benchmarks.On ARC-Challenge, mismatch harm changes from 2.47 to 2.13/1.79 pp; on MMLU-Redux, it changes from 0.61 to 1.63/1.30 pp.Source: §D.1 · p. 26, Table 9 · p. 26

D.2 GUARD ABLATION: MISMATCH HARM VERSUS PAIRING GAIN

The one-sided guard limits harm from mismatched messages while preserving matched gains. Without it, a larger pairing gain can reflect worsening mismatch performance rather than better communication.

D.2 GUARD ABLATION: MISMATCH HARM VERSUS PAIRING GAIN

  • Removing the guard lowers Matched accuracy by 0.59/0.39 pp on ARC-C/MMLU-Redux and increases mismatch harm by 1.71/0.60 pp.The mean hinge roughly doubles, from 0.038 to 0.091 on ARC-C and from 0.052 to 0.104 on MMLU-Redux.Source: §D.2 · p. 27
  • Removing the guard raises P to 47.02/28.44 pp even as G falls, because Deranged deteriorates faster than Matched improves.A larger pairing gap obtained this way reflects mismatch damage, not better communication.Source: §D.2 · p. 27
  • Without the guard, Deranged NLL reaches 3.72 nats, nearly 2.5 nats above Receiver-only; with the guard, it stays at 1.71, within 0.48 nats.Matched NLL is essentially identical in the two runs; the guard prevents mismatched-message NLL from drifting upward during training.Source: §D.2 · p. 27
  • During guarded training, Matched task NLL falls from 0.90 to 0.46 nats while Deranged rises from 1.37 to 1.71, widening their gap from 0.47 to 1.25 nats.The gap is acquired rather than optimized: the guard bounds Deranged NLL instead of rewarding separation.Source: §D.2 · p. 27

E SHARER SCALING ANALYSIS

Sharer scaling increases system gains when receiver, interface capacity, and training conditions are held fixed, although draft length complicates interpretation for the smallest sharer. The receiver-use analysis compares per-question outcomes under shared questions and drafts.

E SHARER SCALING ANALYSIS

  • From Qwen3-0.6B to Qwen3-8B, mean system gain rises from 9.63 to 42.25 pp while average draft length grows 5.2×.All four scaling points use a frozen Qwen2.5-0.5B-Instruct receiver and interfaces with exactly 1,048,584 trainable parameters; one interface is trained per sharer.Source: E SHARER SCALING ANALYSIS · pp. 27, 29
  • Qwen3-4B drafts average 223 tokens versus 262 for Qwen3-1.7B, yet its mean system gain is 8.57 pp higher.Draft length is not monotone in sharer size, so the measured lengths do not order the gains.Source: E SHARER SCALING ANALYSIS · p. 29
  • For the 0.6B sharer, length and capability are hard to separate: draft-length standard deviation exceeds the mean on MMLU-Redux and C-EVAL.The variability reflects drafts that stop early; its weak gains are consistent with both a weaker draft and a shorter one.Source: E SHARER SCALING ANALYSIS · p. 29
  • Other pairings average 233 draft tokens for the Llama-3.2-3B-Instruct sharer and 303 for the reversed Qwen2.5-0.5B-Instruct direction.The Qwen3-8B to Qwen3-4B pairing reuses the Qwen3-8B drafts.Source: E SHARER SCALING ANALYSIS · p. 29

F RECEIVER USE OF THE DRAFT

  • The receiver-use comparison holds questions and cached drafts constant across conditions and examines outcomes question by question.It uses Qwen3-1.7B as sharer and Qwen2.5-0.5B-Instruct as receiver on ARC-Challenge and MMLU-Redux under the Public protocol.Source: F RECEIVER USE OF THE DRAFT · p. 29

F.1 DISAGREEMENT WITH THE SHARER

Per-question outcome partitions show that Draft-KV can repair sharer errors while sometimes losing sharer successes, and that the balance differs by task and communication medium.

F.1 DISAGREEMENT WITH THE SHARER

  • The four-way per-question partition records whether a method is correct and whether the answer extracted from the sharer’s draft is correct.This joint outcome distinguishes repairs from losses that aggregate accuracy alone cannot separate.Source: §F.1 · p. 29
  • Draft-KV recovers 58 of 241 sharer errors on ARC-Challenge (24.1%) while losing 10 of 931 sharer successes (1.1%); on MMLU-Redux it recovers 299 of 2,170 (13.8%) and loses 96 of 3,462 (2.8%).Figure 9 partitions each evaluation set and rescales correction and loss cells by the subsets where each outcome is possible.Source: §F.1 · p. 29, Figure 9 · p. 30
  • With the same draft, Text-to-Text loses 8.98% of sharer-correct MMLU-Redux answers versus Draft-KV’s 2.77%, and 7.41% versus 1.07% on ARC-Challenge.Text-to-Text corrects a similar share of MMLU-Redux sharer errors, 11.0% versus Draft-KV’s 13.8%, but ends below Sharer-only overall.Source: §F.1 · p. 31
  • Replacing the matched message makes 609 of 979 Draft-KV-correct ARC-Challenge answers (62.2%) and 2,265 of 3,665 MMLU-Redux answers (61.8%) incorrect.The reported content dependence therefore appears for most individual questions, not only in aggregate accuracy.Source: Figure 9 · p. 30, §F.1 · p. 31
  • Stage 3 trains on ARC, where corrections are more frequent and losses less frequent; on untrained MMLU-Redux, corrections are less frequent and losses more than twice as likely.The authors describe selective draft use as partly task-specific, while retaining a large net gain on the held-out benchmark.Source: §F.1 · p. 31

G BASELINE REPRODUCTION AND IMPLEMENTATION CHECKS

The appendix documents baseline reproduction and implementation checks, then illustrates how paired messages affect receiver answers in corrected and medium-comparison cases.

G BASELINE REPRODUCTION AND IMPLEMENTATION CHECKS

  • The C2C retraining agrees with published accuracies within 0.44 pp on every benchmark for the three checked pairings.One Table 1 C2C row uses released weights; six other directions were trained by the authors, and no DLC intervention claim is made because its implementation was unavailable.Source: G BASELINE REPRODUCTION AND… · pp. 31–32
  • Implementation checks confirm Receiver-only matches the native receiver exactly, deranged assignments contain no fixed points, and the C2C adapter-only control reconstructs the matched cache within 10−6.These checks verify disabled communication, donor assignment, and the C2C control path.Source: G BASELINE REPRODUCTION AND… · p. 32
  • Thirteen of twenty C2C bootstrap intervals contain zero; the released C2C checkpoint and LatentMAS runs have point estimates only.The intervals cover four reproduced fuser configurations, not the released C2C checkpoint or LatentMAS.Source: G BASELINE REPRODUCTION AND… · p. 32, Table 13 · p. 33

H CASE STUDY

  • The case study groups multiple-choice examples into corrected, medium, mismatch, and lost outcomes, with split-evidence cases requiring composition across the message.The examples use a Qwen3-1.7B sharer and Qwen2.5-0.5B-Instruct receiver on ARC-Challenge and MMLU-Redux, plus HotpotQA and 2WikiMultihopQA split-evidence examples.Source: H CASE STUDY · p. 33

H.1 CORRECTED CASES

  • In the unmarked-samples case, the receiver alone answers correctly, Text-to-Text follows the sharer’s wrong letter, and Draft-KV retains the receiver’s answer.The deranged message pulls the receiver to a third option.Source: §H.1 · p. 34
  • Across seven corrected cases, Draft-KV preserves receiver-held answers in Cases 1–5, while only the paired message produces the answer in Cases 6–7.In Cases 1–5, text transmits the sharer’s error three times out of three; Cases 6–7 illustrate correction as instruction rather than preservation.Source: §H.1 · pp. 33, 36

H.2 MEDIUM CASES

  • In the radio-wave case, the receiver’s confidently wrong answer remains under Text-to-Text but flips to the correct answer under Draft-KV.The deranged message is equally decisive in the wrong direction.Source: §H.2 · p. 37
  • In the octagon-diagonals medium case, Text-to-Text leaves the receiver at 8 while Draft-KV moves it to the correct value, 20.The draft contains a complete derivation, but its text version does not move the receiver off the number of vertices.Source: §H.2 · p. 37

H.3 MISMATCH CASES

The cases illustrate how mismatched messages can redirect answers, how fixed-size messages can lose process-heavy reasoning, and how matched messages can help combine split evidence.

H.3 MISMATCH CASES

  • In all three mismatch cases, deranged predictions match the donor’s gold letter, but three examples cannot distinguish answer identity from content.At scale, averaging interface output over five donors drops accuracy below Receiver-only.Source: §H.3 · pp. 38–39

H.4 LOST CASES

  • Roughly 1% of ARC-Challenge and 3% of MMLU-Redux sharer successes are lost when decisive processes outstrip fixed-size messages.The paper contrasts this limitation with full text drafts, which can carry derivations step by step.Source: §H.4 · p. 39

H.5 SPLIT-EVIDENCE GENERATION CASES

  • Across four split-evidence examples, Draft-KV answers correctly even though the sharer answers wrongly, composing complementary evidence held separately by the models.In one case, each model alone misses the answer, while the matched message lets the receiver combine the two evidence halves.Source: §H.5 · pp. 40–41
Loading 2609.34754v1…