PaperlayerView exact revision on arXiv

Source-linked AI summary

Receiver-Conditioned Latent Communication gives 94% CacheBack

Maximillian Rossi, Prajwal Raghunath, Haoqing Xuan, Yusen Zhang, Eugene Wu

arXiv:2609.32046v1cs.AI

TL;DR

Text messages are costly to generate, while full-cache communication can re-expand context at the receiver. The paper conditions sender-state selection on the receiver’s information need, and reports accuracy and latency improvements over same-size text communication across benchmarks.

  • Problem

    Full-cache communication re-expands the sender contexts at the receiver, while text communication requires costly token generation.

    Source: §1 · p. 2, §2.1 · p. 3

  • Method

    CacheBack uses a receiver query to select sender positions for communication, applying a simple, training-free attention-based selector.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    CacheBack improves strict accuracy by 7.3–20.7 percentage points and reduces task-completion latency by 1.3×–8.0× on FanOutQA, with accuracy and speed gains also reported on LongBench v2.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Communication protocols should determine not only how messages are represented, but also what information they contain, based on receiver needs.

    Source: §6 · p. 9

    The experiments do not determine how receiver queries should be written, when to ask again, or how much state to request.

    Source: §5 · p. 9

Abstract

from arXiv · show

Multi-agent systems distribute large contexts across agents that communicate to solve a task. Text messages are compact but require decoding and may omit evidence the receiving agent needs. Recent latent communication instead transfers KV caches. This avoids text generation and can improve accuracy and latency. However, a full KV cache grows linearly with both the context an individual agent processes, and the number of agents that coordinate together. This raises memory and context costs, often far exceeding available GPU resources and context window sizes. Our key observation is that agents need only send what the receiving agent requires for its local task -- which we call receiver-conditioned communication. The receiver agent passes the sender a small description of its information needs, which serves to filter and compress the sender agent's KV cache. CacheBack is a simple, robust, training-free instance of receiver conditioning based on the sender's attention weights. On FanOutQA, CacheBack with Qwen 3 removes 75% of the state the agent would otherwise receive, improving accuracy by 14.7 percentage points and reducing median task-completion latency by 3.2x relative to text communication. We show comparable improvements across model families that span dense Transformers, Mamba-attention hybrids, and sliding-window attention.

1 INTRODUCTION

Text messages are costly to generate, while full-cache latent communication can re-expand context at the receiver. The paper introduces receiver-conditioned selection and reports better accuracy and latency than same-size text communication.

1 INTRODUCTION

  • FanOutQA can require one agent to identify five large states and other agents to find each state’s highest elevation in separate articles.The example illustrates a parallel information-gathering task whose results are combined by a receiver.Source: §1 · p. 2
  • Generating three Qwen 3 8B text messages takes a median 73.2 seconds in the FanOutQA evaluation, and discrete tokens are less expressive than continuous hidden representations.The paper identifies text generation cost and limited expressiveness as motivations for latent communication.Source: §1 · p. 2
  • LatentMAS sends the sender’s full KV cache after generating latent steps, and prior work reports accuracy and latency gains over text communication in sequential and hierarchical topologies.The introduction distinguishes this latent communication approach from selecting only information the receiver needs.Source: §1 · p. 2
  • Full-cache transfer can make the receiver’s context, transferred data, memory pressure, and attention grow with the number of agents and article sizes, leaving insufficient context for generation in the reported FanOutQA settings.This can remove the context-management benefit of partitioning evidence across agents.Source: §1 · p. 2
  • CacheBack uses a receiver’s information need to select sender positions, providing a simple, robust, training-free instance of receiver-conditioned communication.The broader contribution is to make message content depend on the receiving agent’s task needs.Source: §1 · p. 2
  • CacheBack improves strict accuracy by 7.3–20.7 percentage points and reduces task-completion latency by 1.3×–8.0× on FanOutQA; on LongBench v2, it improves accuracy by 9.3–14.7 percentage points with 2.5×–3.7× speedups.Across four model families, the accuracy–latency Pareto frontier shifts toward faster, more accurate task completions.Source: §1 · p. 2

2 AGENT COMMUNICATION

Agent communication can transmit text or latent state, including KV entries for positions in a sender’s sequence. Sending full states re-expands context at the receiver, motivating limits on which positions are selected and sent.

2 AGENT COMMUNICATION

  • Each sender-sequence position has a KV entry per layer; input positions have token IDs, while latent-step positions exist only as continuous vectors, and selected positions are included in the message.Prior latent methods may send all sequence positions while varying the representation or layers transmitted.Source: §2 · p. 3

2.1 COMMUNICATION COSTS

  • Full-cache communication sends every sender position at each communicated layer, and the receiver loads all those positions in addition to its own prompt, causing context re-expansion.Figure 2 contrasts full-cache accumulation with receiver-conditioned messages retaining selected source and latent positions.Source: §2.1 · p. 3, Figure 2 · p. 3

2.2 THE NEED FOR RECEIVER-CONDITIONED SELECTION

  • The receiver’s prompt, selected positions from each agent, and reserved output must fit within the receiver’s context limit.A position budget bounds how much an agent sends, but does not decide which positions to send or how to represent them.Source: §2.2 · pp. 3–4

3 RECEIVER-CONDITIONED COMMUNICATION

Receiver-conditioned communication lets the receiver specify needed information, which the sender uses to select and represent a bounded subset of its state. CacheBack adjusts mean query attention to preserve positions supported by only part of the query, and diagnostics show stronger evidence selection with model scale.

3 RECEIVER-CONDITIONED COMMUNICATION

  • Receiver-conditioned communication has the receiver state its information need, which the sender uses to select positions and represent them for transmission.The message can use token IDs and continuous vectors rather than KV contents.Source: §3 · p. 4

3.1.1 NAIVE SELECTOR

  • The sender appends the receiver’s query after forming its state and scores existing positions using query-token attention.Mean query attention averages across query tokens and heads, sums eligible layers, and pools nearby positions.Source: §3.1.1 · p. 4
  • Mean query attention can rank a weakly supported position above evidence that receives strong attention from only a few query tokens.Averaging dilutes support specific to one part of the query.Source: §3.1.1 · p. 4, Figure 3 · p. 4

3.1.2 CACHEBACK

  • CacheBack reweights mean query-attention scores to give extra weight to positions whose attention is concentrated in particular query tokens.The correction is intentionally simple and training-free, without introducing a learned module.Source: §3.1.2 · p. 5

3.2 SELECTION BEHAVIOR

  • At 8× compression, CacheBack retains 93% of evidence versus 68% for the strongest receiver-independent selector.These diagnostics motivate using CacheBack selection in the end-to-end experiments.Source: Figure 4 · p. 5, §3.2 · p. 5
  • With scale, median gold-token percentile rises from 0.61 for 1.7B to 0.76 for 32B, and CacheBack’s largest gains over mean query attention shift to higher compression ratios.CacheBack evidence recall at 8× compression also rises from 68% for 1.7B to 93% for 32B.Source: Figure 4 · p. 5, §3.2 · p. 6

4 END-TO-END EVALUATION

At 16× compression, CacheBack removes 94% of sender positions and achieves lower latency and higher accuracy than same-size text across tested model families and communication topologies. The evaluation covers parallel fan-in on FanOutQA and sequential communication on LongBench v2.

4 END-TO-END EVALUATION

  • 94% of sender positions are removed at 16× compression, with lower latency and higher accuracy than same-size text across tested model families and communication topologies.The comparison uses the same receiver query for text and latent communication.Source: §4 · p. 6

4.1 EXPERIMENTAL SETUP

  • The evaluation uses 50-task batches, testing three-sender fan-in on FanOutQA and a sequential chain on LongBench v2 across four model families.Receivers are the largest model in each family; LongBench v2 uses Qwen and Nemotron.Source: §4.2 · p. 6, §4.1 · p. 6

4.2 FANOUTQA: PARALLEL COMMUNICATION

  • 7.3–20.7 percentage points higher strict accuracy and 1.3×–8.0× lower median TTEOA than same-size text are reported on FanOutQA.CacheBack 32× beats all tested text settings on both measures for Qwen and Ministral.Source: Table 1 · p. 7, §4.2 · p. 7
  • On FanOutQA, CacheBack's larger selected messages raise transfer and receiver prefill, while text spends more time generating messages and queueing under concurrent workload.At selected operating points, CacheBack message recall is 91.5%–95.8%, versus 84.8%–91.1% for same-size text.Source: §4.2 · p. 7, Figure 6 · p. 7
  • At 16× on FanOutQA, an unrelated receiver query reduces message recall from 91% to 65% and strict accuracy from 46% to 23%.Sender computation, message budget, and final task question are held unchanged.Source: Figure 7 · p. 8, §4.2 · p. 8

4.3 LONGBENCH V2: SEQUENTIAL COMMUNICATION

  • Further compression beyond selected FanOutQA operating points offers little latency benefit and can sharply reduce accuracy.For LongBench v2, Qwen compression from 16× to 128× reduces median TTEOA from 165 to 143 seconds while accuracy falls from 44.7% to 32.7%.Source: §4.2 · p. 8, §4.3 · p. 9
  • In the LongBench v2 chain, each agent reranks inherited and new positions; relative budgets grow by H/r per agent, while a fixed budget caps outgoing message size.The final panel uses the Easy subset after many initial-run text messages hit generation limits.Source: §4.3 · p. 8
  • Relative CacheBack improves LongBench v2 accuracy by 14.7 points for Qwen and 9.3 for Nemotron, with 3.7× and 2.5× lower median TTEOA.Fixed CacheBack improves accuracy by 7.3 and 6.0 points, with 2.5× and 1.9× lower median TTEOA.Source: Table 2 · p. 8, §4.3 · pp. 8–9

5 DISCUSSION AND LIMITATIONS

Receiver-conditioned latent communication avoids sender decoding, while its transfer and receiver-side costs still depend on message size. The discussion identifies fixed-query and fixed-selector choices that leave important design questions open.

5 DISCUSSION AND LIMITATIONS

  • Selecting more computed state increases transfer volume, receiver memory use, and prefill work, but requires no additional sender decoding.This lets the sender send more state without decoding a longer text message.Source: §5 · p. 9
  • The time savings are greatest when decoding long or repeated messages delays dependent work or keeps sender GPUs busy; they narrow when transfer or prefill costs exceed avoided decoding.Repository exploration is one example of a task with detailed handoffs; editing may involve mostly file changes, tests, or tools before a short message.Source: §5 · p. 9
  • The same completed sender state can serve different receiver queries without rerunning the sender task, but the number of queries it can support before selection and transfer become limiting is unevaluated.Each new query still requires another selection.Source: §5 · p. 9
  • The evaluation uses one task-question query after sender computation and fixes the message budget, leaving query-writing, follow-up timing, and changing requests untested.Queries and message budgets could change as the receiver’s task changes.Source: §5 · p. 9
  • CacheBack uses a fixed attention-based scoring rule, with hybrid models exposing only global-attention layers; the experiments do not establish the best selector.Learning the scoring rule or using more model state may improve selection further.Source: §5 · p. 9

6 CONCLUSION

Latent communication can avoid text generation, but full-cache transfer still sends every sender position and can restore context partitioned across agents. CacheBack illustrates that conditioning content on receiver needs shifts the quality–latency frontier across model families and communication topologies.

6 CONCLUSION

  • Full-cache latent methods send every sender-sequence position, allowing the receiver to accumulate context partitioned across agents.The conclusion distinguishes choosing a message representation from choosing what information it contains.Source: §6 · p. 9
  • CacheBack shifts the quality–latency frontier across model families and communication topologies with a simple, training-free method.Source: §6 · p. 9
  • The conclusion argues that communication protocols should decide both how a message is represented and what it contains, with content depending on the receiver’s needs.Source: §6 · p. 9

ETHICS STATEMENT

The work uses public benchmarks and reports no human subjects or crowd workers. The authors identify risks from lossy handoffs and latent messages derived from private context, while reporting no work-specific fairness or bias concerns beyond those in the underlying models and benchmarks.

ETHICS STATEMENT

  • The experiments use FanOutQA, built from English Wikipedia, and LongBench v2, whose documents are public-source and questions were written by benchmark annotators.The authors report no human subjects or crowd workers.Source: ETHICS STATEMENT · p. 10
  • The authors identify a risk that compressed handoffs discard needed detail and lead to downstream errors.Source: ETHICS STATEMENT · p. 10
  • Latent messages derived from a worker’s private context offer no confidentiality guarantee, although they are not human-readable.The paper follows LatentMAS’s latent protocol, whose debug mode generates parallel text probes for inspection (Zou et al., 2026).Source: ETHICS STATEMENT · p. 10
  • The authors report no fairness or bias concerns specific to this work beyond those already present in its underlying models and benchmarks.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper provides evaluation code and configurations, prepared FanOutQA inputs, and scripts to reconstruct LongBench v2 inputs. It cautions that rerun outputs and latency may vary with decoding, serving, and hardware conditions.

REPRODUCIBILITY STATEMENT

  • The repository provides evaluation code and configurations for FanOutQA and LongBench v2, prepared FanOutQA inputs, and scripts to reconstruct LongBench v2 inputs.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Reruns may not match reported numbers exactly because sampled decoding and batch-dependent vLLM numerics can change outputs even with fixed seeds.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Latency depends on hardware, interconnect, vLLM version, and scheduling; Appendix E reports receiver sampling variation across three draws.Appendices F and G provide settings for reported runs and the eight-GPU serving configuration used for latency measurements.Source: REPRODUCIBILITY STATEMENT · p. 10

AI USE STATEMENT

The appendix formalizes receiver-conditioned communication and specifies CacheBack’s scoring, position-selection, and encoding choices. It also documents AI assistance and contrasts communication settings.

AI USE STATEMENT

  • The research idea, method, and experimental design were the authors’ own, while AI assistants supported implementation, experiments, figures and tables, related-work surveying, and manuscript editing.The authors report reviewing all code and pipelines and taking responsibility for the final work.Source: AI USE STATEMENT · p. 10

A FORMAL MODEL OF RECEIVER-CONDITIONED COMMUNICATION

  • The formal model separates receiver-query-conditioned content selection from message representation and evaluates designs by task loss and measured cost under position budgets.The receiver decodes the message to produce its output; varying the acceptable task-loss bound traces a quality–cost frontier.Source: A FORMAL MODEL OF RECEIVER-CONDITIONED… · p. 15

B COMPARISON WITH OBF

  • Table 3 contrasts information access and context growth for shared-question reasoning and communication of private evidence.Source: Table 3 · p. 16

C.1 GENERAL SCORING RULE

  • CacheBack adapts SnapKV scoring by using receiver-query attention to rank sender positions, with a correction comparing attention before and after averaging query tokens.For models with local and global attention, only global-attention layers contribute to scoring; Pool7 takes a local maximum over sender positions.Source: §C.1 · p. 16

C.2 BUDGETS AND POSITION SELECTION

  • Position selection retains the first position and 40 current latent positions, then adds ranked spans until the message budget is full.Spans have width 16; when the next span exceeds the budget, selection adds a shorter interval around its highest-scoring position.Source: §C.2 · p. 17

C.3 MESSAGE ENCODING

  • Experiments transmit selected positions as continuous rows and reconstruct the receiver cache with a fresh prefill, while token-ID encoding can reduce transmitted bytes.Direct KV transfer would require handling positional mismatch, which the authors do not evaluate; all reported experiments use continuous rows.Source: §C.3 · p. 17, Figure 8 · p. 18

D.1 EVIDENCE RECALL

The appendix evaluates whether selected sender positions retain evidence, tests CacheBack’s correction parameters, and reports how much FanOutQA evidence reaches the receiver. CacheBack generally retains the most evidence, while recall declines at higher compression.

D.1 EVIDENCE RECALL

  • 26 of 28 Qwen 3 model-size–ratio settings have the highest evidence recall with CacheBack; H2O is higher at 32× and 64× for Qwen 3 1.7B.The comparison uses 13 development questions and fixed selector budgets; CacheBack is used for end-to-end evaluation.Source: §D.1 · p. 18, Figure 9 · p. 18

D.2 PARAMETER SENSITIVITY

  • CacheBack uses p = α = 2, selected on 13 FanOutQA development questions and held fixed across model sizes, compression ratios, and end-to-end evaluation.Source: §D.2 · p. 19

E.1 FANOUTQA MESSAGE RECALL

  • CacheBack’s matching rule has a 96.4% recall ceiling because 18 reference groups are absent from sender contexts; eight more occur in the question framing.The remaining 469 groups can be located in sender context.Source: §E.1 · p. 20, Table 5 · p. 20
  • At 2×, CacheBack messages contain 474 to 477 of 495 reference groups; recall stays above 91% through 16× in every family, then declines at higher compression.The reported ranges are 86.1–91.5% at 32×, 74.7–82.0% at 64×, and 50.3–62.8% at 128×.Source: §E.1 · p. 20, Table 5 · p. 20
  • At CacheBack’s Table 1 operating points, messages contain 91.5–95.8% of reference groups versus 84.8–91.1% for same-size text, though one missing group can make a question strictly incorrect.Message recall pools over reference groups, whereas strict accuracy requires every reference group for a question.Source: §E.1 · p. 20

E.2 FULL FANOUTQA RESULTS

The full-results appendix presents FanOutQA accuracy and latency comparisons, receiver-sampling procedures, LongBench results, latency breakdowns, and serving settings. On LongBench Easy, selected CacheBack settings improve accuracy over same-size text, and latency reductions are concentrated in sender work.

E.2 FULL FANOUTQA RESULTS

  • Figure 11 plots strict accuracy against median TTEOA across four model families; TTEOA runs from question submission through completion of the first receiver answer.The 50 questions are submitted together, so the measure includes queueing as well as sender computation, message preparation and transfer, receiver prefill, and decoding.Source: Figure 11 · p. 21, §E.2 · p. 21

E.3 FANOUTQA RECEIVER SAMPLING

  • The receiver-sampling procedure holds messages fixed across three receiver draws and varies the receiver seed, with uncertainty estimated by hierarchical bootstrap over questions and draws.The bootstrap uses 10,000 samples; the reported interval is the 2.5th and 97.5th percentiles.Source: §E.3 · p. 22

E.4 LONGBENCH

  • At selected LongBench v2 settings, CacheBack improves mean accuracy over same-size text by 6.0–14.7 percentage points; for Qwen and Nemotron, both relative and fixed budgets also offer settings with higher accuracy and lower latency.Source: §E.4 · p. 23

E.5 LONGBENCH LATENCY BREAKDOWN

  • CacheBack mainly reduces sender queueing and processing in the LongBench latency breakdown, with the relative budget faster than the fixed budget.Figure 13 decomposes mean TTEOA by stage.Source: §E.5 · p. 23, Figure 13 · p. 23

F SERVING AND EVALUATION SETTINGS

  • Evaluations use one node with eight H100 80GB GPUs and BF16 weights; the work used approximately 500 eight-H100 node-hours.Models use thinking mode, top-p = 0.95, and their recommended sampling settings.Source: F SERVING AND EVALUATION SETTINGS · p. 24

F.1 TIMING AND OUTPUT

  • TTEOA measures submission to completion of the first receiver answer and reports p50 and p95 over 50 questions submitted together.It includes queueing for both sender and receiver GPUs.Source: §F.1 · p. 25

F.2 FANOUTQA EVALUATION AND SCORING

The appendix specifies FanOutQA sampling and scoring, reports a reference audit, describes the LongBench Easy panel and chain, and documents prompts and CacheBack row placement. Comparisons use shared questions, receiver prompts, and model-specific formats.

F.2 FANOUTQA EVALUATION AND SCORING

  • 50 FanOutQA development questions were selected after requiring at least 40,000 context tokens per sender; strict accuracy requires every reference group, while loose accuracy counts the fraction matched.The authors corrected nine reference strings in seven questions before final runs; rescoring saved answers preserved CacheBack’s higher accuracy than same-size text at reported operating points for all four families.Source: §F.2 · p. 25, Table 13 · p. 26

F.3 LONGBENCH EVALUATION AND SCORING

  • The LongBench v2 evaluation uses 50 documents of 100K–250K tokens and restricts the final panel to Easy questions after long Qwen text reports often reached the output ceiling.Documents are split into quarters; each later agent receives the previous message, reads the next quarter, and sends a new message for the receiver’s answer.Source: §F.3 · p. 26

G.1 FANOUTQA PROMPTS

  • FanOutQA text messages fill receiver report slots, while CacheBack supplies continuous rows at selected positions; the question and answer instruction remain unchanged.Source: §G.1 · p. 27

G.2 LONGBENCH PROMPTS

  • LongBench uses the same question, choices, and final receiver prompt for text and CacheBack; text passes messages through the chain while CacheBack passes selected positions.Source: §G.2 · p. 28

G.3 CHAT-TEMPLATE PACKING

  • The four model families retain their native prompt formats, with CacheBack rows inserted according to documented model-specific placements.Packing is consistent across settings within each benchmark and model family.Source: §G.3 · p. 29
Loading 2609.32046v1…