Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Tianyu Fu; Zihan Min; Hanling Zhang; Jichao Yan; Guohao Dai; Wanli Ouyang; Yu Wang

Source-linked AI summary

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang

arXiv:2510.03215v2Oct 3, 2025cs.CLcs.LG

TL;DR

Existing text-based multi-LLM communication loses rich semantic information and incurs sequential decoding latency, motivating communication beyond text. C2C projects and fuses source and target KV-Caches for direct semantic transfer, achieving higher accuracy than individual models and text communication with lower latency. Its main scope boundary is that noisy information from weaker Sharers can degrade Receiver performance, and scaling beyond pairwise communication remains open.

Problem
Text-to-text interfaces bottleneck multi-LLM communication by compressing rich internal representations into ambiguous token sequences and requiring sequential generation.
Method
C2C projects and fuses a source model’s KV-Cache with a target model’s cache to enable direct semantic communication between LLMs.
Results
Across tasks and model configurations, C2C improves performance and efficiency over text-to-text communication, including 6.4-14.2% higher average accuracy than individual models and an average 2.5× latency speedup.
Takeaways & Limitations
KV-Cache can serve as a practical alternative to token-based communication for low-latency multi-LLM systems.
Takeaways & Limitations
Performance can degrade when a much weaker Sharer provides noisy information, and scaling beyond pairwise communication with O(N) training cost remains open.

Abstract

from arXiv · show

Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.

1 INTRODUCTION

Multi-LLM systems exploit complementary model capabilities, but text-based communication can bottleneck and distort rich semantic exchange. Cache-to-Cache (C2C) uses KV-Cache as a richer, parallel communication medium and improves accuracy and latency over individual models and text communication.

1 INTRODUCTION: C2C directly projects and merges KV-Caches from different LLMs instead of passing information through explicit generated text.The approach targets complementary semantic representations across specialized models.
1 INTRODUCTION: Text-to-text communication compresses high-dimensional representations into linear strings, creating information bottlenecks and ambiguity between differently specialized models.The paper illustrates this problem with ambiguous interpretations of <p> as a section marker or paragraph separator.
1 INTRODUCTION: KV-Cache communication supports richer semantics and fully parallel transfer, avoiding the sequential decoding latency of text exchanges.Oracle experiments show that cache enrichment improves accuracy without increasing context length and that KV-Caches are convertible across models.
1 INTRODUCTION: 6.4-14.2% higher average accuracy than individual models, C2C also exceeds text-to-text communication by approximately 3.1-5.4%.C2C projects and fuses source and target KV-Caches for direct semantic transfer.
1 INTRODUCTION: C2C delivers an average 2.5× latency speedup over text communication.The speedup comes from avoiding intermediate token-by-token text generation.

2 RELATED WORK

Prior multi-LLM research primarily communicates through natural-language messages or coordinates models through routing and token-level selection. Cache-sharing research largely targets reuse and inference efficiency, whereas C2C uses KV-Cache for semantic transfer across LLMs.

Cache sharing: Existing cache-sharing methods reuse KV-Cache within the same model or across closely related models, mainly to accelerate inference or reuse context.DroidSpeak extends reuse to models fine-tuned from the same base model.
Cache sharing: C2C differs by using KV-Cache as a medium for semantic transfer between LLMs rather than only as a computational reuse mechanism.This distinction separates C2C from intra-model and cross-query cache reuse.
Collaborative multi-LLM systems: Collaborative multi-LLM systems exchange natural-language messages through sequential or layered communication architectures.Examples include Chain-of-Agents, MetaGPT, Mixture-of-Agents, DyLAN, and Owl.
Routing-based multi-LLM inference systems: Routing-based systems select models or tokens with different capabilities and costs to balance inference efficiency and performance.These methods coordinate model computation rather than directly transferring KV-Cache semantics.

3 METHOD

The method investigates KV-Cache as a direct communication medium between LLMs and develops C2C to enrich a Receiver’s cache with a Sharer’s contextual information. Oracle experiments test cache enrichment and cross-model convertibility, while the learned fuser projects, combines, and selectively injects cache information.

3.2 ORACLES FOR CACHE-TO-CACHE COMMUNICATION: Oracle enrichment improves response quality at the same cache length by retaining only the question-aligned cache slice after exemplar-conditioned prefilling.Comparing Direct and Oracle isolates semantic enrichment from the benefit of attending to additional exemplar tokens.
3.2 ORACLES FOR CACHE-TO-CACHE COMMUNICATION: Selective layer enrichment guides C2C’s gating: enriching strong-performing layers improves accuracy, whereas enriching weak-performing layers can reduce it.The supplied figure description reports increased accuracy when more best-performing layers are enriched and declining accuracy for the worst-performing layers.
3.2 ORACLES FOR CACHE-TO-CACHE COMMUNICATION: Cross-model transformation maps Qwen3-4B KV-Caches into Qwen3-0.6B’s representation space, demonstrating that caches from different models are generally convertible.The transformed cache lies within the target representation space but occupies only a subset of it.
3.3.1 OVERVIEW: C2C fuses a Sharer’s contextual understanding into a Receiver’s KV-Cache through projection, residual integration, and learnable layer-wise gating.The fuser combines corresponding caches, dynamically reweights projected information, and uses a trainable gate to decide whether to inject the Sharer’s context.
3.3 METHOD: During training, both LLMs remain frozen while the C2C module learns cache fusion from Receiver response-prediction loss.The procedure encodes both inputs, fuses their caches, replaces the Receiver cache, and backpropagates through C2C.

4 EXPERIMENT

Across diverse models, benchmarks, sizes, and communication settings, C2C improves accuracy and reduces latency relative to text communication, with gains linked to complementary semantics and cache-fusion design choices.

Performance: C2C increases Receiver accuracy by 11.00%, 9.64%, and 11.88% across three Sharers, while improving over T2T by 5.36%, 4.15%, and 3.06%.The comparison covers four benchmarks with the Receiver fixed as Qwen3-0.6B and three different Sharers.
Efficiency: C2C achieves 3.46×, 1.51×, and 14.41× speedups over T2T by replacing 80-token sequential decoding with 90ms parallel cache fusion.T2T incurs 1312ms of decoding overhead in the reported MMLU-Redux setting.
Scaling model sizes: When Sharer size increases, C2C accuracy improvements generally grow faster than T2T, although gains are less pronounced for larger Receivers.Larger Receivers have stronger baselines and greater knowledge overlap with the Sharer.
Performance: Swapping Sharer and Receiver models yields a 5.05% C2C accuracy increase, whereas T2T produces a 6.30% decrease.This result supports robustness across communication direction.
Sources of improvement: Heterogeneous Sharers outperform both standard Receiver fine-tuning and identical-model cache communication, indicating that complementary contextual understanding contributes to C2C gains.Identical-model cache communication still outperforms the Single baseline, showing that cache-level self-communication can also help.
Fuser architecture: Retaining the Receiver cache through residual fusion raises accuracy by 24.18% over pure projection, and learnable layer gating adds a further 3.07%.These ablations isolate the contributions of cache fusion and fused-layer selection.
Accuracy breakdown: C2C can fail when the Sharer’s contextual understanding is inaccurate, misleading the Receiver into generating a wrong answer.The source of accuracy gains also varies with relative model capacity and task subcategory.
Progressive behavior: Above 50% context-cache updating, increasing the updated percentage continuously improves performance, while gate behavior differs between general-purpose and task-specific training.General-purpose training favors broad gate activation; task-specific training favors sparse activation with stronger reliance on selected layers.

5 DISCUSSION

The discussion identifies future applications for cache communication and two boundaries: noisy weak Sharers can degrade stronger Receivers, and multi-LLM scaling remains unresolved.

Future work: C2C may support multiagent reasoning, coding, tool use, and cross-modal collaboration as a communication primitive.The paper also points to privacy-aware collaboration through transmitting KV-Cache segments without explicit text.
Limitations: A much weaker Sharer can provide noisy information that degrades a stronger Receiver, affecting both T2T and C2C.Receiver performance depends directly on Sharer semantic quality.
Limitations: Scaling pairwise KV-Cache communication to more LLMs remains open because the current training cost grows as O(N).The paper reports preliminary efforts toward this extension.

6 CONCLUSION

The paper presents C2C as a practical alternative to token-based communication, combining direct KV-Cache semantic transfer with higher performance and better efficiency across diverse settings.

Conclusion: C2C transforms and fuses KV caches across models to enable direct semantic communication beyond text.Across diverse tasks and model configurations, it achieves higher task performance and better efficiency than T2T communication.

ETHICS STATEMENT

The study reports no ethical issues because it involved neither human subjects nor sensitive personal data.

Ethics statement: The experiments involved no human subjects or sensitive personal data.The paper therefore reports no ethical issues for this study.

A.1 DESIGN CHOICE EXPLORATION

This section details the design of C2C and discusses alternative design choices.

The section presents the design of C2C.
It examines alternative design choices for C2C.
The discussion focuses on design decisions rather than introducing a separate system.

A.1.1 LAYER ALIGNMENT

The paper compares terminal and depth-normalized layer alignment strategies, then adopts terminal alignment for its simpler mapping and slightly better empirical performance.

A.1.1 LAYER ALIGNMENT: Terminal alignment pairs corresponding layers from the output side, prioritizing deeper semantic representations.The final layers are paired first, followed by penultimate layers and so on.
A.1.1 LAYER ALIGNMENT: Depth-normalized alignment matches layers whose normalized positions across model depths are closest.The smaller model provides the anchor layers, and correspondences are distributed approximately uniformly across depth.
A.1.1 LAYER ALIGNMENT: C2C adopts terminal alignment because it is simpler, more direct, and slightly better empirically.

A.1.2 TOKENIZATION ALIGNMENT

Tokenization alignment treats chat templates structurally and message content semantically, combining padding with tokenizer-based mappings for robust cross-model correspondences.

A.1.2 TOKENIZATION ALIGNMENT: The resulting design aligns template sections structurally and message sections semantically at token level.
A.1.2 TOKENIZATION ALIGNMENT: Template tokens are aligned by padding the shorter sequence with <pad> tokens.This preserves sequence consistency for structural markers that differ across tokenizers and carry no semantic content.
A.1.2 TOKENIZATION ALIGNMENT: Message tokens are decoded and re-encoded across tokenizers to establish direct or many-to-one correspondences.Special tokens are mapped directly when possible, while regular-token mappings depend on the resulting source-tokenization.
A.1.2 TOKENIZATION ALIGNMENT: Maximal-coverage selection is adopted by default because over 80% of sequences yield identical alignments and it reduces information loss risk.

A.1.3 FUSER ARCHITECTURE

C2C transfers contextual understanding through fused KV-Caches, with its benefits varying according to model capacity, task category, and cache-selection strategy. The experiments show stronger gains in strong-to-weak settings and identify adaptive layer gating as an important design behavior.

Models with disparate capacity: 51.01% PGR over the weak-to-strong gap is achieved when C2C pairs Qwen3-0.6B with the stronger Qwen3-4B Sharer across LongBenchV1 input lengths.C2C consistently outperforms both the receiver alone and the T2T baseline in this setting.
Models with disparate capacity: 72.11% of questions answered correctly by the stronger Sharer are also answered correctly by C2C, versus 50.97% with comparable-capacity models.C2C tends to integrate more of the stronger model’s understanding when model capacities differ substantially.
Subcategory breakdown: C2C outperforms T2T on 12 of 17 categories for one model pair and all 17 categories for the Qwen3-4B/Qwen3-0.6B pair.Relative gains include 7, 7.6, and 7.5 accuracy points in history, law, and chemistry for the smaller pair.
Progressive behavior: After more than 50% of the Receiver cache is replaced with fused cache, C2C accuracy continues increasing, with latter-cache projection having the larger impact.Accuracy initially decreases and then increases as the replacement ratio grows; latter cache is closer to the final response.
Gating behavior: Task-specific training opens gates sparsely at a 52.67% average activation ratio, whereas general-purpose training activates over 98.21% of gates with finer weight modulation.The findings indicate that task-specific training selects fewer layers but assigns stronger dynamic weights to those selected layers.