Source-linked AI summary
Dual-Cache Latent Space Communication between Heterogeneous Language Models
Jiyao Liu, Qi Zhang, Yaoyi Jia, Ziwen Kan, Song Wang
TL;DR
Communication between agents is difficult when complementary evidence resides in different model contexts, and text exchange adds decoding overhead. XKV jointly translates frozen models’ KV caches into receiver-specific native updates, achieving the strongest overall results across 45 settings while reducing translation and end-to-end costs. Its scope includes heterogeneous architectures and tokenizers, with only the translator trained.
Problem
Complementary evidence can reside in separate agents’ contexts, while existing cache protocols remain limited by receiver-independent compression, layer-local summaries, or matched architectural geometry.
Method
XKV pools both frozen models’ KV caches into a learned cross-layer joint memory, then lets each receiver position retrieve a native-geometry residual.
Results
XKV achieves the best overall score across 45 settings, improves on LCF-X on every dataset, and translates cache pairs 10.3× faster with 76% fewer trainable parameters.
Takeaways & Limitations
Receiver-aware, position-specific cache communication can outperform text exchange and sender-centric latent summaries while reducing communication cost.
Takeaways & Limitations
The frozen Sharer and Receiver may differ in layers, KV heads, head dimensions, and tokenizer, but the models themselves remain frozen and only the translator is trained.
Abstract
from arXiv · showhide
Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.
Introduction
XKV addresses key limitations of latent communication between heterogeneous language models by jointly translating both KV caches into receiver-specific updates. Across 45 settings, it improves quality while reducing translation and end-to-end communication costs.
- Multi-agent systems require communication when complementary evidence is distributed across agents, but text exchange adds autoregressive decoding and re-encoding overhead.
- LCF-X remains receiver-independent, layer-local, and constrained by matched layer count and KV geometry.It pools the sharer alone, broadcasts one summary per receiver layer, and cannot retrieve across multiple summaries or layers.
- XKV pools both caches into k candidate summaries, reconciles depths with a learned layer map, and mixes them into a compact joint cross-layer memory.
- Each receiver cache position queries the shared memory and receives a distinct residual in the receiver’s native KV geometry.
- 52.4 macro score is achieved by XKV across 45 settings, versus 49.8 for LCF-X and 48.9 for T2T.XKV improves on LCF-X on every dataset and exceeds text communication on four of five datasets.
- 10.3× faster translation and 76% fewer trainable parameters accompany XKV’s quality gains, while end-to-end communication is 6.8× faster than T2T.The translator averages 5.8 ms versus 59.9 ms for the LCF-X fusor.
Related Work
Prior work uses text, hidden states, or KV caches to communicate between models, but existing cache protocols impose shared-content or architectural constraints. XKV extends this line with joint, cross-layer translation between frozen heterogeneous models.
- Text-based multi-agent communication is portable and interpretable but incurs autoregressive generation and re-encoding at every handoff.
- C2C supports heterogeneous models by fusing position-aligned caches, but assumes both models process the same underlying input.
- LCF-X removes the shared-context requirement by pooling the sharer cache into a position-free summary before layer-local residual fusion.
- XKV builds a cross-layer memory from both caches and lets each receiver position retrieve a distinct update through shared parameters.
- Learned layer maps and query-based pooling connect XKV to prior methods for bridging frozen representation spaces and compressing sets of vectors.
Method
XKV is a lightweight translator between two frozen language models that jointly compresses both KV caches into receiver-aligned memory, then writes position-specific residuals in the receiver’s native geometry. Its shared components support differing depths, cache dimensions, tokenizers, and architectures while adding a one-shot cache update before decoding.
- Joint cache communication: XKV jointly pools the Sharer and Receiver caches, aligns layer axes, and mixes summaries into a compact cross-layer memory.Learned queries produce k summaries per layer and head; a learned layer map handles depth differences before bottlenecking and self-attention.
- Position-specific reconstruction: Each raw receiver cache position queries the joint memory and retrieves its own receiver-native KV residual.A shared cross-attention decoder uses receiver cache entries as queries, while shared output heads produce gated updates for each position, head, and cache type.
- Heterogeneous models: XKV supports frozen model pairs with different layer counts, KV-head counts, head dimensions, and tokenizers.The translator produces receiver-shaped residuals, operates on the common valid cache suffix where needed, and leaves both base models frozen.
- Training and inference: Zero-initialized shared output heads make an untrained translator reproduce receiver-only decoding while training learns a residual correction.The residual update is added once to the prompt cache; generated-token cache entries are appended normally and are not rewritten.
- Efficiency: Shared pooling, translation, decoding, gating, and output components reduce depth-dependent parameters and avoid attention over all raw receiver cache rows.Cross-layer attention operates only on receiver-layer memory tokens, while the position decoder attends from each receiver position to those tokens.
Experimental Setup
The evaluation uses split-evidence reasoning tasks with frozen language models arranged across the complete 3 × 3 sharer–receiver grid, comparing XKV with text and latent-cache communication.
- Models: The model grid contains nine ordered pairings across Qwen, Gemma, and Llama, including six heterogeneous and three same-model settings.The heterogeneous pairs span differences in model family, depth, attention geometry, and tokenizer.
- Tasks: Five benchmarks split complementary evidence between Sharer and Receiver models, reporting exact match and F1 for generative QA and accuracy for classification.The question and classification answer choices remain visible to both models, while evidence is partitioned.
- Baselines: XKV is compared with T2T text communication and LCF-X latent-cache communication, with LCF-X extended to accept mismatched layer counts and KV dimensions.T2T requires autoregressive message generation and re-encoding, while LCF-X transfers gated KV residuals from a position-free sharer summary.
- Training and latency: Only the lightweight communication module is trained for XKV and LCF-X; all Sharer and Receiver model weights remain frozen.Communication latency measures only the latent module, whereas end-to-end latency includes both model forward passes and answering.
- Efficiency: Figure 5 reports pair-level communication and end-to-end latency savings for XKV, averaged over five datasets with sharers as rows and receivers as columns.Positive values indicate that XKV is faster than the comparison baseline.
Results
XKV delivers the strongest overall quality across the nine model pairings while also reducing communication and end-to-end latency; ablations attribute gains to receiver-aware design choices and direct pooling.
- Overall quality: XKV achieves the best or tied-best cross-dataset average in eight of nine settings and the best result in 31 of 45 dataset–pair cells.It improves on LCF-X across every dataset and leads T2T on four of five datasets.
- Overall quality: +2.61 macro points is XKV’s improvement over LCF-X across datasets, including up to +4.2 F1 on ROPES.XKV leads T2T on four datasets by +3.52 macro points overall, while trailing by 0.27 points on QASC.
- Latency: 5.8 ms versus 59.9 ms is the average translator-versus-fusor communication latency, making XKV 10.3× faster than LCF-X.Across all 45 dataset–pair cells, mean end-to-end latency falls from 227.9 to 167.6 ms, a 26.4% reduction.
- Latency: 6.8× faster end to end is XKV’s advantage over T2T, whose autoregressive message generation dominates its latency budget.XKV is fastest end to end in all fifteen same-model dataset cells and cuts end-to-end time by 26–28% relative to LCF-X.
- Ablations: −0.83 average score results when receiver cross-attention is removed, compared with −0.46 when receiver pooling is removed.The cross-attention ablation costs −1.98 on QASC, while neither removal changes end-to-end latency by more than 3.5%.
- Ablations: 6.7× higher translator latency and a 0.94-point lower mean score result when XKV’s direct pooling is replaced by LCF-X-style span pooling.The replacement also increases end-to-end latency by 39.4% and lowers ROPES F1 by 2.00.
Conclusion
XKV is a latent communication protocol for frozen heterogeneous language models with complementary evidence, combining joint cache translation with receiver-position retrieval.
- Conclusion: XKV pools both KV caches into a compact cross-layer memory and lets each receiver cache position retrieve its own receiver-native residual.The protocol is evaluated across five datasets and six heterogeneous plus three same-model ordered pairings.
- Conclusion: XKV improves on LCF-X on every dataset while using 76.1% fewer trainable parameters and translating caches 10.3× faster.End-to-end communication is 6.8× faster than text exchange.
- Conclusion: Ablations show that receiver-aware components improve quality at negligible inference cost.The conclusion identifies receiver-aware, position-query cache communication as removing the observed quality–efficiency trade-off.