PaperlayerView exact revision on arXiv

Source-linked AI summary

LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration

Shinan Zhang, Tao Zhang, Qihui Zhu, Mengjie Zhang, Dong Jin, Yunpeng Hou, Shuangwu Chen, Xiaobin Tan, Quan Zheng, Jian Yang

arXiv:2609.37017v1cs.CL

TL;DR

Forwarding all sender latents makes receiver context grow with the number of agents and reasoning length, while independent compression leaves cross-agent redundancy unresolved. LatCom compresses multiple senders into fixed receiver-readable slots through readability and fusion learning. Across multiple benchmarks, it reports faster inference and lower output token usage while maintaining comparable average accuracy.

  • Problem

    Forwarding all sender latents scales receiver context with agent count and reasoning length, while independent sender compression leaves cross-agent redundancy unresolved.

    Source: §1 · p. 1

  • Method

    LatCom maps multiple sender latents into fixed receiver-readable slots, training first for single-sender readability and then for multi-sender fusion and redundancy removal.

    Source: §1 · p. 2

  • Results

    For Qwen3-4B, LatCom achieves 2.46× average inference speed-up over LatentMAS and reduces output token usage by 70.3%, while maintaining comparable average accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across multiple benchmarks and models, LatCom preserves strong task performance while substantially reducing inference latency and token usage.

    Source: §6 · p. 9

    LatCom relies on aligned hidden representations within the same model family, so compression and receiver conditioning may be suboptimal in heterogeneous multi-agent systems.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM-based multi-agent systems (MAS) increasingly use latent collaboration to avoid the information loss and repeated encoding-decoding overhead of natural-language communication. However, directly forwarding all sender latents makes the receiver-side context scale with both the number of agents and the reasoning length, increasing computation, memory usage, and collaboration latency. A natural solution is latent compression. But we find that cross-agent redundancy remains unresolved in existing latent compression approaches, which typically compress each sender independently and then concatenate the results. We propose LatCom, a cross-agent latent compression framework for efficient multi-agent latent collaboration. LatCom maps multiple sender latents into a fixed number of receiver-readable and task-relevant slots. Rather than reconstructing all sender hidden states, it optimizes the compressed latents for receiver-side task utility. LatCom trains the compressor in two stages: single-sender readability learning first establishes a latent interface interpretable by the frozen receiver, and multi-sender fusion learning then trains the compressor to fuse complementary evidence and remove redundancy across agents. Experiments on multiple benchmarks with Qwen3-4B show that LatCom achieves an average 2.46x inference speed-up over LatentMAS and reduces output token usage by 70.3% while maintaining comparable average accuracy.

1 Introduction

Latent collaboration avoids natural-language encoding and decoding overhead, but forwarding all sender thoughts makes receiver context grow with agent count and reasoning length. LatCom jointly compresses sender latents into fixed, receiver-readable slots, addressing redundancy while preserving task utility.

1 Introduction

  • Forwarding all latent thoughts makes receiver context scale with agent count and reasoning length, increasing computation, memory use, and collaboration latency.Source: §1 · p. 1
  • Independent sender compression leaves cross-agent redundancy: cosine similarity is 0.553–0.771, and joint effective rank is about 43% below summed individual ranks.The measurements cover GSM8K, ARC-Easy, and ARC-Challenge.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • 2.46× average inference speed-up over LatentMAS and 70.3% lower output token usage, with comparable average accuracy, are reported for Qwen3-4B.Experiments span multiple benchmarks.Source: §1 · p. 2
  • LatCom maps multiple sender latents into fixed receiver-readable slots, using readability learning followed by fusion learning to combine evidence and remove redundancy.It optimizes for receiver-side task utility rather than reconstructing all sender hidden states.Source: §1 · p. 2

2 Preliminary

Senders communicate through continuous latent thoughts rather than decoding intermediate text. Latent compression aggregates those thoughts and maps them to a smaller receiver-readable context, which is optimized for the receiver’s generation task.

2.1 Multi-Agent Latent Collaboration

  • Senders map last-layer hidden states into aligned continuous latent messages and reuse each aligned vector as the next latent input, avoiding intermediate text generation.Latent communication preserves continuous reasoning signals while avoiding repeated decoding and re-encoding.Source: §2.1 · p. 2

2.2 Latent Compression

  • The compressor aggregates sender trajectories with task, sender-identity, and separator embeddings, then maps the sequence into K slots where K < L.The sequence preserves sender boundaries while exposing complementary evidence and redundancy to the compressor.Source: §2.2 · p. 2
  • Compressed slots are prepended as continuous context to the receiver prompt, and compression is trained to support the frozen receiver’s generation of the target answer.The objective focuses on retaining task-critical information rather than reconstructing all sender states.Source: §2.2 · p. 3, Figure 2 · p. 3

3 LatCom

LatCom aggregates sender latents and compresses them into fixed-size slots for a frozen receiver. Its two-stage training first establishes a receiver-readable interface, then fuses multi-source evidence while reducing irrelevant or redundant content.

3.1 Overview of LatCom

  • Sender latent thoughts are aggregated into a multi-sender sequence and compressed into fixed receiver-readable slots, which condition the frozen receiver’s final generation.The slots are prepended to the receiver prompt as continuous context.Source: §3.1 · p. 3

3.2 Two-stage Compressor Training

  • Only the compressor is updated; Stage 1 uses gold-evidence single-sender trajectories to learn a receiver-readable interface through task, contrastive, and alignment objectives.The alignment loss discourages answer-triggering soft prompts that ignore sender information.Source: §3.2 · pp. 3–4
  • Stage 2 compresses multiple senders’ gold, non-gold, and mixed evidence, training fusion under varying source quality and distracting or redundant content.The selected inputs are processed by frozen senders, and the receiver-side decoding loss provides task supervision.Source: §3.2 · p. 4
  • Multi-source contrast compares matched inputs with variants replacing gold-evidence senders; non-gold and mixed evidence remain in the input rather than serving as negatives.This contrast trains the compressor to identify task-supporting sources under non-uniform source quality.Source: §3.2 · pp. 4–5

4 Experiments

Across seven benchmarks, LatCom achieves the highest average accuracy in the reported Qwen3-4B and Qwen3-8B results while reducing inference time and output token use. The experiments also examine latent structure, training objectives, and sensitivity to sender and slot counts.

4.1 Experimental Setup

  • LatCom is evaluated on seven benchmarks with Qwen3-4B and Qwen3-8B backbones, using a three-sender setting and 64 compressed slots.Training uses multi-hop QA data; sender and receiver parameters remain frozen while only the compressor is updated.Source: §4.1 · p. 5

4.2 Main Results

  • 76.27 on Qwen3-4B and 80.15 on Qwen3-8B are LatCom’s highest average accuracies among the compared methods.It improves average accuracy over InterLat by 1.9 points and 1.77 points, respectively, while maintaining comparable performance to LatentMAS at lower inference cost.Source: §4.2 · p. 5, Table 1 · p. 6

4.3 Efficiency Analysis

  • 2.46× average inference speed-up and 70.3% lower output token usage are reported across seven benchmarks on Qwen3-4B.Speed-ups range from 1.85× on ARC-Challenge to 3.75× on GSM8K; token reductions range from 63.6% on ARC-Easy to 81.3% on GSM8K.Source: §4.3 · p. 6, Figure 3 · p. 6, Figure 4 · p. 6

4.4 In-depth Analyses on LatCom

  • LatCom’s slots form a more concentrated distribution than overlapping sender states, while receiver entropy is lower across most reasoning positions.The entropy comparison uses three 50-token windows from the beginning, middle, and final parts of GSM8K reasoning traces.Source: §4.4 · pp. 6–7, Figure 5 · p. 7, Figure 6 · p. 7

4.5 Ablation Study

  • Adding multi-sender utility learning improves accuracy by an average of 9.0 points across benchmarks; removing Lcon, Lalign, or Lms lowers GSM8K accuracy by 5.53, 11.65, or 2.37 points, respectively.The largest objective-removal drop is from Lalign; the paper identifies alignment to dense sender trajectories as critical for receiver-readable slots.Source: §4.5 · p. 7, Figure 7 · p. 7, Table 2 · p. 7

4.6 Parameter Sensitivity Analysis

  • With a fixed slot budget, performance is stable from three to four senders and drops by only about 2.2 points on average at eight senders.The reported average covers GSM8K, ARC-Easy, and ARC-Challenge.Source: §4.6 · pp. 7–8, Figure 8 · p. 8
  • K = 8 causes clear accuracy drops, performance is best or near-best around K = 32 to K = 64, and K = 128 brings no further gain.The main experiments use K = 64 as a stable accuracy-compactness trade-off.Source: §4.6 · p. 8, Figure 9 · p. 8

5 Related Works

Prior work uses natural-language exchange or latent representations for multi-agent collaboration, while compression methods often operate on individual senders or single-model caches. LatCom instead compresses multiple sender trajectories into receiver-readable slots.

LLM-based multi-agent systems

Latent collaboration in multi-agent systems

Compression for multi-agent systems

  • Text-oriented compression methods ICAE (Ge et al., 2024) and AutoCompressor (Chevalier et al., 2023) still require text-to-latent conversion, while H2O, StreamingLLM (Xiao et al., 2024), and SnapKV (Li et al., 2024) target mainly single-model cache management.Source: §5 · p. 8
  • Closest to LatCom, Du et al. (2025) compresses each sender’s latent trajectory independently, without explicitly modeling cross-sender redundancy or complementarity before aggregation.LatCom instead performs receiver-aware multi-source compression into a fixed number of receiver-readable slots optimized for downstream task utility.Source: §5 · pp. 8–9

6 Conclusion

LatCom addresses cross-agent latent redundancy with fixed-budget, receiver-readable slots and two-stage training. Across multiple benchmarks and models, it preserves strong task performance while reducing inference latency and token usage.

6 Conclusion

  • LatCom compresses multiple sender latents into fixed-budget, receiver-readable, task-relevant slots to mitigate receiver-side context growth.Source: §6 · p. 9
  • Its two-stage training first establishes a latent interface for the frozen receiver, then learns to fuse complementary evidence while reducing cross-agent redundancy.Source: §6 · p. 9
  • Experiments across multiple benchmarks and models show that LatCom preserves strong task performance while substantially reducing inference latency and token usage.Source: §6 · p. 9

Limitations

LatCom’s scope is constrained by its assumptions about model alignment and fixed-slot compression. The paper identifies potential losses in heterogeneous systems and some code-generation tasks.

Limitations

  • LatCom relies on aligned hidden representations within the same model family, so compression and receiver conditioning may be suboptimal in heterogeneous systems.The stated sources of heterogeneity include architecture, hidden dimensionality, tokenizers, and communication topologies.Source: Limitations · p. 9
  • LatCom uses a fixed-slot budget for multi-source trajectories, and full latent relay remains stronger on some code-generation tasks.The authors suggest fixed-slot compression may discard fine-grained information required for certain forms of reasoning.Source: Limitations · p. 9
  • The paper identifies heterogeneous model settings, adaptive communication budgets, and reliability-sensitive applications as directions for future work.Source: Limitations · p. 9

Ethics Statement

LatCom may inherit limitations of its underlying LLMs, including biased, hallucinatory, or factually incorrect outputs. The authors advise verification, especially for high-stakes uses.

Ethics Statement

  • Because LatCom is built on existing LLMs, it may inherit biased predictions, hallucinations, and factual errors.Source: Ethics Statement · p. 9
  • Users should verify outputs in real-world deployments, especially for safety-, privacy-, or high-stakes applications.Source: Ethics Statement · p. 9
  • Experiments use public benchmarks, Qwen3 models, PyTorch, and Hugging Face Transformers, with their respective licenses and usage policies followed.Source: Ethics Statement · p. 9

A.1 Quantitative Analysis of Cross-Agent Redundancy

Cross-agent latent trajectories share representational and trajectory-level structure, while joint multi-sender fusion outperforms simple pooling and additional single-source training. These findings support compressing sender trajectories jointly.

A.1 Quantitative Analysis of Cross-Agent Redundancy

  • The redundancy analysis compares pairwise trajectory similarities and the effective ranks of individual versus joint agent trajectories.Raw average cosine measures directional similarity, centered cosine removes the mean direction, and linear CKA measures overall representation-structure similarity.Source: §A.1 · p. 11
  • Across GSM8K, ARC-Easy, and ARC-Challenge, Science–Code raw cosine similarity is 0.771, 0.747, and 0.717, while Math–Science linear CKA is 0.600, 0.577, and 0.543.The analysis measures pairwise raw and centered cosine similarity and linear CKA; CKA compares overall representation structures.Source: §A.1 · p. 11, Table 3 · p. 12
  • Rank compression remains close to 43% across all three datasets, with joint effective rank substantially below the sum of individual ranks.The reported result indicates that combined-trajectory effective dimensionality does not grow independently with agent count.Source: §A.1 · p. 11, Table 4 · p. 12

A.2 Controlled Analysis of Cross-Agent Fusion

  • Joint Mean Pool is 10.92 percentage points below LatCom, and Single-source Continuation is 3.37 points below LatCom, favoring learned joint fusion over pooling or extra single-source training.Single-source Continuation uses the same compressor architecture, slot number, training-data scale, and optimization budget.Source: §A.2 · pp. 11–12, Table 5 · p. 12

B Training Details

LatCom training uses multi-hop QA examples selected to emphasize multi-source evidence compression, then trains receiver-readable slots and multi-sender fusion using evidence-structured inputs.

B.1 Training Data Construction

  • Training examples come from HotpotQA and MuSiQue-Ans, and filtering removes single-model-solvable and precompression-unanswerable cases to focus on multi-source evidence compression.Rationale-augmented targets combine an evidence-grounded rationale with the original gold answer, which remains consistent with the dataset annotation.Source: §B.1 · pp. 12–13

B.2 Evidence-structured Training Instances

  • Stage 1 uses gold-evidence inputs to learn a receiver-readable latent interface; Stage 2 trains on heterogeneous multi-source inputs with 2 to 6 senders.Stage 2 also includes some mathematical and code reasoning examples, with task-supporting, distracting, and redundant sources.Source: §B.2 · p. 13

B.3 Example of Constructed Training Instance

  • Figure 10 illustrates converting a multi-hop QA example into evidence-structured sender inputs and a rationale-augmented target.The target is formatted with reasoning followed by the answer.Source: §B.3 · p. 13, §B.1 · p. 13

B.4 Training Hyperparameters

  • Sender and receiver parameters stay frozen during compressor training, and only compressor parameters are updated.Optimization hyperparameters are reported in Table 6.Source: §B.4 · p. 13

C.1 Evaluation Benchmarks

The evaluation covers seven benchmarks and compares text-based and latent communication methods under shared hierarchical multi-agent settings. The compressor adds a reported average of 0.116 seconds of runtime.

C.1 Evaluation Benchmarks

  • The seven-benchmark suite spans mathematical, scientific and medical QA, commonsense reasoning, and code generation.The benchmarks are GSM8K, ARC-Easy, ARC-Challenge, MedQA, MBPP-Plus, HumanEval-Plus, and GPQA-Diamond.Source: §C.1 · p. 13

C.2 Compared Methods

  • Comparisons include TextMAS, LatentMAS, InterLat, LatentMAS-Hidden, and LatentMAS-H2O under a shared hierarchical multi-agent setting.Unless otherwise specified, methods use the same sender roles, receiver role, prompts, decoding settings, and evaluation scripts.Source: §C.2 · pp. 14–15

C.3 Implementation Details

  • All methods use three senders and one receiver; latent-communication senders perform 40 reasoning steps with latent-space realignment enabled.GSM8K, ARC-Easy, and ARC-Challenge use 2,048-token output limits; MedQA, MBPP-Plus, and HumanEval-Plus use 4,096 tokens.Source: §C.1 · p. 14, §C.3 · p. 15

C.4 Compressor Overhead

  • 0.116 seconds is the average time for global compression from aggregated latent sequence U into fixed-slot message M.The measurement excludes sender rollout, receiver prompt construction, and receiver decoding; Table 8 describes the compressor overhead.Source: §C.4 · p. 15, Table 8 · p. 16
Loading 2609.37017v1…