PaperlayerView exact revision on arXiv

Source-linked AI summary

Communicating Activations Between Language Model Agents

Vignav Ramesh, Kenneth Li

arXiv:2501.14082v2cs.CLcs.AIcs.LG

TL;DR

Natural-language communication between language models can be costly and may leave richer internal representations unused. This paper combines models’ intermediate activations, reporting up to 27.0% improvement over natural-language communication across datasets with less than one quarter the compute.

  • Problem

    Natural-language communication incurs high inference costs and may discard richer information encoded in intermediate activations.

    Source: §1 · p. 1

  • Method

    The method combines one model’s intermediate activation with another’s and continues the receiving model’s forward pass, using frozen LMs without task-specific parameters or data.

    Source: §1 · p. 2

  • Results

    Up to 27.0% improvement over natural language communication across coordination games and reasoning benchmarks, using <1/4 the compute.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Activation communication applies across diverse domains and settings while scaling up LMs on new tasks with zero additional task-specific parameters and data.

    Source: §5 · p. 8

    Without a learned model-specific mapping, the method assumes the models’ embedding spaces are aligned; the learned mapping can relax this assumption.

    Source: §5 · p. 8

Abstract

from arXiv · show

Communication between multiple language model (LM) agents has been shown to scale up the reasoning ability of LMs. While natural language has been the dominant medium for inter-LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be otherwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via activations; concretely, we pause an LM $\textit{B}$'s computation at an intermediate layer, combine its current activation with another LM $\textit{A}$'s intermediate activation via some function $\textit{f}$, then pass $\textit{f}$'s output into the next layer of $\textit{B}$ and continue the forward pass till decoding is complete. This approach scales up LMs on new tasks with zero additional parameters and data, and saves a substantial amount of compute over natural language communication. We test our method with various functional forms $\textit{f}$ on two experimental setups--multi-player coordination games and reasoning benchmarks--and find that it achieves up to $27.0\%$ improvement over natural language communication across datasets with $<$$1/4$ the compute, illustrating the superiority and robustness of activations as an alternative "language" for communication between LMs.

1. Introduction

Natural-language communication can be costly and may discard richer information available in intermediate activations. The paper proposes activation-based communication and reports improved performance over natural-language communication with lower compute across coordination games and reasoning benchmarks.

1. Introduction

  • Natural-language communication incurs inference costs that scale with agent count and message length and number, while decoding may discard richer information in intermediate representations.The paper notes that intermediate layers can encode enriched entity representations that later layers transform toward next-word prediction.Source: §1 · p. 1
  • The method combines one model’s intermediate activation with another’s, then resumes the latter model’s forward pass, using frozen LMs without task-specific parameters or data.It requires a partial forward pass through A and one forward pass through B.Source: §1 · p. 2
  • Up to 27.0% improvement over natural language communication across coordination games and reasoning benchmarks, using <1/4 the compute.The evaluation includes two multi-player coordination games and seven reasoning benchmarks spanning multiple domains.Source: §1 · p. 2
  • The paper introduces an activation-based inter-model communication protocol and evaluates its performance and compute savings against natural-language communication.The authors frame activations as an alternative communication medium for LMs.Source: §1 · p. 2

2. Related Work

Prior work spans emergent communication, natural-language LLM interaction, embedding transfer, activation editing, and model grafting. This paper positions activation communication as a compute-efficient way to combine model representations without task-specific training.

2. Related Work

  • Emergent communication can develop in cooperative or competitive games, but learning meaningful languages from scratch remains difficult even with centralized training.The cited examples include Sukhbaatar et al. (2016), Foerster et al. (2016), and Lazaridou et al. (2017).Source: §2 · p. 2
  • Natural-language LLM communication supports cross-domain knowledge integration and complementary abilities, while prior debate studies focused on large models; this work tests smaller models too.The paper reimplements Du et al. (2023) with models from 1–70B parameters.Source: §2 · p. 2
  • CIPHER passes probability-weighted average tokenizer embeddings between models, but remains token-based and does not save compute relative to natural-language communication.The paper distinguishes this embedding transfer from communication through intermediate activations.Source: §2 · p. 2
  • Activation editing has been used to steer generated text; this work instead uses activation edits for communication between LLM agents.Prior steering methods include latent steering vectors and prompt-difference direction vectors.Source: §2 · pp. 2–3
  • Unlike model-composition methods with training or alignment constraints, the framework explores simple activation-combination functions that require no task-specific parameters or data.It unifies activation-level grafting under a function f, contrasting with CALM’s pair-specific attention retraining.Source: §2 · p. 3

3. Communicating Activations Between Language Models

The paper combines one model’s intermediate activation with another’s and analyzes why this may transmit richer information than token-level messages. Its compute analysis concludes that the procedure is substantially cheaper in practical cases.

3.1. Method

  • B pauses at layer j, combines its last-token activation with A’s last-token activation from layer k, then resumes its forward pass to generate an output.The activation function maps the paired representations to a vector in B’s activation dimension.Source: §3.1 · p. 3, Figure 1 · p. 4
  • The method considers sum, mean, and replacement functions, and can use a task-agnostic learned linear mapping when the models’ activation spaces differ.The mapping is trained once per model pair on general text, without task-specific parameters or data.Source: §3.1 · pp. 3–4

3.2. Compute Analysis

  • The activation procedure requires substantially less compute than traditional natural-language communication in all practical cases considered.The analysis compares repeated message-generation passes with a partial pass through A, a pass through B, and activation replacement.Source: §3.2 · p. 4

3.3. Why should this work?

  • The authors argue that late-enough activations encode context knowledge, predicted completions, and confidence, preserving information that later layers discard for next-token prediction.They present activation communication as a single grafting step rather than an iterative exchange of rounds.Source: §3.3 · pp. 4–5

4. Experiments

The experiments test activation communication in coordination games and reasoning benchmarks. AC outperforms natural-language debate and single-model baselines on the reported reasoning benchmarks, with favorable performance-compute tradeoffs and results across model families.

4. Experiments

  • The evaluation includes Countries and Tip Sheets coordination games and seven reasoning datasets: Biographies, GSM8k, and five MMLU subject datasets.The reasoning evaluation uses a randomly sampled 100-example subset of each dataset.Source: §4.1 · p. 5, §4.2 · p. 6

4.1. Multi-player coordination games

  • In both coordination games, replace-based AC nearly recovers the gap between zero communication and the single-agent skyline, outperforming natural-language communication with far less compute.The experiments use two instances of the same model as agents.Source: §4.1 · p. 5

4.2. Reasoning Benchmarks

  • AC consistently beats both single-model baselines and NLD across seven reasoning benchmarks, with up to 27.0% improvement over NLD on six.The table compares LLaMA-3.2-3B and LLaMA-3.1-8B individually with NLD and AC variants.Source: §4.2 · p. 7, Table 3 · p. 8
  • On Biographies, AC consistently achieves larger accuracy gains per added inference-time compute than NLD across the tested LLaMA model scales.The comparison normalizes compute by a single LLaMA-3.2-1B forward pass on the prompt.Source: §4.2 · p. 8
  • For model pairs across Qwen-2.5, Gemma-2, and LLaMA-3, AC beats NLD across the board and beats both individual models for 4/5 of six pairs on Biographies/GSM8k, respectively, without training W.Source: §4.2 · p. 8

5. Conclusion

The conclusion presents activation communication as a compute-efficient way to scale frozen language models to new tasks without task-specific parameters or data. It also identifies access and alignment constraints and discusses interpretability as a tradeoff and future research direction.

5. Conclusion

  • Activation communication injects one model’s activations into another’s forward pass, scaling frozen LMs to new tasks with zero task-specific parameters and data.The conclusion also describes the method as applicable across diverse domains and settings while saving substantial compute.Source: §5 · p. 8
  • AC achieves the greatest performance gains per additional unit of inference-time compute over each baseline on Biographies.Figure 3 plots accuracy against FLOPs normalized by a single LLaMA-3.2-1B forward pass.Source: Figure 3 · p. 8
  • Without a learned model-specific mapping, the method assumes aligned embedding spaces, though a learned mapping can relax this requirement.The authors report that no mapping is needed for state-of-the-art results among LLaMA-family models.Source: §5 · p. 8
  • The method requires access to embeddings and does not work with black-box API access.The authors point to recent open-source model releases as motivation for embedding-based techniques.Source: §5 · pp. 8–9
  • The authors place the interpretability–information-preservation tradeoff beyond this work’s scope and identify activation translation and harder coordination games as future directions.Activation translation would interpret activations through the beliefs they induce as messages in listening agents.Source: §5 · p. 9

A. Qualitative Results

The qualitative material includes AC examples on biography, GSM8k, and MMLU tasks, alongside tables describing token-modification and chain-of-thought variants.

A. Qualitative Results

B.1. Modifying Activations of All Tokens

Modifying all token activations performs worse than modifying only the last token, while adding chain-of-thought generation for A yields no significant improvement over vanilla AC.

B.1. Modifying Activations of All Tokens

  • A’s last-token activation already attends to preceding tokens, incorporating information from the full sequence; the experiment also sums activations across all tokens.Source: §B.1 · p. 20
  • Applying f to all tokens decreases performance relative to applying it only to the last token.The authors expect the decline because the edited activations may be out of distribution for B; with summation, the last-token activation has roughly twice the original norm.Source: §B.1 · p. 20

B.2. Incorporating Chain-of-Thought Prompting

  • Generating a chain-of-thought response from A before passing its last-token activation to B yields no significant improvement over vanilla AC.Source: §B.2 · p. 20

B.3. Learning W In-Distribution

The appendix reports an in-distribution test of the learned linear layer and relates activation-space similarity to activation-communication gains.

B.3. Learning W In-Distribution

  • On GSM8k, learning W in-distribution significantly boosts performance, confirming the authors’ hypothesis; they could not run the test on MMLU because its public data were used for testing.The experiment trained W on the GSM8k training set and evaluated it on the test set.Source: §B.3 · p. 21
  • The authors suggest AC(W) should improve over vanilla AC when W is trained on data covering many tasks and distributions.They propose coverage that includes sentences semantically similar to prompts across downstream datasets.Source: §B.3 · p. 21

B.4. Activation Space Similarity ∝AC Performance Gain

  • Activation-distribution similarity has a clear positive correlation with AC performance gain; the analysis compares six model pairs using activations from 3,072 C4 sentences.The authors interpret greater alignment between activation spaces as making the grafted embedding more semantically meaningful and useful.Source: §B.4 · p. 21

B.5. Communicating Activations Between Identical Models

The authors extend activation communication to identical model instances and report comparisons with natural-language debate across reasoning benchmarks.

B.5. Communicating Activations Between Identical Models

  • For identical models, AC grafts a sampled completion’s last-token activation from A into B, retaining compute savings over NLD.The authors describe this as an extension of AC, which otherwise supports distinct models.Source: §B.5 · p. 21
  • Although communication between multiple model instances does not always improve over the single model itself, AC matches or outperforms NLD on five of the seven datasets.Source: §B.5 · p. 21
  • The authors connect this result to the intuition that sampled completions from identical instances offer diverse reasoning paths that can be recombined.They say the same intuition holds when responses are passed through AC rather than natural-language messages.Source: §B.5 · pp. 21–22

B.6. Additional Rounds of Natural Language Debate

The authors find that activation communication remains competitive with natural-language debate when debate uses additional rounds.

B.6. Additional Rounds of Natural Language Debate

  • For five of seven reasoning benchmarks, AC beats NLD even with three to four rounds while using substantially less compute.Section 4.2 fixed NLD to two agents and two rounds; these additional experiments test more rounds.Source: §B.6 · p. 22

B.7. Full MMLU Benchmark Results

The full MMLU comparison reports AC matching or outperforming NLD on most datasets, with substantially less compute.

B.7. Full MMLU Benchmark Results

  • AC matches or outperforms NLD on 48 of 57 full-MMLU datasets, using substantially less compute.The table presents the comparison across the full MMLU benchmark.Source: §B.7 · p. 22, Table 10 · p. 23
Loading 2501.14082v2…