PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systems

Haibo Jin, Peng Kuang, Ye Yu, Xiaopeng Yuan, Haohan Wang

arXiv:2602.03695v2cs.MAcs.AIcs.CL

TL;DR

Task-specific MAS architectures and natural-language communication present challenges for reuse and stability in long interactions. Agent Primitives provide reusable computation blocks with KV-cache exchange and automatic composition; across benchmarks, they improve accuracy over single-agent baselines.

  • Problem

    Existing MAS often rely on task-specific roles and interaction patterns, while natural-language communication can degrade in long-context interactions.

    Source: §1 · pp. 1–2

  • Method

    Agent Primitives package recurring MAS computations into reusable blocks, use KV-cache communication, and are selected and composed by an Organizer guided by a Knowledge Pool.

    Source: §1 · pp. 1–2

  • Results

    Across eight benchmarks and five open-source LLM backbones, primitives-based MAS improve average accuracy by 12.0–16.5% over single-agent baselines.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The experiments support Agent Primitives as a scalable, task-agnostic foundation for building MAS.

    Source: §6 · p. 9

    Experiments restrict agents to the same LLM to ensure valid input-output alignment under the KV-cache assumption.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

While existing multi-agent systems (MAS) can handle complex problems by enabling collaboration among multiple agents, they are often highly task-specific, relying on manually crafted agent roles and interaction prompts, which leads to increased architectural complexity and limited reusability across tasks. Moreover, most MAS communicate primarily through natural language, making them vulnerable to error accumulation and instability in long-context, multi-stage interactions within internal agent histories. In this work, we propose \textbf{Agent Primitives}, a set of reusable latent building blocks for LLM-based MAS. Inspired by neural network design, where complex models are built from reusable components, we observe that many existing MAS architectures can be decomposed into a small number of recurring internal computation patterns. Based on this observation, we instantiate three primitives: Review, Voting and Selection, and Planning and Execution. All primitives communicate internally via key-value (KV) cache, which improves both robustness and efficiency by mitigating information degradation across multi-stage interactions. To enable automatic system construction, an Organizer agent selects and composes primitives for each query, guided by a lightweight knowledge pool of previously successful configurations, forming a primitive-based MAS. Experiments show that primitives-based MAS improve average accuracy by 12.0-16.5\% over single-agent baselines, reduce token usage and inference latency by approximately 3$\times$-4$\times$ compared to text-based MAS, while incurring only 1.3$\times$-1.6$\times$ overhead relative to single-agent inference and providing more stable performance across model backbones.

1. Introduction

Multi-agent systems coordinate agents to tackle complex problems, but task-specific designs and natural-language communication can constrain reuse and stability. Agent Primitives package recurring computations into reusable building blocks, with KV-cache communication and automatic composition; experiments report higher accuracy and lower costs than baselines.

1. Introduction

  • Existing MAS use manually designed roles and natural-language protocols; task-specific architectures raise design overhead and limit reuse, while long histories destabilize communication.Natural-language communication accuracy degrades in long-context settings and is sensitive to accumulated communication noise.Source: §1 · pp. 1–2
  • Agent Primitives abstract recurring MAS computations into reusable building blocks that expose the same external interface as a standard LLM agent.The paper instantiates Review, Voting and Selection, and Planning and Execution primitives for plug-and-play composition.Source: §1 · pp. 1–2
  • An LLM-based Organizer selects and composes primitives for each query, guided by a Knowledge Pool of previously successful configurations.Primitives exchange information through KV cache to reduce information degradation across multi-stage interactions without repeated text decoding.Source: §1 · p. 2
  • 12.0–16.5% higher average accuracy over single-agent baselines, with 3×–4× lower token usage and inference latency than text-based MAS.The approach incurs 1.3×–1.6× overhead relative to single-agent inference and is more stable across model backbones.Source: §1 · p. 2

2. Related Work

Related work ranges from manually assigned agent roles and debate to dynamically constructed systems and latent communication. Agent Primitives distinguish themselves by making recurring internal computation patterns, rather than agents, the reusable unit.

LLM-based Multi-Agent Systems

  • Early MAS used predefined agent teams, then added iterative collaboration and debate to refine reasoning and reach consensus (Hong et al., 2023; Qian et al., 2024; Chen et al., 2023; Du et al., 2023; Liang et al., 2024).Source: §2 · p. 2
  • Dynamic approaches select agents, refine collaboration structures, optimize interaction graphs, or generate task-specific systems (Liu et al., 2024; Zhuge et al., 2024; Qian et al.; Hu et al., 2024; Zhang et al., 2024a; Ye et al., 2025).Source: §2 · p. 2

Latent Communication in MAS

3. Preliminary: Overcoming Communication Bottlenecks with KV Cache

The preliminary analysis tests natural-language communication under long histories and noise, then describes KV-cache exchange between agents and its alignment and positional assumptions.

3.1. Background

  • Agent A’s processed sequence yields a KV cache that is concatenated with agent B’s fixed system-prompt cache; B decodes from the combined cache without A’s text.Source: §3.1 · p. 3
  • KV-cache alignment is assumed distributionally equivalent to token-sequence conditioning only when agents share model parameters and positional encoding; RoPE keys are re-indexed by B’s prompt length.Source: §3.1 · p. 3

3.2. Communication Bottleneck via Natural Language

  • At midpoint injection, Injection Compliance is 15.6% for natural-language communication and 73.3% for KV-cache communication.Accuracy remains relatively stable across injection positions, which the authors attribute to communication failure rather than increased task difficulty.Source: §3.2 · p. 4, Table 1 · p. 4
  • Under communication noise, natural-language accuracy degrades rapidly, while KV-cache communication remains robust at moderate noise and declines only under extreme contamination.Source: §3.2 · p. 4, Table 2 · p. 4

4. Agent Primitives

Agent Primitives package recurring multi-agent patterns as reusable latent operators, coordinated through KV caches and composed into systems by an Organizer.

4.1. Overview

  • Agent Primitives capture recurring multi-agent patterns as reusable, plug-and-play operators with a standard LLM-agent interface and latent KV-cache coordination.They are presented as an alternative to manually designed task-specific roles and interaction protocols.Source: §4.1 · p. 4, §4.2 · pp. 4–5

4.2. Primitive Design

  • The Review Primitive loops a Solver and Critic through latent feedback, with intermediate latent states governing adaptive refinement depth.The Critic's corrective feedback is fed back to the Solver for subsequent computation.Source: §4.2 · p. 5
  • Voting and Selection runs parallel Solvers on the same prompt, then selects over their latent candidates to produce an aggregated state and final output.The Selector operates in latent space rather than using text-level majority voting.Source: §4.2 · p. 5
  • Planning and Execution separates a Planner's latent task decomposition from an Executor's subsequent task-specific reasoning.The Executor conditions its computation on the Planner's representation of intermediate steps or subgoals.Source: §4.2 · p. 5

4.3. Primitive-based Multi-Agent System

  • An Organizer maps each query to a composition plan specifying which primitives to instantiate and their execution structure.The resulting composition is executed as a complete MAS configuration.Source: §4.3 · pp. 5–6
  • The Organizer retrieves prior query–system pairs from a Knowledge Pool of 45 MAS structures, abstracts them into primitive compositions, and executes the selected structure through a standard MAS interface.The pool provides structural guidance rather than directly reusing full agent-level designs.Source: §4.3 · p. 6

5. Experiments

Across benchmarks and model backbones, primitives-based MAS outperform single-agent baselines and existing MAS methods, with gains across task categories. Ablations examine composition, Organizer selection, the knowledge pool, and KV-cache RoPE.

5.1. Experimental Setup

  • Evaluation spans eight math, code-generation, and Q&A benchmarks on five backbones, with comparisons to single agents, TextMAS, sequential LatentMAS, and individual primitives.Systems are evaluated within the same model architecture; single-primitive baselines and TextMAS use four agents.Source: §5.1 · p. 6

5.2. Performance of Agent Primitives Across Tasks

  • Composing primitives improves performance by an additional 3.5-7.0% over the strongest individual primitive, although no single primitive dominates across tasks.Source: §5.2 · p. 6
  • Primitives-based MAS consistently outperform single-agent baselines across model families and task categories, with math gains up to 26.7% on smaller models and 6.7%-14.4% on larger backbones.The table reports accuracy across math, code-generation, and Q&A benchmarks for three models.Source: §5.2 · p. 6, Table 3 · p. 7

5.4. Efficiency Analysis

  • Against TextMAS, Agent Primitives use substantially fewer tokens and less latency while achieving higher accuracy, with only 1.3×-1.6× overhead over single-agent inference.The paper attributes this efficiency–accuracy balance to avoiding excessive natural-language interaction and describes the overhead as moderate.Source: §5.4 · pp. 7–8

5.3. Comparison with Existing MAS Methods

  • Primitives-based MAS beats existing MAS methods across all benchmarks and reaches 53.2% accuracy on GPQA, versus prior methods’ 33.6-40.2%.It also improves over MAS-GPT by 3.7% on MATH and 3.4% on HumanEval+.Source: §5.3 · p. 7, Table 4 · p. 8

5.5. Ablation Study

  • Random structure selection trails an LLM Organizer by about 5-7% on Qwen3-8B and up to 12.4% on DeepSeek-R1-Distill-LLaMA-70B; different LLM Organizers differ by 0-2%.Source: Table 5 · p. 8, §5.5 · p. 8
  • Removing the knowledge pool lowers Qwen3-8B accuracy about 10% on AIME25 and 5-6% on HumanEval+ and MedQA; the LLaMA-based model drops roughly 6-12% by task.Source: Table 6 · p. 8, §5.5 · p. 8
  • Disabling RoPE drops LLaMA-based accuracy from 56.7% to 26.7% on AIME25, 85.3% to 31.1% on HumanEval+, and 81.9% to 36.6% on MedQA.On Qwen3-8B, AIME25 falls from 73.3% to 60.0%, and MedQA from 76.7% to 74.0%.Source: Table 7 · p. 8, §5.5 · p. 8

6. Conclusion

The conclusion presents Agent Primitives as reusable latent building blocks for MAS. Across eight benchmarks and five open-source LLMs, the approach improves accuracy and reduces tokens and inference latency relative to text-based MAS.

  • Agent Primitives abstract recurring MAS computation patterns into reusable latent building blocks.Source: §6 · p. 9
  • The approach uses KV-cache communication to reduce architectural complexity and alleviate communication degradation in long-context settings.Source: §6 · p. 9
  • Across eight benchmarks and five open-source LLMs, primitives-based MAS improve accuracy and reduce tokens and inference latency compared with text-based MAS.Source: §6 · p. 9

Impact Statements

The impact statement presents Agent Primitives as modular, task-agnostic building blocks for systematic MAS design. The accompanying prompts specify reusable agent roles and an Organizer that selects and composes primitives rather than solving queries.

Impact Statements

  • Agent Primitives offer reusable, task-agnostic components intended to reduce engineering overhead and support systematic MAS design.Source: Impact Statements · p. 9

A. System Prompt for Each Agent Primitive

  • Primitive prompts define role boundaries and interaction assumptions without encoding task-specific content or explicit problem-solving strategies.All primitives share a standard LLM-agent interface for plug-and-play composition and substitution.Source: A. System Prompt for Each Agent… · p. 12

A.1. Review Primitive

  • The Review Primitive iteratively refines a solution: Solvers generate or revise intermediate states, while Critics identify errors and provide targeted feedback.Source: §A.1 · p. 12

A.2. Voting and Selection Primitive

  • The Voting and Selection Primitive has independent Solvers produce candidates and a Selector evaluate and select or aggregate them into a final result.Source: §A.2 · p. 12

A.3. Planning and Execution Primitive

  • The Planning and Execution Primitive separates task decomposition by a Planner from execution by agents following its plan.Source: §A.3 · p. 13

B. Prompt for Organizer

  • The Organizer constructs a system by selecting and composing primitives for a query, optionally using prior system designs as structural guidance; it does not solve the task.Source: B. Prompt for Organizer · p. 13

C. Addition Experiments of Agent Primitives Across Tasks

Results on Qwen3-4B and Qwen3-14B follow the trends observed on larger models. Primitive composition also yields additional gains over the strongest individual primitive.

C. Addition Experiments of Agent Primitives Across Tasks

  • No single primitive dominates across tasks; composing primitives yields additional gains over the strongest individual primitive across model scales.Source: C. Addition Experiments of Agent… · p. 14
  • Across smaller and medium-scale backbones, primitives-based MAS outperform single-agent baselines and existing MAS methods on most tasks, with gains stable across task categories.The table covers math, code-generation, and Q&A benchmarks on Qwen3-4B and Qwen3-8B.Source: Table 8 · p. 14, C. Addition Experiments of Agent… · p. 14

D.1. Generalization to Out-of-Pool Tasks

On ARC-Challenge, which has no entries in the 45-entry Knowledge Pool, the Organizer generalizes beyond the pool’s recorded tasks. The pool adds a further average gain despite having no ARC-specific entries.

D.1. Generalization to Out-of-Pool Tasks

  • ARC-Challenge tests generalization to a benchmark absent from the 45-entry Knowledge Pool.Source: §D.1 · p. 14
  • The Organizer beats the single-agent baseline by +2.5% and +2.3% and random structure selection by +1.9% and +1.1% on Qwen3-4B and Qwen3-8B, respectively.Table 9 reports accuracy on ARC-Challenge with no pool entries.Source: Table 9 · p. 15, §D.1 · p. 15
  • The Knowledge Pool adds a further +1.2% average gain despite having no ARC-specific entries, suggesting that retrieved structural patterns transfer to novel task types.Source: §D.1 · p. 15

D.2. Disentangling KV-Cache from Primitive Abstraction

Ablations show that primitive abstraction and KV-cache communication each contribute to performance, while Organizer sensitivity and composition analyses examine how the framework selects structures across tasks.

D.2. Disentangling KV-Cache from Primitive Abstraction

  • KV-cache communication improves accuracy across all three primitives by +10.0%, +10.0%, and +13.4%, while each primitive also beats the single-agent baseline without the Organizer or Knowledge Pool.The comparison uses AIME25 with Qwen3-8B; Table 10 covers text-based versus KV-cache communication.Source: Table 10 · p. 15, §D.2 · p. 15

D.3. Organizer Model Sensitivity

  • Closed-source Organizers achieve comparable performance, while Qwen3-32B shows a moderate drop; the framework remains competitive with Llama-3-70B-Instruct in the iso-model setting.The authors conclude that effectiveness does not solely rely on a proprietary Organizer.Source: §D.3 · p. 15, Table 11 · p. 15

D.4. Primitive Composition Statistics

  • Selected compositions vary by task: math favors Voting and Planning, code invokes Planning more often, and QA relies more on Review.The analysis covers 100 examples from each task category; Table 12 reports primitive selection percentages and average primitives per query.Source: §D.4 · pp. 15–16, Table 12 · p. 16

E. Efficiency Analysis Across Model Backbones

Across model backbones, Agent Primitives balance accuracy gains with moderate computational cost. They use fewer tokens and run faster than text-based MAS, while LatentMAS is more efficient but less stable across models.

E. Efficiency Analysis Across Model Backbones

  • Agent Primitives reduce token usage and inference latency versus text-based MAS while keeping latency within 1.3×–1.6× of single-agent inference.The paper characterizes the resulting balance as strong, consistent accuracy gains with moderate and predictable computational cost.Source: E. Efficiency Analysis Across Model… · p. 16
  • Agent Primitives typically reduce token usage by 30%–40% on smaller and medium-sized models, while TextMAS often consumes more than 2×–4× the single-agent baseline.Source: E. Efficiency Analysis Across Model… · p. 16
  • LatentMAS generally uses the fewest tokens but exhibits unstable performance across backbones; Agent Primitives trade modestly more tokens for improved stability and accuracy.Source: E. Efficiency Analysis Across Model… · p. 16
  • TextMAS often increases inference time by 4×–6×, whereas LatentMAS is typically faster because chunked latent reasoning reduces decoding steps.Source: E. Efficiency Analysis Across Model… · p. 16

F. Efficiency Analysis Against Existing MAS Methods

Against existing MAS methods, Agent Primitives use fewer tokens and achieve favorable cost-normalized accuracy. The Organizer’s token cost is included, and the Llama-3-70B setting leads on cost-normalized accuracy across benchmarks.

F. Efficiency Analysis Against Existing MAS Methods

F.1. Token Usage Comparison

  • Agent Primitives use fewer tokens than all MAS baselines across benchmarks, typically 2–6× fewer than most methods, including Organizer inference costs.The Organizer contributes only 200–400 output tokens per query; Table 15 covers output-token usage with Llama-3-70B-Instruct.Source: §F.1 · p. 16, Table 15 · p. 19

F.2. Cost-Normalized Efficiency

  • The Llama-3-70B Organizer setting achieves the highest cost-normalized accuracy across benchmarks, and the GPT-5.2 Organizer setting remains more cost-efficient than all MAS baselines.Table 16 reports estimated dollar cost per query and accuracy per $0.01 spent.Source: Table 16 · p. 19, §F.2 · p. 19
Loading 2602.03695v2…