PaperlayerView exact revision on arXiv

Source-linked AI summary

Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL

Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, Hongxuan Lu, Tianrui Qin, Chenghao Zhu, Yi Yao, Shuying Fan, Xiaowan Li, Tiannan Wang, Pai Liu, King Zhu, He Zhu, Dingfeng Shi, Piaohong Wang, Yeyi Guan, Xiangru Tang, Minghao Liu, Yuchen Eleanor Jiang, Jian Yang, Jiaheng Liu, Ge Zhang, Wangchunshu Zhou

arXiv:2508.13167v1cs.AIcs.CL

TL;DR

Existing multi-agent systems achieve complex problem-solving but depend on costly, manually engineered workflows and do not support data-centric learning. The paper introduces Chain-of-Agents with multi-agent distillation and agentic reinforcement learning, producing Agent Foundation Models that achieve state-of-the-art results across diverse agent benchmarks.

  • Problem

    Existing multi-agent systems rely on sophisticated prompt and workflow engineering, creating computational overhead, limited adaptability, and no data-centric learning pathway.

  • Method

    Chain-of-Agents enables one model to dynamically activate tool and role agents, while multi-agent distillation and agentic reinforcement learning train end-to-end agent models.

  • Results

    AFMs establish new state-of-the-art performance across nearly 20 diverse agent benchmarks, including web, code, and mathematical reasoning tasks.

  • Takeaways & Limitations

    The work provides an end-to-end agent-modeling paradigm and releases its research, model weights, training and evaluation code, and training data openly.

  • Takeaways & Limitations

    Specialized multi-agent architectures remain constrained by coordination overhead, state fragmentation, and limited cross-agent tool synergies as agent interactions grow.

Abstract

from arXiv · show

Recent advances in large language models (LLMs) and multi-agent systems have demonstrated remarkable capabilities in complex problem-solving tasks such as deep research, vibe coding, and mathematical reasoning. However, most existing multi-agent systems are built upon manual prompt/workflow engineering with sophisticated agent frameworks, making them computationally inefficient, less capable, and can not benefit from data-centric learning. In this work, we introduce Chain-of-Agents (CoA), a novel paradigm of LLM reasoning that enables native end-to-end complex problem-solving in the same way as a multi-agent system (i.e., multi-turn problem solving with multiple tools and multiple agents) within one model. In chain-of-agents problem-solving, the model dynamically activates different tool agents and role-playing agents to simulate multi-agent collaboration in an end-to-end fashion. To elicit end-to-end chain-of-agents problem-solving abilities in LLMs, we introduce a multi-agent distillation framework to distill state-of-the-art multi-agent systems into chain-of-agents trajectories for agentic supervised fine-tuning. We then use agentic reinforcement learning on verifiable agentic tasks to further improve the models' capabilities on chain-of-agents problem solving. We call the resulting models Agent Foundation Models (AFMs). Our empirical studies demonstrate that AFM establishes new state-of-the-art performance across diverse benchmarks in both web agent and code agent settings. We make the entire research, including the model weights, code for training and evaluation, and the training data, fully open-sourced, which offers a solid starting point for future research on agent models and agentic RL.

1 Introduction

Existing multi-agent systems achieve strong complex-task performance but rely on costly coordination and manual workflow design. The paper introduces Chain-of-Agents and trains Agent Foundation Models through multi-agent distillation and agentic reinforcement learning.

  • Motivation: Multi-agent systems support complex problem solving through collaboration among agents with diverse roles and tool sets, but incur computational and engineering limitations.The stated limitations include redundant inter-agent communication, substantial prompt and workflow reconfiguration, and limited data-centric learning.
  • Approach: Chain-of-Agents integrates multi-agent collaboration capabilities within a single model for native end-to-end problem solving.The model dynamically activates agents corresponding to different tools and roles to simulate collaboration.
  • Training: Multi-agent distillation converts successful trajectories from state-of-the-art multi-agent frameworks into CoA-compatible data for supervised fine-tuning.The framework then applies agentic reinforcement learning on verifiable agentic tasks to optimize the resulting models.
  • Results: AFMs establish new state-of-the-art performance across nearly 20 diverse agent benchmarks spanning web, code, and mathematical reasoning.Reported results include 55.3% on GAIA, 47.9% on LiveCodeBench v5, and a 59.8% solve rate on AIME2025.
  • Results: AFM reduces inference cost by 84.6% in token consumption compared with traditional multi-agent systems while achieving competitive performance.The paper releases model weights, training and evaluation code, and training data for future research.

2 Background

The background distinguishes ReAct, multi-agent systems, and tool-integrated reasoning as paradigms for structured decomposition, specialized capabilities, and external tool use. These paradigms differ in whether reasoning is organized through interleaved actions, communicating specialized agents, or a global tool-selection state.

  • ReAct: ReAct interleaves thought, action, and observation steps, with each thought conditioned on the preceding trajectory history.The history contains prior thoughts, actions, and observations used to determine the next action.
  • Multi-Agent Systems: A multi-agent system consists of specialized agents with individual internal states and policies over their respective action spaces.Agents communicate through messages carrying information from other agents.
  • Tool-Integrated Reasoning: Tool-integrated reasoning enables one agent to select among external tools using a global reasoning state and update that state from tool outcomes.The tool set contains multiple tools, and the policy selects a tool conditioned on the current global state.

3 Method

CoA models multi-agent collaboration within one unified model by dynamically activating role-playing and tool agents. Its tuning framework distills successful multi-agent trajectories, filters them for quality and complexity, and formats them for supervised training.

  • Chain-of-Agents Paradigm: CoA orchestrates specialized role-playing and tool agents through dynamic agent activation within a unified model.The framework includes thinking, reflection, verification, search, crawl, and code-generation agents.
  • Chain-of-Agents Paradigm: Unlike rigid TIR pipelines, CoA supports adaptive multi-agent workflows while preserving contextual continuity and reducing intra-agent communication overhead.The unified model can also be directly optimized with supervised training and reinforcement learning.
  • Multi-Agent Distillation: Multi-agent distillation records expert system interactions as structured Chain-of-Agents trajectories containing reasoning states, activated agents, observations, and outputs.OAgents is monitored to capture agent selection, pre-action reasoning states, and agent outputs for CoA-compatible training data.
  • Progressive Quality Filtering: Progressive quality filtering removes simplistic or invalid samples and prioritizes complex, reflective, and error-correcting trajectories for supervised fine-tuning.Filtering includes a minimum of 5 agent-tool interactions, correctness checks, reflection enrichment, and upsampling of successful iterative error correction.

4 Experiments

The experiments evaluate AFM on multi-hop question answering, web-agent, mathematical reasoning, and code-generation tasks using curated SFT and RL datasets. Across these settings, AFM achieves strong benchmark performance, while staged training and multi-agent distillation contribute measurable gains.

  • Experimental Setup: The experiments use separate MHQA and Web Agent datasets, combining synthetic and filtered real-world sources to cover varied task types and difficulty levels.The Web Agent data integrates generated agentic tasks with filtered NQ, TQ, and HotpotQA examples.
  • Experimental Setup: 16,433 high-quality trajectories support SFT, including 8,826 MHQA and 7,607 Web Agent trajectories with reasoning chains spanning 5–20 hops.The RL datasets contain 169,615 MHQA instances and 10,427 Web Agent instances.
  • Web-Agent Results: AFM reaches 55.3% on GAIA, 11.1% on BrowseComp, and 63.0% on WebWalker, establishing strong results on complex web tasks.On HLE, AFM achieves 18.0%, exceeding WebThinker-RL at 15.8% and WebDancer at 7.2%.
  • Web-Agent Results: AFM-SFT-32B scores 50.5% on GAIA, 61.5% on WebWalker, and 10.0% on BrowseComp, outperforming the compared SFT baselines.These comparisons use Qwen2.5-32B-Instruct as the backbone.
  • Mathematical Reasoning Results: AFM-RL-7B averages 64.3% across five mathematical benchmarks, while AFM-RL-32B averages 78.0%, 3.6% above ReTool-32B.At 32B, absolute improvements on AIME25 and OlympiadBench are 10.5% and 5.7%, respectively.
  • Training-Stage Analysis: SFT improves average accuracy by 22.0% at 7B and 23.4% at 32B, while RL adds 20.8% and 18%, respectively, over the SFT baseline.The reported interpretation is that SFT provides Chain-of-Agents capabilities and RL further strengthens them.

5 Analysis

The analysis evaluates AFM’s efficiency, zero-shot tool generalization, and test-time scaling across agentic benchmarks. AFM uses fewer tools and tokens, while Pass@3 substantially improves performance over base AFM.

  • Efficiency: AFM uses the fewest tool calls and lowest token consumption among the compared frameworks on sampled GAIA instances.Tool efficiency counts calls per successful task, while token efficiency measures prompt-engineering cost per successful task.
  • Zero-shot agent generalization: Code-agent models remain robust under strict tool-invocation formats, whereas web-agent performance degrades substantially on character-level precision requirements.The web-agent model often produces parser-invalid invocations for Python and visual-inspection tools.
  • Test-time scaling: AFM-Bo3 raises GAIA performance from 55.3 to 57.3 and HLE performance from 18.0 to 23.0 through best-of-three trajectory selection.AFM-Bo3 selects the optimal trajectory from three candidate answers.
  • Test-time scaling: AFM-Pass@3 reaches 69.9 on GAIA, a 14.6-point increase over AFM, and also reaches 78.7 on WebWalker and 33.2 on HLE.On GAIA, this gain exceeds the reported improvements of WebDancer-Pass@3 and WebSailor-Pass@3.
  • Test-time scaling: On GAIA, AFM’s Pass@3 score of 69.9 approaches SmolAgents-Pass@3 at 73.9, narrowing the Pass@1 gap from 11.4 points to 4.0 points.AFM scores 55.3 at Pass@1, compared with SmolAgents’ 66.7.

6 Related Work

Related work progresses from multi-agent coordination and chain-of-thought reasoning to tool-integrated reasoning. These approaches improve complex problem solving but face coordination, scalability, tool-selection, and multi-step reward challenges.

  • Multi-Agent Systems: Multi-agent systems distribute expertise across specialized agents but incur pairwise coordination overhead and fragmented state representations.The coordination cost grows with the number of agents and can limit deployment and cross-agent tool synergies.
  • Multi-Agent Systems: Multi-agent architectures can outperform monolithic models while still constraining emergent synergies through pipelined coordination.The passage attributes this constraint to distributed coordination and the absence of a global state representation.
  • Chain-of-Thought: Chain-of-thought reasoning decomposes complex problems into stepwise traces and is effective for closed-world tasks such as mathematical reasoning.Its limitations arise when queries require external information.
  • Tool-Integrated Reasoning: Tool-integrated reasoning extends reasoning with external tools through think-action-observation loops, including ReAct-style tool use.Search-R1 and WebThinker train models to invoke search at appropriate reasoning steps.
  • Tool-Integrated Reasoning: Existing tool-integrated approaches face computationally intractable tool selection, limited adaptability from static supervision, weak multi-tool coordination, and sparse multi-step rewards.These limitations restrict bidirectional tool dependencies in dynamic information-seeking scenarios.

7 Conclusion

The paper introduces Chain-of-Agents as a native end-to-end paradigm for multi-agent problem solving and trains Agent Foundation Models with distillation and agentic reinforcement learning. Experiments report state-of-the-art results across diverse agent benchmarks and open-source the research artifacts.

  • Conclusion: Chain-of-Agents supports end-to-end multi-agent problem solving within a single model.The paper positions this paradigm as a basis for native agent models.
  • Conclusion: Multi-agent distillation generates supervised training data, while agentic reinforcement learning optimizes the resulting agent models.The approach trains a series of Agent Foundation Models.
  • Conclusion: AFM significantly outperforms existing tool-integrated-reasoning methods across RAG-based agents, web agents, and code agents.The paper reports this result across various domains.
  • Conclusion: The authors open-source the code and data used for training and evaluation to facilitate research on agent models and agentic reinforcement learning.The conclusion describes open sourcing as support for future research.

8 Contributions

The contribution list identifies separate leadership and execution roles across web agents, code agents, data, and paper writing. The listed contributors cover training, reinforcement learning, evaluation, agent paradigms, data generation, and project leads.

  • Contributions: Web-agent work assigns SFT and RL to multiple contributors, with separate evaluation and lead roles.The listed web-agent roles include SFT, RL, evaluation, and leadership.
  • Contributions: Code-agent work includes RL, SFT, evaluation, and leadership responsibilities.Each responsibility is assigned to named contributors in the contribution list.
  • Contributions: Data contributions span agent-paradigm development, data generation, and data leadership.The contribution list distinguishes these three data-related roles.
  • Contributions: Paper-writing responsibilities are shared among five listed contributors.The contribution list names the contributors responsible for paper writing.

A.1 Web Agent

The web agent uses separate search and page-crawling tool agents to retrieve and summarize online information. Web searches use Serpapi with model-generated queries, while crawled pages are summarized before being concatenated.

  • The web agent comprises web-search and crawl-page tool agents.
  • The web-search agent accesses Google through Serpapi using model-generated queries and retrieves the top 10 results.
  • Each search result includes a title, snippet, and additional result information.
  • The crawl-page agent uses model-generated URLs and Jina to retrieve page information for summarization.
  • Qwen2.5-72B-instruct summarizes each crawled page, and the resulting summaries are concatenated.

A.2 Code Agent

The code sandbox uses nsjail to isolate Python execution and restrict filesystem access. Its container compatibility supports migration across training and testing environments.

  • The code sandbox uses nsjail to create isolated execution environments for Python code.
  • Namespace isolation strengthens filesystem security by mitigating unauthorized access to host resources.
  • Nsjail is compatible with containerized environments such as Docker, enabling migration across training and testing setups.
  • The sandbox supports fine-grained resource controls.

B Dynamics Analysis during RL Training of Code Agent

The RL-training analysis tracks reward, response length, and AIME2025 accuracy over time. The reported trends show increasing reward and response length alongside steady accuracy improvement, with context expanded from 16k to 32k at step 40.

  • Figure 7 tracks training reward, average response length, and AIME2025 avg@16 during training.
  • AIME25 accuracy steadily improves during RL training, according to Figure 7c.
  • The model’s context length expands from 16k to 32k at training step 40 to improve stability.
  • Mean critic rewards consistently trend upward during RL training.The authors interpret this as evidence of progressive performance improvement and DAPO training stability.
  • Average response length consistently increases across training phases.The authors associate this trend with more complex coding strategies and additional tool interactions.

C Case Study

The case study traces two complex research tasks: one connecting a NASA APOD to Chicago architecture, and another identifying an anonymous blog post and related DVD Talk reviewer. The trajectories show both iterative correction of search strategies and successful extraction of final answers.

  • Task decomposition: The first task required identifying a NASA APOD city, its namesake, a related Chicago landmark, and the architectural firm that designed it.The planned chain proceeded from the APOD and Marquette to Jacques Marquette, the Marquette Building, and its architectural firm.
  • Task decomposition: The agent identified the APOD city as Marquette, Michigan, and connected it to Jacques Marquette and the Chicago Marquette Building.The trace records this connection while noting that the landmark relationship required verification.
  • Reflection and correction: The agent recognized that an assumed direct connection between Marquette’s namesake and a Chicago landmark might be incorrect and proposed verifying or revising the plan.This reflection illustrates explicit adaptation after detecting a possible information conflict.
  • Task resolution: The architectural firm was identified as Holabird & Roche, yielding “Holabird” as the requested first name.The trace presents Holabird & Roche as the firm and the suggested answer as Holabird.
  • Task decomposition: A second task sought an anonymous blogger’s first post and the DVD Talk reviewer of the referenced movie under narrow date and content constraints.The task constrained the blog to 2005–2013, the movie to U.S. releases from 2002–2010, and the review to February 2000–2008.
  • Task resolution: The trace reports October 23, 2011, for the first blog post and Scott Weinberg as the DVD Talk reviewer.The suggested answer supplies both requested outputs and is described as satisfying the task requirements.

D Prompts

The prompts specify structured procedures for web-page summarization, mathematical problem solving, code generation, and answer judging. Together, they require staged reasoning, tool or code execution where appropriate, reflection, and concise final answers.

  • D.1 Summary Prompt: The crawler prompt instructs summarizing all query-relevant webpage content, including text, tables, lists, code blocks, and links.It requests three concise points tied to the search query and supplied clues.
  • D.2 Mathematical problem-solving Prompt: The mathematical-solving prompt requires think, plan, code, observation, reflection, and answer functions in a structured sequence.Plans decompose the task, while code supports numerical calculation, symbolic computation, verification, edge-case testing, or visualization.
  • D.2 Mathematical problem-solving Prompt: Mathematical reflection must verify computational results, assess soundness, identify special cases, and determine whether further checking or alternative approaches are needed.The reflection function must end with </reflection>.
  • D.2 Mathematical problem-solving Prompt: The mathematical prompt restricts outputs to designated functions and requires the sequence plan, repeated think-code-observation-reflection cycles, then think and answer.Special tokens must not appear in free text, including inside think functions.
  • D.3 Code generation Prompt: The code-generation prompt requires complete Python snippets with necessary imports, test inputs or expected outputs, readable debugging output, and no file operations.It supports two code formats: Python-only snippets or Python plus shell input when using sys.stdin or input().
  • D.3 Code generation Prompt: Code-generation reflection compares observations with expected results, checks bugs and edge cases, and considers performance, readability, and maintainability.Its output guides the next think function and must end with </reflection>.
  • D.4 LLM-as-Judge Prompt: The answer-judging prompt asks whether predicted and labeled answers are semantically equivalent despite ordering, formatting, or measurement-unit differences.The required output contains a rationale and a judgement restricted to correct or incorrect.

MULTI TOOL PROMPT

The multi-tool prompt defines a constrained agent procedure: reason before planning or acting, use web search, page crawling, or code as needed, reflect on results, and provide a definite concise answer.

  • MULTI TOOL PROMPT: The agent may use nine functions: think, plan, tool, observation, reflection, suggested_answer, double_check, code, and answer.Each function contributes to constructing the reasoning path toward the final response.
  • MULTI TOOL PROMPT: Before planning, searching, crawling, coding, reflecting, double-checking, or suggesting an answer, the agent must first use think.The plan breaks the question into detailed subquestions for tool execution.
  • MULTI TOOL PROMPT: The tool function supports web_search, crawl_page, and executable Python code, with observations returning external tool or code-execution results.Web searches can be followed by crawling selected URLs for deeper information.
  • MULTI TOOL PROMPT: If information is irrelevant, the agent should search again, and poor reflection scores require replanning before continuing.Before the final answer, it should produce a suggested answer and double-check it; a double-check score below 3 triggers replanning.
  • MULTI TOOL PROMPT: Precise calculations, statistical analysis, and mathematical operations must be verified with the code function before the final answer.The prompt also requires concise, definite, accurate answers without ambiguity.
Loading 2508.13167v1…