Source-linked AI summary
Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline
Jiawei Xu, Arief Koesdwiady, Sisong Bei, Yan Han, Baixiang Huang, Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, Pan Li, Ying Ding
TL;DR
The paper asks whether homogeneous multi-agent workflows require multiple model instances, evaluates single-agent simulation across seven benchmarks, and introduces OneFlow for efficient workflow design. Single-agent execution reaches comparable homogeneous-workflow performance with lower cost and can match one automatically discovered heterogeneous workflow, although it cannot reproduce true heterogeneity across different models.
Problem
Most multi-agent systems are homogeneous, raising whether one agent can simulate their role-specialized workflows through multi-turn conversations.
Method
The study evaluates single-agent execution across seven benchmarks and introduces OneFlow to automatically design workflows optimized for single-agent execution.
Results
Single-agent execution achieves comparable performance to homogeneous workflows at lower cost and can match one automatically discovered heterogeneous workflow.
Takeaways & Limitations
Single-LLM execution is a strong, cost-efficient baseline for homogeneous multi-agent workflow research.
Takeaways & Limitations
Single-LLM methods cannot simulate truly heterogeneous workflows because different base models cannot share KV caches.
Abstract
from arXiv · showhide
Recent advances in LLM-based multi-agent systems (MAS) show that workflows composed of multiple LLM agents with distinct roles, tools, and communication patterns can outperform single-LLM baselines on complex tasks. However, most frameworks are homogeneous, where all agents share the same base LLM and differ only in prompts, tools, and positions in the workflow. This raises the question of whether such workflows can be simulated by a single agent through multi-turn conversations. We investigate this across seven benchmarks spanning coding, mathematics, general question answering, domain-specific reasoning, and real-world planning and tool use. Our results show that a single agent can reach the performance of homogeneous workflows with an efficiency advantage from KV cache reuse, and can even match the performance of an automatically optimized heterogeneous workflow. Building on this finding, we propose \textbf{OneFlow}, an algorithm that automatically tailors workflows for single-agent execution, reducing inference costs compared to existing automatic multi-agent design frameworks without trading off accuracy. These results position the single-LLM implementation of multi-agent workflows as a strong baseline for MAS research. We also note that single-LLM methods cannot capture heterogeneous workflows due to the lack of KV cache sharing across different LLMs, highlighting future opportunities in developing \textit{truly} heterogeneous multi-agent systems.
RETHINKING THE VALUE OF MULTI-AGENT WORKFLOW: A STRONG SINGLE AGENT BASELINE
The paper is authored by Jiawei Xu, Arief Koesdwiady, Sisong Bei, Yan Han, Baixiang Huang, Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, and Pan Li.
- Jiawei Xu, Arief Koesdwiady, Sisong Bei, Yan Han, and Baixiang Huang are listed among the authors.
- Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, and Pan Li are also listed among the authors.
1 INTRODUCTION
The paper asks whether homogeneous multi-agent workflows can be simulated by one agent, investigates this across seven benchmarks, and introduces OneFlow for efficient single-agent execution.
- Most current multi-agent systems are homogeneous, with agents sharing a base LLM but differing in prompts, tools, and workflow positions.
- A single agent can role-play homogeneous agents sequentially while reusing a shared KV cache across interactions.
- The study tests single-agent simulation across seven benchmarks covering coding, mathematics, question answering, domain reasoning, planning, and tool use.
- OneFlow automatically designs streamlined workflows with longer system prompts and fewer agents for single-agent execution.
- The single-agent baseline can match one automatically discovered heterogeneous workflow, while true heterogeneity remains an open research direction.
2 PRELIMINARY
The paper formalizes LLM workflows as agent graphs, distinguishes homogeneous from heterogeneous model assignments, and illustrates structured workflow execution through a shopping-query example.
- The shopping-query example combines session context, product-relationship analysis, and recommendation generation through specialized processing stages.
- An LLM-based multi-agent workflow is defined as a directed graph G = (N, E).
- Homogeneous vs. Heterogeneous Workflows: Homogeneous workflows use one base LLM, whereas heterogeneous workflows use multiple base LLMs.
- Design Complexity of heterogeneous workflows: Heterogeneous workflow design expands the search space across models, prompts, tools, and routing logic.
- KV Cache: Shared KV caching enables efficient single-agent simulation when workflow steps call the same base model.
3 METHODOLOGY
This methodology formalizes when a single LLM can simulate a homogeneous multi-agent workflow, reusing shared context and KV cache while preserving behavior under explicit conditions. It then describes OneFlow’s workflow-search procedure for optimizing performance and cost.
- Single-agent simulation: A single-LLM simulator sequentially adopts each workflow agent’s prompt, preserves the visible history, executes required tools, and follows the original routing logic.The simulator maintains one conversation state and advances through agent turns until termination.
- Simulation guarantee: Under deterministic tools, history-based routing, and deterministic or shared-randomness decoding, the simulator induces the same transcript distribution as separate homogeneous agents.Both procedures query the same base model at the same sequence of states.
- Cost with KV cache: Shared KV-cache reuse makes single-agent execution asymptotically no worse and often cheaper when agent contexts substantially overlap.Separate instances re-encode overlapping prefixes, whereas the simulator pays for incremental growth between turns.
- OneFlow optimization: Figure 2 depicts OneFlow’s dual meta-LLMs and MCTS across successive workflow-design rounds, including the generation of workflow 6.The left panel covers the first five rounds; the right panel shows workflow 6 generation.
- OneFlow optimization: OneFlow initializes a single-agent input-output workflow, evaluates candidates, and searches workflow modifications with MCTS using a designer, reviewer, validation, and backpropagation.The search score balances performance and cost through αP − βC.
4 EXPERIMENTS
The experiments evaluate automated and single-agent workflow variants across diverse benchmarks, measuring both task performance and inference cost. Results show that single-agent execution preserves homogeneous-workflow performance while reducing cost, including on open-weight and tool-intensive planning tasks.
- Evaluation setup: The evaluation spans coding, mathematics, question answering, domain-specific reasoning, and real-world planning and tool use.The benchmark suite includes MBPP, HumanEval, GSM8K, MATH, HotpotQA, DROP, Shopping-MMLU, and TravelPlanner.
- Evaluation setup: Performance is reported with task-specific metrics, while computational cost is measured from input and output token expenditure across three independent trials.Metrics include pass@1, F1, solve rate, accuracy, and TravelPlanner task success rate.
- Performance: Single-agent execution matches or slightly exceeds the corresponding multi-agent performance for AFlow- and OneFlow-designed homogeneous workflows.Automatically designed workflows and their single-agent executions also substantially outperform manual baselines.
- Cost: Single-agent execution substantially reduces inference cost through KV-cache reuse while maintaining comparable performance.For OneFlow, performance can even increase because shared context is retained across agent turns.
- Heterogeneous workflows: The heterogeneous pilot reaches an F1-score of 85.5 on DROP, above GPT-4o mini AFlow at 83.1 but below Claude 3.5 Haiku OneFlow at 87.5.The paper cautions that the heterogeneous workflows were not perfectly optimized.
- Generalization: On TravelPlanner, a single LLM matches AFlow and OneFlow task success rates while incurring lower inference cost.The OneFlow-discovered workflow is especially compact and efficient.
5 RELATED WORK
Related work covers multi-agent task decomposition, automatic workflow design, and KV-cache-based single-LLM execution. This paper situates its contribution at the intersection of these lines by testing whether homogeneous workflows can be simulated through multi-turn conversations.
- Multi-Agent Workflows and Task Decomposition: Multi-agent systems decompose complex tasks through specialized roles, tools, and sequential, iterative, or deliberative communication patterns.Examples include role-playing, tool-augmented collaboration, ReAct, and debate.
- Multi-Agent Workflows and Task Decomposition: Most existing multi-agent frameworks are homogeneous, with agents sharing a base model while differing in prompts, tools, and workflow positions.The paper contrasts this common setting with heterogeneous systems using different base models.
- Research focus: The paper examines when single-agent multi-turn simulation faithfully reproduces homogeneous workflows and where heterogeneous workflows may add benefits or costs.The question connects model homogeneity with the feasibility of shared-cache execution.
- Automatic Design of Multi-Agent Workflows: Automatic workflow design research optimizes prompts, communication topologies, or complete workflow architectures.Representative methods include DSPy and TextGrad for prompts, GPTSwarm and related methods for topology, and ADAS, AFlow, and MaAS for workflow search.
- KV Cache and Single-LLM Execution: KV caching reuses previously computed key/value states, enabling a single LLM to maintain one shared conversation across homogeneous agent turns.This avoids redundant re-encoding and can improve consistency while reducing execution cost.
6 CONCLUSION
The paper concludes that a single LLM can execute homogeneous multi-agent workflows through role-playing and shared KV caching, preserving performance at lower cost. OneFlow further discovers compact workflows, while true heterogeneity remains a distinct research direction.
- Conclusion: In homogeneous workflows, single-LLM execution can match or slightly exceed multi-agent performance at substantially lower cost.The mechanism combines multi-turn role-playing with shared KV-cache reuse.
- Conclusion: OneFlow discovers compact workflows that provide additional efficiency gains when executed by a single LLM.The conclusion presents workflow compactness and single-agent execution as complementary efficiency advantages.
- Scope and impact: The study reports no novel ethical or societal considerations beyond those already associated with large-scale language models.It uses existing models without training, fine-tuning model weights, or creating new LLMs.
- Reproducibility: The paper provides OneFlow pseudocode, explanations, visual illustrations, prompts, and executor and optimizer settings for reproducibility.These materials are included in the main methodology section and accompanying documentation.
A.1 THE USE OF LARGE LANGUAGE MODELS
The paper states that LLMs were used only for sentence-level proofreading, while the authors conducted the research ideation and writing.
- Use of large language models: LLMs were used solely for sentence-level proofreading.The authors state that all research ideation and paper writing were conducted entirely by the authors.
A.2 MONTE CARLO TREE SEARCH FOR WORKFLOW OPTIMIZATION
OneFlow uses Monte Carlo Tree Search to explore workflow configurations, iteratively selecting, expanding, evaluating, and backpropagating workflow performance, cost, and failures.
- MCTS treats each workflow configuration as a node in a search tree and systematically explores possible workflows.
- Selection: Selection retains the initial workflow and the top three tree workflows, then chooses one for expansion using exploration–exploitation weighting.
- Expansion: Expansion uses designer and critic meta-LLMs to produce a runnable improved workflow from the selected workflow and validation failures.
- Evaluation: Evaluation measures accuracy, token consumption, and failed samples, including the reasoning processes behind failures.
- Backpropagation: Backpropagation stores performance, cost, and failure outcomes in workflow nodes and continues optimization for up to 20 iterations.
A.3 DUAL META-LLMS FOR BALANCED PERFORMANCE AND COST OPTIMIZATION
OneFlow’s dual-meta-LLM framework balances workflow innovation with cost efficiency by pairing a creative designer with a critical reviewer.
- Two specialized meta-LLMs collaboratively design new workflows: a Creative Designer proposes modifications and a Critical Reviewer critiques and improves them.
- Creative Designer: The Creative Designer receives the current workflow, performance history, and error samples, then records the rationale for proposed changes in workflow-code comments.
- Creative Designer: The designer is additionally given detailed failure reasoning, helping it understand workflow errors before proposing modifications.
- Critical Reviewer: The Critical Reviewer examines the designer’s runnable workflow against existing candidates’ cost and performance, then proposes a revised workflow.
- Critical Reviewer: The reviewer’s output becomes the final workflow for the round.
A.4 RESULTS ON CLAUDE 3.5 HAIKU
On Claude 3.5 Haiku, OneFlow is evaluated on public and shopping-specific benchmarks, with reported performance, cost, and workflow-prompt design comparisons.
- Public benchmarks: OneFlow’s Claude 3.5 Haiku evaluation reports performance and cost on public coding, question-answering, and mathematics benchmarks, averaged over three runs.
- Public benchmarks: OneFlow achieves competitive or superior performance across most public benchmarks while substantially reducing inference cost.
- Shopping-MMLU: OneFlow achieves state-of-the-art performance on 9 out of 10 Shopping-MMLU tasks using accuracy as the evaluation metric.
- Shopping-MMLU: +4.8% improvement over AFlow is reported for Multilingual Query Understanding, with scores of 0.569 vs. 0.521.
- Prompt design: The workflow-design prompts instruct agents to optimize graphs and prompts while avoiding hardcoded answers, excessive changes, and invalid outputs.