PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline

Jiawei Xu, Arief Koesdwiady, Sisong Bei, Yan Han, Baixiang Huang, Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, Pan Li, Ying Ding

arXiv:2601.12307v1cs.MAcs.CLcs.LG

TL;DR

Because most multi-agent workflows use the same base LLM across roles, it is unclear whether a single agent can reproduce them through multi-turn conversations. Across seven benchmarks, this paper tests that possibility and finds comparable performance to homogeneous workflows at lower cost.

  • Problem

    The paper asks whether a single agent can simulate homogeneous multi-agent workflows through multi-turn conversations when their agents share the same base LLM.

    Source: §1 · p. 1

  • Method

    OneFlow automatically designs streamlined workflows tailored for single-agent execution to optimize performance and computational efficiency.

    Source: §1 · p. 2

  • Results

    Across seven benchmarks, a single agent using multi-turn conversations with KV cache achieved performance comparable to homogeneous multi-agent workflows while reducing cost.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Single-LLM execution is a strong, cost-efficient baseline for homogeneous workflows, while heterogeneous composition remains a complementary research direction.

    Source: §6 · p. 11

    Single-LLM methods cannot simulate truly heterogeneous workflows because KV caches cannot be shared across different models.

    Source: §1 · p. 2

Abstract

from arXiv · show

Recent advances in LLM-based multi-agent systems (MAS) show that workflows composed of multiple LLM agents with distinct roles, tools, and communication patterns can outperform single-LLM baselines on complex tasks. However, most frameworks are homogeneous, where all agents share the same base LLM and differ only in prompts, tools, and positions in the workflow. This raises the question of whether such workflows can be simulated by a single agent through multi-turn conversations. We investigate this across seven benchmarks spanning coding, mathematics, general question answering, domain-specific reasoning, and real-world planning and tool use. Our results show that a single agent can reach the performance of homogeneous workflows with an efficiency advantage from KV cache reuse, and can even match the performance of an automatically optimized heterogeneous workflow. Building on this finding, we propose \textbf{OneFlow}, an algorithm that automatically tailors workflows for single-agent execution, reducing inference costs compared to existing automatic multi-agent design frameworks without trading off accuracy. These results position the single-LLM implementation of multi-agent workflows as a strong baseline for MAS research. We also note that single-LLM methods cannot capture heterogeneous workflows due to the lack of KV cache sharing across different LLMs, highlighting future opportunities in developing \textit{truly} heterogeneous multi-agent systems.

1 INTRODUCTION

The paper argues that single-agent multi-turn execution can match homogeneous multi-agent workflows while reusing KV cache, and introduces OneFlow to optimize workflows for this setting. It also finds that a single-LLM baseline can match one automatically discovered heterogeneous workflow, but cannot simulate truly heterogeneous systems.

1 INTRODUCTION

  • Because homogeneous agents share a base LLM and differ by prompts, tools, and workflow positions, the paper asks whether one agent can role-play them sequentially.Sequential execution can reuse a shared KV cache, avoiding additional prefill cost and potentially improving consistency.Source: §1 · pp. 1–2
  • Across seven benchmarks, single-agent multi-turn execution with KV cache achieves performance comparable to homogeneous multi-agent setups while reducing cost.The benchmarks span coding, mathematics, general question answering, domain-specific reasoning, and real-world planning and tool use.Source: §1 · p. 2
  • OneFlow uses a dual-LLM designer to find streamlined workflows with longer agent prompts and fewer agents, preserving similar performance while reducing inference cost.It targets automatic workflow design for efficient single-agent execution.Source: §1 · p. 2
  • In a pilot study, a single-LLM baseline matches one automatically discovered heterogeneous workflow with less computational cost, but cannot simulate truly heterogeneous workflows.Different base models cannot share KV caches; developing effective heterogeneous workflows remains an open direction.Source: §1 · p. 2

2 PRELIMINARY

The section defines LLM-based multi-agent workflows as graph-structured systems of collaborating agents and distinguishes homogeneous workflows, whose shared base model permits KV-cache sharing, from heterogeneous workflows.

LLM-based Multi-Agent Workflows

  • Multi-agent workflows coordinate specialized LLM stages through executable Python control flows supporting sequential execution, branching, iterative refinement, or deliberation.The section illustrates this approach with session-based shopping-query recommendations.Source: §2 · p. 2

Formal Definition

  • Workflows are directed graphs of LLM agents whose communication edges encode orchestration and control flow.Agents are parameterized by a base LLM, system prompt, and available tools; Python-implemented edges can include routing conditions and message transformations.Source: §2 · pp. 2–3

Homogeneous vs. Heterogeneous Workflows

  • Homogeneous workflows use one shared base LLM, while heterogeneous workflows use multiple base LLMs across agents.Agents in homogeneous workflows differ in prompts, tools, and workflow positions; heterogeneous workflows can draw on different model capabilities.Source: §2 · p. 3

Design Complexity of heterogeneous workflows

  • A workflow’s model assignments expand its design space beyond prompts, tools, and routing, so automatic design systems often default to homogeneous configurations.Heterogeneous workflows offer flexibility to combine models with complementary strengths, but model selection is an additional empirical design decision.Source: §2 · p. 3

KV Cache

  • Shared KV caches avoid recomputing prior attention states and can be reused across homogeneous agents because their interactions have substantial contextual overlap.Without caching, attention states for previous tokens are recomputed during generation, leading to quadratic computational complexity.Source: §2 · pp. 3–4

3 METHODOLOGY

The methodology formalizes when a single LLM can simulate a homogeneous multi-agent workflow with the same transcript behavior and lower prefill cost, then specifies a simulator and OneFlow’s workflow-search procedure.

3.1 SINGLE AGENT CAN BE AS STRONG AS MULTI-AGENT FRAMEWORK.

  • Under deterministic tool effects, visible-history routing, and deterministic decoding or shared randomness, a single-LLM simulator produces the same transcript distribution as separate agents.Both procedures query the same model distribution at the same states, so induction over steps gives transcript equivalence.Source: §3.1 · p. 4
  • With contextual overlap, reusing one KV cache makes single-LLM execution asymptotically no worse and often cheaper than separate instances, which re-encode overlapping prefixes.The costs are equal only when agent contexts are disjoint.Source: §3.1 · p. 4

3.2 SINGLE AGENT IMPLEMENTATION OF MULTI-AGENT WORKFLOW.

  • The simulator maintains one chat history, follows the workflow’s routing logic, appends each agent’s prompt and tool results, and updates state until termination.Each step queries the same base model; deterministic summarization can compact turn windows while preserving routing decisions and Proposition 1’s applicability.Source: §3.2 · pp. 4–5

3.3 ALGORITHM: ONEFLOW

  • OneFlow searches workflows by maximizing a weighted performance-minus-cost score, beginning with a single-agent input-output workflow evaluated on validation data.The search space includes agent prompts, tools, and communication patterns; the initial workflow stores performance metrics and failure cases.Source: §3.3 · pp. 5–6
  • OneFlow uses iterative MCTS with a Creative Designer proposing performance-focused changes and a Critical Reviewer refining them for cost efficiency.Each iteration selects a workflow, generates and evaluates a proposal, and backpropagates its performance and cost comparison; experiments use 20 iterations.Source: Figure 2 · p. 5, §3.3 · p. 6

4 EXPERIMENTS

Across the experiments, single-agent executions of homogeneous workflows match or slightly exceed their multi-agent counterparts, and OneFlow’s single-agent setting typically outperforms homogeneous workflow baselines. They also reduce inference cost while maintaining performance across public benchmarks, open-weight models, and TravelPlanner.

4.1 EXPERIMENTAL SETUP

  • The evaluation spans seven benchmarks across coding, mathematics, question answering, domain-specific reasoning, and real-world planning and tool use.Metrics include pass@1, F1, solve rate, accuracy, and task success rate; computational cost is USD token expenditure per workflow, and results average three trials with standard deviations.Source: §4.1 · pp. 6–7

4.2.1 PERFORMANCE ON PUBLIC BENCHMARKS

  • AFlow- and OneFlow-designed workflows executed by one agent match or slightly exceed their multi-agent counterparts, while single-agent OneFlow typically leads homogeneous workflow baselines.Automatically designed workflows substantially outperform manual baselines.Source: Table 1 · p. 7, §4.2.1 · p. 8

4.2.2 COST ON PUBLIC BENCHMARKS

  • Single-agent execution substantially lowers inference cost at comparable performance, with KV-cache reuse enabling savings and slightly improved OneFlow performance.OneFlow is more cost-efficient than AFlow without single-agent execution; both workflows realize cost gains when executed by one LLM.Source: Table 2 · p. 8, §4.2.2 · p. 8

4.2.3 PILOT STUDY ON AUTOMATICALLY DESIGNED HETEROGENEOUS MULTI-AGENT WORKFLOWS

  • On DROP, the automatically designed heterogeneous workflow scored 85.5 F1, above GPT-4o mini-based AFlow’s 83.1 but below homogeneous Claude 3.5 Haiku OneFlow’s 87.5.The pilot workflows were not perfectly optimized; the paper notes that effective heterogeneous systems could route simple tasks to smaller models and complex tasks to stronger reasoning models.Source: Table 3 · p. 8, §4.2.3 · pp. 8–9

4.2.4 KV CACHE REUSE WITH OPEN-WEIGHT LLMS

  • On Qwen-3 8B HumanEval, single-agent AFlow and OneFlow maintain or improve pass@1 while latency and throughput remain stable despite added conversation history.Average input tokens increase by 967 for AFlow and 368 for OneFlow; longer multi-turn responses slightly offset efficiency gains seen with GPT-4o-mini.Source: Table 4 · p. 9, §4.2.4 · p. 9

4.2.5 ADDITIONAL EXPERIMENTS ON THE TOOL-USING BENCHMARK: TRAVELPLANNER

  • On TravelPlanner, single-agent AFlow and OneFlow match their multi-agent counterparts’ task success rates at lower inference cost, with OneFlow’s workflow especially compact and efficient.TravelPlanner tests context- and tool-intensive real-world planning.Source: §4.2.5 · p. 9

5 RELATED WORK

Related work covers multi-agent task decomposition, automatic workflow design, and KV-cache reuse. This paper positions single-LLM simulation as a competitive, cost-efficient baseline for homogeneous workflows while clarifying limits imposed by heterogeneity.

Multi-Agent Workflows and Task Decomposition

  • Multi-agent workflows decompose complex tasks across specialized roles and interactions, while most frameworks use homogeneous agents that share a base LLM and differ in prompts, tools, or routing.Heterogeneous systems use different base models, but many emphasize ensembling or discussion rather than end-to-end execution with explicit control flow; this paper studies single-agent simulation of homogeneous workflows.Source: §5 · pp. 9–10

Automatic Design of Multi-Agent Workflows

  • Most automatic workflow-design methods reduce manual construction effort but assume homogeneous executors, which this work complements with a cost-efficient single-LLM baseline.Prior directions include prompt, communication/topology, and workflow search or generation; heterogeneity is often included only tangentially.Source: §5 · p. 10

KV Cache and Single-LLM Execution

  • Shared KV caching lets a single-LLM execution reuse computed states across agent turns, whereas different base models in heterogeneous workflows cannot share KV states.This reuse avoids redundant re-encoding and can improve consistency, motivating single-LLM simulation as a strong baseline when behavior is preserved under mild conditions.Source: §5 · pp. 10–11

6 CONCLUSION

The paper finds that single-LLM execution can reproduce homogeneous multi-agent workflows with lower cost and that OneFlow can improve efficiency, while true heterogeneity remains beyond single-LLM simulation.

6 CONCLUSION

  • Single-LLM role-play matches or slightly exceeds homogeneous workflows at substantially lower cost; OneFlow adds efficiency, and a pilot matches AFlow-optimized heterogeneous workflows.Single-LLM simulation cannot realize true heterogeneity; the authors identify end-to-end single-agent training and principled heterogeneous composition as complementary research directions.Source: §6 · p. 11

ETHICS STATEMENT

  • The work evaluates existing LLMs without training, fine-tuning, or creating models, and reports no novel ethical or societal impacts beyond those associated with large language models broadly.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The paper provides OneFlow pseudocode and explanation, a visual illustration, exact prompts, and settings for the executor and optimization models.Source: REPRODUCIBILITY STATEMENT · p. 11

A APPENDIX · A.1 THE USE OF LARGE LANGUAGE MODELS

Large language models were used only for sentence-level proofreading; the authors conducted all research ideation and paper writing.

A.1 THE USE OF LARGE LANGUAGE MODELS

  • Large language models were used solely for sentence-level proofreading.Source: §A.1 · p. 14
  • The authors conducted all research ideation and paper writing.Source: §A.1 · p. 14

A.2 MONTE CARLO TREE SEARCH FOR WORKFLOW OPTIMIZATION

MCTS systematically explores workflow configurations as nodes in a search tree, iteratively selecting, expanding, evaluating, and updating candidate workflows.

Selection

  • Selection combines the initial workflow with the top three performers, then mixes uniform exploration and score-weighted exploitation to choose a workflow for expansion.The candidate set contains at most four workflows; better-scoring candidates have a higher selection probability.Source: §A.2 · p. 14

Expansion

  • During expansion, a designer meta-LLM proposes a workflow from the selected design and validation failures, then a critic reviews and improves it into runnable Python code.The critic assesses validity and efficiency and records changes and rationale in the workflow code.Source: §A.2 · p. 14

Evaluation

  • The proposed workflow is evaluated on the validation dataset for performance, cost, and failed samples, including the reasoning processes behind failures.Example metrics include accuracy and token consumption.Source: §A.2 · p. 14

Backpropagation

  • Backpropagation stores each workflow’s performance, cost, and failures at its node and records relative success or failure in its parent to discourage similar proposals.Optimization continues for up to 20 iterations.Source: §A.2 · p. 14

A.3 DUAL META-LLMS FOR BALANCED PERFORMANCE AND COST OPTIMIZATION

The section introduces two specialized meta-LLMs: a Creative Designer proposes performance-focused workflow changes, and a Critical Reviewer critiques and revises them using broader cost and performance information.

A.3 DUAL META-LLMS FOR BALANCED PERFORMANCE AND COST OPTIMIZATION

  • The two meta-LLMs collaboratively design workflows, with the reviewer critiquing the designer’s proposal against existing candidates’ cost and performance trade-offs.The framework’s right panel is shown in Figure 2.Source: §A.3 · pp. 14–15
  • The Creative Designer uses the current workflow, prior modifications, instructions, and failed-question reasoning logs to propose creative, commented changes; alone, it can achieve excellent performance at high inference cost.Its setup resembles AFlow, but adds rationale comments and more detailed execution error logs.Source: §A.3 · pp. 14–15
  • The Critical Reviewer revises the designer’s runnable workflow using candidate workflows’ cost and performance metrics, and its output becomes the round’s final workflow.It also writes comments documenting changes and reflections to guide future workflow-design rounds.Source: §A.3 · p. 15

A.4 RESULTS ON CLAUDE 3.5 HAIKU

Claude 3.5 Haiku experiments assess OneFlow on public benchmarks and shopping-specific tasks, reporting competitive performance and lower inference costs.

A.4 RESULTS ON CLAUDE 3.5 HAIKU

  • OneFlow achieves competitive or superior performance across most public benchmarks on Claude 3.5 Haiku.Table 5 reports pass@1 for HumanEval and MBPP, F1 for HotpotQA and DROP, and solve rate for GSM8K and MATH; results average three runs.Source: Table 5 · p. 15, §A.4 · p. 15
  • OneFlow achieves substantial inference-cost reductions on public benchmarks while maintaining competitive performance.Table 6 reports total inference cost in USD, averaged over three runs.Source: Table 6 · p. 15, §A.4 · p. 15

Performance on Shopping-Specific Tasks

  • OneFlow improves over AFlow by 4.0% on Product Selection (0.678 vs. 0.638), 2.5% on Sentiment Analysis (0.777 vs. 0.752), and 4.8% on Multilingual Query Understanding (0.569 vs. 0.521).These gains are reported for complex shopping-specific reasoning tasks involving domain knowledge and user intent.Source: §A.4 · pp. 15–16
  • 9 of 10 shopping-specific tasks: OneFlow achieves state-of-the-art performance, outperforming single-LLM baselines and existing multi-agent frameworks.The comparison uses Shopping-MMLU accuracy averaged over three runs.Source: §A.4 · p. 15, Table 7 · p. 16
  • OneFlow demonstrates superior cost efficiency across all Shopping-MMLU tasks while maintaining competitive performance.Table 8 measures inference cost in USD, averaged over three runs.Source: Table 8 · p. 16

A.5 SYSTEM PROMPT FOR ONEFLOW … A.8 MORE RESULTS ON THE COST ANALYSIS.

This appendix provides supplementary materials on workflow design and review, heterogeneous executors, and evaluation datasets, including Shopping-MMLU. It also identifies supplementary tables on executor-specific token usage and inference costs across public benchmarks.

A.5.1 SYSTEM PROMPT FOR THE CREATIVE DESIGNER META-LLM

  • The designer prompt encourages creative workflow changes such as review, revision, ensembling, and problem-dependent strategies.It asks the designer to optimize an existing graph and record the specific modification.Source: §A.5.1 · p. 16
  • The designer must propose principled rather than hardcoded answers, changing only one detail point and no more than five lines of code per modification.The prompt also requires the modified graph to differ from the tested example and identify its changes.Source: §A.5.1 · p. 17

A.5.2 SYSTEM PROMPT FOR THE CRITICAL REVIEWER META-LLM

  • The critic should preserve the designer’s idea when possible, making it more cost-efficient and proposing fundamental changes only with strong evidence.If the proposed workflow is already cost-effective, the critic may leave it unchanged.Source: §A.5.2 · p. 19
  • The critic evaluates proposed improvements against old-workflow error logs and existing workflows’ performance and cost, while avoiding unnecessary API calls.The prompt warns that workflows with more than five unnecessary API calls per problem can be slow and costly.Source: §A.5.2 · pp. 18–20
  • The critic prompt separates deep reasoning from strict output formatting, recommending code-based formatting or a separate formatting agent.The prompt argues that combining reasoning and formatting instructions can cause output-format failures.Source: §A.5.2 · p. 20

A.6 SYSTEM PROMPT USING FOR AFLOW OPTIMIZED HETEROGENEOUS MULTI-AGENT SYSTEMS

  • The heterogeneous executor guidance assigns steps to Claude 3.5 Haiku and GPT-4o mini according to problem characteristics and cost-performance tradeoffs.It encourages meaningful roles for both models, such as drafting with one and verifying or refining with the other.Source: §A.6 · p. 20

A.7 DETAILS ABOUT THE DATASETS

  • Evaluation uses seven datasets spanning coding, mathematics, question answering, and real-world planning and tool use.Test counts are HumanEval (131), MBPP (341), MATH (486), GSM8K (1055), HotpotQA (800), DROP (800), and TravelPlanner (180).Source: §A.7 · p. 21
  • Shopping-MMLU evaluation selected 10 of 33 shopping-related questions where Claude 3.5 Sonnet achieved less than 80% accuracy.The selection targets challenging e-commerce reasoning tasks.Source: §A.7 · p. 21
  • The 10 selected Shopping-MMLU tasks cover e-commerce domains including product selection, sentiment, attributes, multilingual queries, and review helpfulness.Other covered domains include product complementarity, semantic classification, brand relationships, and keyword intent.Source: §A.7 · p. 21
  • Table 7 presents Shopping-MMLU evaluation results across the selected shopping-specific reasoning tasks.The passage describes the results as demonstrating the approach across diverse shopping scenarios but supplies no values.Source: §A.7 · p. 21

A.8 MORE RESULTS ON THE COST ANALYSIS.

  • Table 9 reports executor-specific input and output token usage alongside a heterogeneous baseline on public benchmarks.Values are averaged over three runs.Source: Table 9 · p. 21
  • The supplied cost-analysis captions specify table scope and aggregation but provide no token-usage or cost values for comparing approaches.The tables cannot establish a winner from the supplied passages alone.Source: Table 9 · p. 21, Table 10 · p. 22
  • Table 10 reports executor-specific inference cost in USD alongside a heterogeneous baseline on public benchmarks.Values are mean ± std over three runs, with lower cost preferred.Source: Table 10 · p. 22

Related papers

Loading 2601.12307v1…