PaperlayerView exact revision on arXiv

Source-linked AI summary

AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration

Jianhao Ruan, Zhihao Xu, Yiran Peng, Fashen Ren, Zhaoyang Yu, Xinbing Liang, Jinyu Xiang, Yongru Chen, Bang Liu, Chenglin Wu, Yuyu Luo, Jiayi Zhang

arXiv:2602.03786v2cs.AIcs.CL

TL;DR

Existing sub-agent designs rely on fixed roles rather than adapting to emerging subtasks in open environments. AOrchestra dynamically creates specialized sub-agents through an Instruction, Context, Tools, Model abstraction. Across three benchmarks, it consistently outperforms popular agentic systems, achieving 16.28% relative improvement against the strongest baseline when paired with Gemini-3-Flash.

  • Problem

    Existing sub-agent designs rely on static roles and hard-wired capabilities or coordination patterns, while open environments present dynamically emerging subtasks beyond predefined agent sets.

    Source: §1 · p. 2

  • Method

    AOrchestra’s orchestrator creates task-tailored sub-agents on demand through a unified (Instruction, Context, Tools, Model) interface, specializing their working memory and capabilities.

    Source: §5 · p. 8

  • Results

    Across Terminal-Bench 2.0, SWE-Bench-Verified, and GAIA, AOrchestra consistently outperforms popular agentic systems, achieving 16.28% relative improvement against the strongest baseline when paired with Gemini-3-Flash.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The orchestration policy is learnable: supervised fine-tuning improves basic task orchestration, while cost-aware routing via in-context learning yields favorable cost–performance trade-offs.

    Source: §1 · p. 3

Abstract

from arXiv · show

Language agents have shown strong promise for task automation. Realizing this promise for increasingly complex, long-horizon tasks has driven the rise of a sub-agent-as-tools paradigm for multi-turn task solving. However, existing designs still lack a dynamic abstraction view of sub-agents, thereby hurting adaptability. We address this challenge with a unified, framework-agnostic agent abstraction that models any agent as a tuple Instruction, Context, Tools, Model. This tuple acts as a compositional recipe for capabilities, enabling the system to spawn specialized executors for each task on demand. Building on this abstraction, we introduce an agentic system AOrchestra, where the central orchestrator concretizes the tuple at each step: it curates task-relevant context, selects tools and models, and delegates execution via on-the-fly automatic agent creation. Such designs enable reducing human engineering efforts, and remain framework-agnostic with plug-and-play support for diverse agents as task executors. It also enables a controllable performance-cost trade-off, allowing the system to approach Pareto-efficient. Across three challenging benchmarks (GAIA, SWE-Bench, Terminal-Bench), AOrchestra achieves 16.28% relative improvement against the strongest baseline when paired with Gemini-3-Flash. The code is available at: https://github.com/FoundationAgents/AOrchestra

1. Introduction

AORCHESTRA dynamically creates specialized sub-agents through a unified four-tuple interface, separating orchestration from task execution. It reports stronger training-free benchmark performance and supports learning both task orchestration and cost-aware routing.

1. Introduction

  • 16.28% relative improvement over the strongest baseline is reported with Gemini-3-Flash across Terminal-Bench 2.0, SWE-Bench-Verified, and GAIA in training-free evaluation.AORCHESTRA consistently outperforms representative sub-agent orchestration approaches and widely used agent frameworks across all three benchmarks.Source: Figure 1 · p. 1, §1 · pp. 2–3
  • Existing sub-agent designs either isolate context without sufficient specialization or hard-wire roles, leaving coverage gaps and requiring substantial human engineering.Context-thread systems include Schroeder et al. (2025); Sun et al. (2025), while static-role systems include Anthropic (2025); Li et al. (2025c). Earlier multi-agent designs also incur coordination overhead and unreliable context routing (Gao et al., 2025b).Source: §1 · pp. 1–2
  • A dedicated orchestrator decomposes objectives, creates tailored sub-agents, and delegates execution through explicit tool calls without executing tasks itself.Customized capabilities and clean working contexts improve execution accuracy. The orchestrator remains agnostic to executor implementations, making sub-agents pluggable.Source: §1 · p. 2
  • +11.51% GAIA pass@1 improvement follows supervised fine-tuning that strengthens the orchestrator’s subtask decomposition and four-tuple synthesis.This demonstrates that the orchestration policy can be learned from experience rather than remaining entirely training-free.Source: §1 · pp. 2–3
  • +3.03% GAIA pass@1 improvement and 18.5% lower average cost result from in-context learning for cost-aware routing.These gains produce a more favorable cost–performance Pareto frontier.Source: §1 · pp. 2–3
  • AORCHESTRA creates specialized executors on demand through a unified (INSTRUCTION, CONTEXT, TOOLS, MODEL) interface.Instruction and context define the objective and relevant working memory, filtering distracting information. Tools and models provide capabilities selected for each subtask.Source: §1 · pp. 2–3

2. Related Work

AOrchestra contrasts its on-demand sub-agent creation with fixed-workflow multi-agent systems and sub-agent tools that lack full specialization or require manual design.

Multi-Agent Systems

  • Most multi-agent systems rely on fixed workflows, limiting their adaptability despite collaborative task-solving capabilities.Even AutoAgents (Chen et al., 2023), which builds different multi-agent systems for individual tasks, retains a fixed workflow.Source: §2 · p. 3

Sub-Agent as Tools

  • THREAD (Schroeder et al., 2025) recursively spawns sub-agents, while Context-Folding (Sun et al., 2025) branches into subtasks and compresses intermediate steps into summaries.The paper argues that these methods underutilize sub-agents because they do not treat them as fully specialized agents.Source: §2 · p. 3
  • AOrchestra dynamically creates sub-agents on demand through orchestration, without relying on a specific human-designed workflow.It treats each sub-agent as a dynamic unit rather than a manually configured specialist.Source: §2 · p. 3
  • Claude Code provides isolated sub-agent contexts, custom prompts, and tool permissions, but typically requires manually designed, fixed specialists.Source: §2 · p. 3

3. Methodology

AORCHESTRA separates orchestration from execution by creating sub-agents from instruction, context, tools, and model tuples. Its orchestrator learns through supervised fine-tuning for task orchestration and iterative prompt optimization for cost-aware decisions.

3.1. Problem Formulation

  • The general orchestration problem permits direct environment actions, delegation, and termination while maximizing task success with an optional execution-cost trade-off.States retain history, intermediate results, and environment feedback. The parameter λ controls the trade-off for costs including token usage, tool calls, latency, or money.Source: §3.1 · pp. 3–4

3.2. AORCHESTRA

  • AORCHESTRA dynamically instantiates framework-agnostic agents as (I, C, T, M): instruction, context, tools, and model, separating working memory from capabilities.Curated context and tailored capabilities specialize each executor, while the abstraction keeps orchestration independent of executor implementation.Source: §3.2 · pp. 4–5
  • The orchestrator never acts directly in the environment: Delegate creates a tuple-specified executor, while Finish returns the final answer.Executors condition only on assigned instructions and context, use the selected model and tools, and return summaries, artifacts, or failure logs. Returned observations update the orchestrator’s state.Source: §3.2 · p. 5

3.3. Learnable Orchestrator

  • Supervised fine-tuning behavior-clones expert trajectories to train a structured-action policy for subtask decomposition and selecting instructions, context, and tools.The implementation uses straightforward SFT to demonstrate the potential of training a specialized orchestrator, without prescribing it as the only training method.Source: §3.3 · p. 5
  • Iterative in-context learning optimizes the orchestrator’s instruction without changing model weights, using performance-and-cost feedback to improve model routing and tool usage.An optimization model analyzes execution trajectories and proposes prompt edits over N rounds of interaction, aiming for Pareto-efficient performance–cost trade-offs.Source: §3.3 · p. 5

4. Experiments

AORCHESTRA reports improved average pass@1 over the best baselines across three benchmarks with Gemini-3-Flash. Further analyses support curated context sharing, learnable cost-aware orchestration, and interchangeable sub-agent implementations.

4.2. Main Results

  • 22.13% average pass@1 improvement over the best baselines is reported for AORCHESTRA with Gemini-3-Flash across three benchmarks.The comparison covers GAIA, Terminal-Bench 2.0, and SWE-Bench-Verified using pass@1/pass@3. AORCHESTRA uses Gemini-3-Flash as orchestrator and a single sub-agent model choice per configuration.Source: Table 1 · p. 6, §4.2 · pp. 6–7
  • 80.00 pass@1 and 86.06 pass@3 on GAIA give AORCHESTRA its strongest reported performance against baselines when both orchestrator and sub-agents use Gemini-3-Flash.GAIA pass@1 improves by 13.94 absolute points over OpenHands, from 66.06 to 80.00. Using Claude-4.5-haiku sub-agents still achieves 60.61 pass@1; Claude Code is not evaluated on GAIA.Source: Table 1 · p. 6, §4.2 · p. 7
  • 52.86 pass@1 on Terminal-Bench 2.0 and 82.00 pass@1 on SWE-Bench-Verified are achieved by AORCHESTRA with Gemini-3-Flash.Terminal-Bench results include 57.14 pass@3, versus Mini-SWE’s 34.29 pass@1. SWE-Bench-Verified results include 86.00 pass@3, outperforming ReAct and OpenHands while remaining competitive with Mini-SWE; pass@1 exceeds 70.00 across all three backbones.Source: Table 1 · p. 6, §4.2 · p. 7

4.1. Experiment Setup

4.3.1. ADVANTAGE 1: CONTEXT SHARING

  • Curated task-relevant context achieves the highest score against No-Context and Full-Context variants while holding sub-agent models, tools, and system prompts fixed.The comparison uses 50 GAIA validation examples. No-Context loses critical execution traces and fine-grained cues, whereas Full-Context introduces irrelevant information and aggravates context degradation.Source: Table 2 · p. 7, §4.3.1 · p. 7

4.3.2. ADVANTAGE 2: A LEARNABLE ORCHESTRATOR

  • 75.15% accuracy at $0.57 average cost is achieved by mixed-model AORCHESTRA after ICL, versus 72.12% accuracy at $0.70 average cost before optimization.The orchestrator instruction is iteratively optimized from interaction trajectories using performance and monetary-cost feedback. Step-wise model routing enables different accuracy–cost profiles, with AORCHESTRA configurations forming the reported Pareto frontier across model choices.Source: Table 3 · p. 8, Figure 4 · p. 8, §4.3.2 · p. 8
  • 68.48% accuracy after orchestration SFT improves on Qwen3-8B’s 56.97%, while increasing average cost per task by $0.32.Gemini-3-Flash remains the sub-agent executor, isolating the weaker orchestrator’s role. Execution-trace analysis associates fine-tuning with stronger long-horizon problem solving and 56% more total attempts than the base model.Source: §4.3.2 · pp. 7–8, Table 3 · p. 8

4.3.3. ADVANTAGE 3: PLUG-AND-PLAY SUB-AGENTS

  • Interchangeable sub-agent backends maintain stable AORCHESTRA performance and consistently outperform their corresponding baselines.Terminal-Bench experiments replace the executor with ReAct-style and Mini-SWE-style implementations while keeping Gemini-3-Flash as orchestrator. The results support robustness without dependence on a particular sub-agent implementation.Source: Table 4 · p. 8, §4.3.3 · p. 8

5. Conclusion · A. Implemention Details

AORCHESTRA automates task-tailored sub-agent creation through a unified four-tuple interface and reports improvements across three benchmarks. Its abstraction supports plug-and-play implementations and orchestrator optimization through supervised fine-tuning and context learning.

5. Conclusion

  • AORCHESTRA automates sub-agent creation through a unified (Instruction, Context, Tools, Model) interface for complex, long-horizon tasks.Instruction and context define specialized working memory, while model and tools define capabilities.Source: §5 · p. 8
  • The orchestrator dynamically spawns task-tailored executors, enabling on-demand specialization with task-sufficient context.Sub-agents are treated as dynamically creatable units rather than fixed executors.Source: §5 · pp. 8–9
  • 16.28% points average gain in pass@1 across GAIA, Terminal-Bench, and SWE-Bench-Verified is reported over established baseline frameworks.The conclusion reports strong, consistent improvements when AORCHESTRA is paired with frontier models like Gemini-3-Flash.Source: §5 · p. 9
  • The abstraction keeps sub-agents plug-and-play across implementations.Source: §5 · p. 9
  • Decoupling abstraction from execution makes orchestration learnable.Source: §5 · p. 9
  • The paper presents supervised fine-tuning and context learning as two ways to optimize the orchestrator.Source: §5 · p. 9

A.1. Datasets

AORCHESTRA is compared with baselines on three datasets covering general assistance, command-line workflows, and software repair. The command-line and software-repair evaluations use random task samples to reduce cost.

GAIA

  • 165 tasks comprise the GAIA (Mialon et al., 2023) validation split, evaluating general assistants on realistic, tool-augmented questions.Questions often involve web browsing and multi-step reasoning.Source: §A.1 · p. 11

Terminal-Bench 2.0

  • 70 tasks are randomly sampled from the 89-task Terminal-Bench 2.0 test split for the main experiments because of cost.The benchmark evaluates end-to-end, real-world workflows in a sandboxed command-line environment, graded by executable tests.Source: §A.1 · p. 11

SWE-Bench Verified

  • 100 tasks are randomly sampled from the 500-task SWE-Bench Verified (Jimenez et al., 2023) test split for evaluation because of cost.Agents generate patches resolving real GitHub issues in real repositories, verified through tests; the Verified split is human-screened to remove problematic cases.Source: §A.1 · p. 11

A.2. SFT Hyper-parameters.

The experiments use the SFT hyperparameters documented in Table 5, but the supplied excerpt does not include their settings or values.

A.2. SFT Hyper-parameters.

  • Table 5 documents the SFT hyperparameters used in the experiments.Only the table caption is supplied; individual hyperparameters and values are unavailable.Source: §A.2 · p. 11, Table 5 · p. 11

A.3. Baseline Implementations … C.1. GAIA Case Study

Task-specific prompts govern delegation, verification, execution budgets, and reporting for GAIA, Terminal-Bench, and SWE-Bench, while strategy optimization balances performance against cost. Baseline coverage excludes configurations judged unsuitable or poorly compatible.

A.3. Baseline Implementations

  • Claude Code results are omitted from the main experiments because its architecture and intended usage were judged unsuitable for GAIA open-world multi-hop question answering.Source: §A.3 · p. 11
  • DeepSeek-V3.2 with Claude Code is excluded from Table 1 because initial investigation and empirical trials found poor native compatibility.For Terminal-Bench and SWE-Bench, Harbor provides a unified execution interface for MiniSWE, OpenHands, and Claude Code.Source: §A.3 · p. 11

GAIA Main Agent Prompt; Gaia Sub-Agent Prompt

  • GAIA orchestration delegates specific tasks with context, models, and tools, reviews prior outcomes, and completes sufficient answers without repeating finished work.Model selection favors cheaper models for simple tasks and more capable models for complex reasoning or critical attempts. Sub-agents record observations in memory and must finish when remaining_steps <= 5.Source: GAIA Main Agent Prompt · pp. 11–12, Gaia Sub-Agent Prompt · p. 15

Terminal-Bench Main Agent Prompt; Terminal-Bench Sub-Agent Prompt

  • Terminal-Bench orchestration verifies requirements and edge cases before submission, accounting for fresh containers that discard previous work upon redelegation.Retries receive context identifying what worked and failed so successful steps can be repeated and failures avoided. Sub-agents must finish before timeout and report successful steps, failed attempts, and current state.Source: Terminal-Bench Main Agent Prompt · pp. 12–13, Terminal-Bench Sub-Agent Prompt · pp. 15–16

SWE-Bench Main Agent Prompt; SWE-Bench Sub-Agent Prompt; Sub-Agent Summary Prompt

  • SWE-Bench orchestration checks code changes and tests before submission triggers official FAIL_TO_PASS and PASS_TO_PASS evaluation; incomplete fixes receive further delegation.Sub-agents use a specialized code interface in a Docker container containing the checked-out repository and must finish before timeout. A trajectory summarizer compares execution against requirements, distinguishing completed work from missing or improperly tested requirements.Source: SWE-Bench Main Agent Prompt · p. 14, SWE-Bench Sub-Agent Prompt · p. 16, Sub-Agent Summary Prompt · p. 17

STRATEGY OPTIMIZE PROMPT; STRATEGY SELECT PROMPT

  • GAIA strategy optimization revises model selection and delegation to favor cheaper models while maintaining performance; strategy selection compares candidates on both performance and cost.Optimization receives pass_rate, avg_reward, total_cost, and summarized trajectories. Selection compares the parent/best-so-far strategy against a new candidate, relating trajectory strengths and weaknesses to strategy text.Source: STRATEGY OPTIMIZE PROMPT · p. 17, STRATEGY SELECT PROMPT · pp. 17–18

C.1.1. CASE OVERVIEW · C.1.2. DETAILED CASE · Case Metadata (JSON)

AOrchestra achieved 80.0% success across 165 GAIA tasks using Gemini-3-Flash for both agents. The case study illustrates correction of failed hypotheses, preservation of partial findings, and verification before answering.

C.1.1. CASE OVERVIEW

  • 80.0% overall success was achieved across 165 GAIA tasks using Gemini-3-Flash for both the Main Agent and Sub Agent.Success rates were 88.7% for Level 1, 80.2% for Level 2, and 61.5% for Level 3.Source: §C.1.1 · p. 18
  • 10 attempts costing $5.93 still produced success on Level-1 task 935e2cff, illustrating successful long-horizon execution.Level-2 task 853c8244 also succeeded after 10 interaction rounds at a cost of $3.06; the authors interpret these cases as evidence of context stability.Source: §C.1.1 · p. 18
  • 34 tasks succeeded after more than five attempts through Main-Agent reflection and replanning coupled with iterative Sub-Agent execution.Two Level-3 tasks succeeded after 10 attempts, another after nine, and another after eight.Source: §C.1.1 · p. 18

C.1.2. DETAILED CASE

  • The detailed Main-Agent decisions illustrate correction of an initial accession-number hypothesis after sub-agent feedback identified the proposed artworks as unrelated.Attempt 2 explicitly prohibited reuse of the incorrect 1975.1.784-795 accession numbers proposed in Attempt 1.Source: §C.1.1 · p. 18, §C.1.2 · p. 19

C.1.1. CASE OVERVIEW

  • The orchestrator preserved a useful finding from a timed-out sub-agent and propagated it into the next attempt’s context.Attempt 5 discovered that the snake’s hands were hidden in sleeves; Attempt 6 received that finding explicitly in its context.Source: §C.1.1 · p. 18
  • 11 was the correct visible-hand count for the Level-2 Met exhibition task, which AOrchestra solved after 10 attempts.The question concerned twelve Chinese zodiac figures in a 2015 exhibition; the final answer included every figure except the snake.Source: §C.1.1 · p. 18

Case Metadata (JSON)

  • The JSON trace specifies image-and-description audits and additional verification that non-snake figures show human-like hands rather than disqualifying paws or hooves.Instructions target accession numbers 02.18.730a through 02.18.730l and specifically scrutinize the monkey, dog, and pig before accepting the proposed count.Source: Case Metadata (JSON) · p. 21

Attempt 10 - Main Decision (Final) · D. Tools And Action Space · D.1. Main Agent Action Space

The final decision concludes that 11 of 12 zodiac figures have a visible hand. The appendix outlines agent action spaces, including the Main Agent’s responsibilities for global planning and subtask delegation.

Attempt 10 - Main Decision (Final)

  • 11 of 12 zodiac figures were judged to have a visible hand, with only the snake’s hands hidden.The final decision describes the snake as sitting with its hands tucked into long, loose sleeves.Source: Attempt 10 - Main Decision (Final) · p. 22
  • 12 Qing-dynasty jade anthropomorphic figures formed the zodiac set highlighted in the Metropolitan Museum’s 2015 exhibition, Celebrating the Year of the Ram.The final decision identifies the exhibition title with the Chinese zodiac animal for 2015.Source: Attempt 10 - Main Decision (Final) · p. 22
  • Museum descriptions and high-resolution images supported the decision’s reasoning that held attributes require visible hands, except for the snake.The cited reasoning lists candles, fans, scrolls, and peaches as examples of these attributes.Source: Attempt 10 - Main Decision (Final) · p. 22

D. Tools And Action Space

  • The appendix places Main Agent planning and delegation within broader coverage of Sub-Agent actions, benchmark tool inventories, and execution constraints.Tool inventories and execution constraints are specified for each benchmark.Source: D. Tools And Action Space · p. 23, §D.1 · p. 23

D.1. Main Agent Action Space

  • Two core actions define the Main Agent’s action space, supporting its responsibilities for global task planning and subtask delegation.The supplied passage names only the delegate task action.Source: §D.1 · p. 23
  • The delegate task action assigns a well-scoped subtask to a specialized Sub-Agent.The passage introduces a parameter schema but does not supply its fields.Source: §D.1 · p. 23

Action Schema: delegate task · Action: complete

The delegation schema requires task_instruction and model, with additional fields for distilled context and an optional tool subset. The complete action submits the final answer and terminates the task.

Action Schema: delegate task

Action: complete

D.2. Sub-Agent Tools For each Benchmark

Table 6 summarizes the tools available to sub-agents for each benchmark and their corresponding constraints.

D.2. Sub-Agent Tools For each Benchmark

  • Sub-agent tool availability and corresponding constraints are documented by benchmark in Table 6.The supplied passage does not identify individual tools or specify their constraints.Source: §D.2 · p. 23

D.2.1. GAIA TOOLS … Cost Table

AOrchestra equips sub-agents with benchmark-specific tools and isolated execution environments, while routing external access through tool interfaces. These sections also document reproducibility measures and model-specific input/output pricing.

D.2.1. GAIA TOOLS

  • GAIA sub-agents use Serper search, Jina webpage extraction, sandboxed Python/Bash execution, and vision- and audio-capable LLM backends.The finish tool reports subtask results to the Main Agent.Source: §D.2.1 · pp. 23–24

D.2.2. TERMINAL-BENCH TOOLS

  • Terminal-Bench sub-agents execute shell commands and return outputs inside Docker/E2B sandboxes; finish reports intermediate progress without triggering tests.Source: §D.2.2 · p. 24

D.2.3. SWE-BENCH TOOLS

  • SWE-Bench sub-agents run shell commands, read specified file-line ranges, edit files through string replacement, and report progress to MainAgent.Shell execution supports operations such as git commands and tests.Source: §D.2.3 · p. 25

D.3. Sandbox and Network Constraints

  • Sandbox and network restrictions isolate code execution, prohibit destructive Bash operations, and mediate web access exclusively through APIs rather than raw HTTP.Code executes in workspace/temp with a default 10-second timeout; Serper search and Jina extraction have 30-second and 50-second timeouts, respectively. Prohibited operations include file deletion, privilege escalation, permission changes, root-level redirection, and system-level commands.Source: §D.3 · p. 25
  • 600 seconds is Terminal-Bench’s default execution timeout, while SWE-Bench uses a 300-second pytest timeout in per-task Docker containers.Terminal-Bench supports Docker, E2B, and Daytona backends and infers the working directory from Dockerfile WORKDIR. SWE-Bench automatically clones the target repository and checks out the specified base commit.Source: §D.3 · p. 25

E.1. Third-Party APIs

  • Third-party APIs support web search and sandbox creation/execution exclusively through tool interfaces; applicable caching and response logging support reproducible evaluation.Neither the main agent nor sub-agents access external services directly; logged metadata includes timestamps, query strings, and URLs.Source: §E.1 · p. 25

Cost Table

  • 0.0005 input and 0.003 output are the listed gemini-3-flash-preview rates; gpt-4o-mini lists 0.00015 input and 0.0006 output.The pricing entries distinguish input and output rates for individual models.Source: Cost Table · pp. 25–26

Related papers

Loading 2602.03786v2…