PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence

Geunbin Yu

arXiv:2602.16873v1cs.MAcs.AI

TL;DR

As LLM capabilities converge, selecting a single best model offers diminishing returns, motivating task-adaptive orchestration. AdaptOrch routes dependency-DAG tasks among four topologies and synthesizes parallel outputs. Across three domains, it reports 12–23% improvement over static baselines, while its scope remains limited by decomposition, coupling-estimation, cost, and benchmark constraints.

  • Problem

    As models achieve comparable benchmark performance, the marginal value of model selection diminishes while the optimal topology for coordinating agents remains unresolved.

  • Method

    AdaptOrch maps task dependency DAGs to parallel, sequential, hierarchical, or hybrid topologies and reconciles parallel outputs with adaptive synthesis.

  • Results

    12–23% improvement over static baselines is reported across coding, reasoning, and retrieval tasks, including experiments using identical models.

  • Takeaways & Limitations

    The results support treating orchestration design as a first-class optimization target when model capabilities converge.

  • Takeaways & Limitations

    The approach depends on decomposition quality, uses coarse coupling estimates, can require many concurrent API calls, and has been evaluated on only three benchmarks.

Abstract

from arXiv · show

As large language models from diverse providers converge toward comparable benchmark performance, the traditional paradigm of selecting a single best model per task yields diminishing returns. We argue that orchestration topology -- the structural composition of how multiple agents are coordinated, parallelized, and synthesized -- now dominates system-level performance over individual model capability. We present AdaptOrch, a formal framework for task-adaptive multi-agent orchestration that dynamically selects among four canonical topologies (parallel, sequential, hierarchical, and hybrid) based on task dependency graphs and empirically derived domain characteristics. Our framework introduces three key contributions: (1) a Performance Convergence Scaling Law, formalizing conditions under which orchestration selection outweighs model selection; (2) a Topology Routing Algorithm that maps task decomposition DAGs to optimal orchestration patterns in O(|V| + |E|) time; and (3) an Adaptive Synthesis Protocol with provable termination guarantees and heuristic consistency scoring for parallel agent outputs. We validate AdaptOrch across coding (SWE-bench), reasoning (GPQA), and retrieval-augmented generation tasks, demonstrating that topology-aware orchestration achieves 12-23% improvement over static single-topology baselines, even when using identical underlying models. Our results establish orchestration design as a first-class optimization target independent of model scaling.

1 Introduction

As model capabilities converge, model selection yields diminishing returns and orchestration topology becomes the dominant system-optimization variable. AdaptOrch formalizes topology routing from task structure and reports improvements over static baselines.

  • Motivation: Model capabilities cluster within 2–5% on standard benchmarks, reducing the marginal value of selecting among individual models.The paper frames this convergence as a shift toward optimizing how models are composed.
  • Problem: Existing frameworks either fix execution topologies or dynamically select outputs without adapting the coordination structure to task demands.The unresolved question is which topology best coordinates agents for a specific task.
  • Problem: Parallel specialized agents can shorten workflows, but existing systems leave task decomposition and agent-role assignment to users.The paper identifies algorithmic topology selection as the remaining gap.
  • Approach: AdaptOrch models tasks as dependency-annotated DAGs whose width, depth, and coupling predict the optimal orchestration topology.These structural properties support automated routing rather than manual topology design.
  • Contributions: The framework routes tasks among parallel, sequential, hierarchical, and hybrid topologies using a linear-time Topology Routing Algorithm.This is one of the paper’s stated core contributions.
  • Contributions: AdaptOrch adds an Adaptive Synthesis Protocol with provable termination guarantees and heuristic consistency scoring for parallel outputs.The protocol reconciles outputs through adaptive re-routing and embedding-similarity-based agreement scoring.
  • Results: 12–23% improvement over static baselines is reported across three domains using identical models.The empirical validation covers coding, reasoning, and retrieval-augmented generation tasks.

2 Related Work

Prior systems demonstrate benefits from multi-agent composition but generally retain fixed structural patterns or manually designed workflows. AdaptOrch distinguishes itself by selecting canonical topologies from task dependency structure while combining formal routing, synthesis guarantees, and cost-accuracy analysis.

  • Convergence context: LLM capabilities have converged across providers, while Self-MoA outperforming diverse model mixing challenges the assumption that model diversity is inherently beneficial.The cited evidence includes top-10 MMLU clustering within a 3-point range and a 6.6% AlpacaEval advantage for Self-MoA.
  • Static orchestration: MCP, LangGraph, CrewAI, and AutoGen provide interfaces or workflows, but topology design remains fixed, predetermined, or manual.These systems differ in implementation while leaving structural coordination choices insufficiently adaptive.
  • Dynamic composition: Mixture-of-Agents, LLM-Blender, and DEI report strong results but use fixed layered, selection-based, or committee structures regardless of task structure.The paper identifies explicit topology selection from dependency structure as the unaddressed gap.
  • Concurrent systems: Recent systems demonstrate parallel execution, lead-agent orchestration, DAG dependencies, inbox communication, and multi-provider routing.These efforts establish practical precedents for coordination but do not by themselves provide AdaptOrch’s formal routing framework.
  • Recent work: DyTopo routes at the agent-pair level, whereas MetaGen co-evolves roles and topologies and sacrifices the predictability of closed-form routing.The comparison highlights differing adaptation granularity and predictability.
  • Positioning: AdaptOrch combines DAG-grounded topology routing, provable synthesis termination, and explicit cost-accuracy Pareto analysis.The paper states that no single prior system integrates all three elements.

3 Problem Formalization

AdaptOrch formalizes convergence, task dependency DAGs, canonical orchestration topologies, and topology dominance under parallelizable workloads. Its formal result states that topology-driven variance can dominate model-selection variance as model differences vanish.

  • 3.1 Model Convergence: ϵ-convergence defines a model set whose benchmark scores are sufficiently close, with scores normalized to [0, 1].The supplied passages identify SB(Mi) as the normalized score of model Mi on benchmark B.
  • 3.2 Task Dependency Graphs: A task dependency DAG GT = (V, E, w, c) represents subtasks, precedence constraints, computational costs, and dependency coupling strengths.An edge (vi, vj) means vi must finish before vj starts; c measures required context sharing.
  • 3.3 Orchestration Topologies: The framework defines four canonical topology choices for mapping task structure to execution plans.The supplied formalization names the topology set as T = {τP, τS, τH, τX}.
  • 3.3 Orchestration Topologies: Sequential topology executes subtasks in topological order, passing prior context to each subsequent subtask.Each topology induces a scheduling function that maps the task DAG to execution ordering and agent assignments.
  • 3.4 Performance Convergence Scaling Law: Under uniform weights, Lipschitz aggregation, and a bounded topology-quality coefficient, the formal proposition compares topology and model-selection performance variance.The result is stated for a task with dependency DAG GT and k subtasks.
  • 3.4 Performance Convergence Scaling Law: When ϵ →0 and the task is parallelizable, Varτ/VarM →∞, so topology variance increasingly dominates model-selection variance.The proof sketch attributes model variance to convergence and topology variance to differences between sequential and parallel schedules, adjusted for coupling.
  • 3.4 Performance Convergence Scaling Law: For coding tasks with typical ω ≥3, γ ≤0.4, k ≤6, and ϵ ≈0.05, the variance ratio is at least 20.The corollary identifies orchestration topology as the dominant performance factor over model selection in this setting.

4 The AdaptOrch Framework

AdaptOrch converts tasks into dependency-annotated DAGs, routes them to one of four execution topologies, and synthesizes the resulting agent outputs adaptively. Its routing uses DAG structure, while synthesis combines consistency scoring with re-routing and termination guarantees.

  • AdaptOrch operates through task decomposition, DAG construction, topology routing, execution, and adaptive synthesis.The pipeline supports parallel, sequential, hierarchical, and hybrid execution patterns.
  • Phase 1: Task Decomposition: The decomposer extracts subtasks, dependencies, estimated token costs, and context-coupling requirements from the input task.Dependency declarations are parsed into a formal DAG G_T = (V, E, w, c).
  • Phase 3: Topology Routing: The router maps DAG structural properties—including parallelism width, coupling, and critical-path depth—to a selected topology.Routing uses thresholds for parallelizability, coupling, and minimum task count for hierarchical execution.
  • Phase 3: Topology Routing: The routing decision runs in O(|V| + |E|) using DAG critical-path computation, coupling-density traversal, and topological layering.Approximate layer-width computation suffices for routing, whereas exact antichain width requires more expensive analysis.
  • Phase 4: Execution Topologies: Parallel execution dispatches subtasks simultaneously with isolated contexts, sequential execution follows topological order, and hybrid execution parallelizes within layers.Hierarchical execution uses a lead agent to track dependencies and resolve inconsistent sub-agent outputs.
  • Phase 5: Adaptive Synthesis Protocol: The consistency score heuristically measures semantic agreement through embedding similarity, detecting potential contradictions without guaranteeing logical coherence.Failed synthesis can increase the estimated coupling and re-route execution; after at most 5 iterations, hierarchical routing guarantees termination, while 94% of tasks converge within 2 iterations empirically.

5 Experiments

Across SWE-bench Verified, GPQA Diamond, and HotpotQA, AdaptOrch selects topologies per task and achieves the highest accuracy while maintaining moderate latency overhead. Its routing reflects domain structure, improves token efficiency over multi-agent baselines, and each major component contributes to performance.

  • Main results: AdaptOrch achieves the highest accuracy across SWE-bench Verified, GPQA Diamond, and HotpotQA while maintaining moderate latency overhead.The evaluation uses task accuracy, wall-clock latency, and efficiency metrics across three benchmark domains.
  • Efficiency: AdaptOrch consumes 41.8K tokens per SWE-bench instance, compared with 84.6K for MoA-3L and 61.7K for LLM-Blender.Among multi-agent baselines, AdaptOrch has the highest accuracy per 1M tokens across benchmarks.
  • Topology routing: Hybrid topology τX is selected most frequently at 49.7% on average, while pure parallel favors low coupling and pure sequential favors high-coupling reasoning tasks.HotpotQA routes 71% of instances to τX; SWE-bench routes 62% to τX, 24% to τP, and 14% to τH.
  • Ablations: The synthesis protocol contributes the largest ablation improvement, followed by adaptive routing and coupling-aware decomposition.Removing these components reduces performance by 5.5, 2.8, and 2.3, respectively; removing task decomposition yields Single Best.
  • Robustness: Performance is robust for θγ ∈ [0.5, 0.7], with optimal performance at θγ = 0.6; extreme thresholds degrade routing quality.Low thresholds force sequential execution on parallelizable tasks, while high thresholds permit parallel execution of tightly coupled subtasks.

6 Discussion

AdaptOrch’s benefits depend on task decomposability and topology adaptation, while practical deployment must balance accuracy gains against decomposition quality, concurrency costs, and domain scope.

  • When Does Orchestration Not Help?: AdaptOrch matches but does not exceed Single Best on single-concept recall GPQA instances and gains least on atomic or fully sequential tasks.Orchestration adds value proportional to task decomposability.
  • Comparison Baselines: Self-MoA (matched) recovers 89% of AdaptOrch’s gains over Single Best under the same token budget.The comparison controls for compute while contrasting structured orchestration with uniform self-consistency voting.
  • Practical Implications: The routing algorithm adds <50ms overhead compared with approximately 2–15s per LLM call, supporting real-time topology adaptation.The implementation can map topologies onto Claude Code, LangGraph, and OpenCode infrastructure.
  • Limitations: AdaptOrch’s main limitations are imperfect decomposition and coupling estimates, concurrent-call costs, and evaluation limited to three benchmarks.Parallel execution may exceed rate limits or budgets, while generalization to creative, long-form, and multimodal tasks remains untested.
  • Parameter Sensitivity: Accuracy sensitivity to coupling threshold θγ is evaluated across SWE-bench and GPQA using 30 trials per setting, with an optimal range of [0.55, 0.65].The figure reports 95% bootstrap confidence intervals.

7 Conclusion

AdaptOrch treats orchestration topology as the dominant performance lever under convergent LLM capabilities and operationalizes that thesis through routing, theory, and benchmark validation.

  • Conclusion: AdaptOrch frames orchestration topology as the dominant system-performance lever when LLM capabilities converge.Its conclusion shifts attention from choosing a model to choosing an orchestration structure.
  • Conclusion: The Topology Routing Algorithm runs in O(|V| + |E|) time and selects among parallel, sequential, hierarchical, and hybrid patterns.The framework grounds routing in task structure and presents it as a practical procedure.
  • Future Work: Future directions include learned, dynamically re-orchestrated, cost-aware, and cross-modal routing.These extensions target routing adaptability, budget optimization, and broader task modalities.
  • Empirical Support: Per-instance accuracy distributions vary by routed topology across benchmarks, supporting the adaptive routing approach.Figure 10 uses violin plots for density and white dots for medians.

A Full Proof of Proposition 1

The proof compares model-selection and topology-selection variance for dependency-graph tasks, arguing that topology effects dominate as model capabilities converge.

  • Model Selection Variance: Model performance is bounded within ε of the best model on each subtask, with aggregation determined by the orchestration topology.The proof assumes normalized aggregation with Lipschitz constant L_f ≤1 and accounts for positive within-model correlation.
  • Topology Selection Variance: The topology comparison contrasts fully sequential execution with maximally parallel execution, whose quality differs through latency-quality tradeoffs and context propagation.Sequential execution preserves inter-subtask context, whereas parallel execution can enable more refinement under a budget.
  • Topology Sensitivity: Topology sensitivity is modeled through parallelism width and coupling, with the (ω −1) and (1 −γ) terms capturing topology-induced variation and information loss.The assumption uses a task-class-dependent constant Cτ > 0.
  • Empirical Validation: Removing topology adaptation degrades performance by 4.7–8.3 points in the reported ablation.This empirical result is cited as validation of the topology-adaptation assumption.
  • Scaling Argument: As ε →0, the topology-to-model variance ratio diverges, establishing topology selection as dominant under capability convergence.For typical coding tasks, the proof uses ω ≈3.4, γ ≈0.35, k ≈5, and ε ≈0.05.

B.1 Decomposition Prompt

The SWE-bench decomposition prompt converts a software task into atomic, dependency-annotated subtasks with coupling and token estimates to support topology routing.

  • Decomposition Prompt: The decomposer receives a bug report and repository context and produces atomic software-engineering subtasks.The prompt assigns decomposition to a task decomposition specialist.
  • Output Schema: Each subtask is represented as JSON with an id, description, dependencies, coupling label, and estimated token count.Dependencies use a depends_on list, while coupling is weak, strong, or critical.
  • Decomposition Rules: The prompt maximizes parallelism by adding dependencies only when semantically required.Its typical workflow localizes files, understands context, generates a patch, and verifies the patch.
  • Coupling Labels: Coupling is strong when one subtask’s output directly feeds another and weak when subtasks share domain knowledge without sharing data.These distinctions provide routing-relevant dependency information.
  • Prompt Inputs: The task description and repository context are inserted through explicit prompt fields.The template ends with Task and Repository placeholders.

B.2 Computational Requirements

AdaptOrch uses a shared five-model pool and reports API costs, routing and synthesis overhead, baseline configurations, and topology-dependent accuracy gains. Its router agrees with oracle topology choices on most test instances while limiting misrouting losses.

  • Computational costs: SWE-bench, GPQA, and HotpotQA API costs were approximately $1,200, $180, and $350, respectively.The reported totals cover 500, 198, and 500 instances, with five methods evaluated for each benchmark.
  • Computational costs: Routing adds <50ms per task on a single CPU core, while synthesis adds one LLM call costing approximately $0.01 per instance.These overheads describe the Python implementation and the additional synthesis step.
  • Experimental configuration: All methods use the same five-model pool, greedy temperature 0.0, and max_tokens = 4096 unless otherwise noted.The pool comprises GPT-4o-mini, Claude 3.5 Haiku, Gemini 2.0 Flash, Llama 3.3 70B, and Qwen 2.5 72B.
  • Baselines: Static-Parallel and Static-Sequential reuse AdaptOrch’s decomposition and synthesis but bypass topology routing, isolating the router’s contribution.Static-Parallel runs subtasks simultaneously across three models, whereas Static-Sequential chains dependencies with GPT-4o-mini.
  • Topology-dependent results: +12.6 pp is the largest cluster-level gain in Wide-Shallow tasks, while Chain tasks gain +3.8 pp.The reported pattern attributes the larger Wide-Shallow gain to distributed independent subtasks and the smaller Chain gain to limited topology flexibility.
  • Router evaluation: 81.2% router accuracy matches the oracle across 567/698 instances, and misrouted cases remain within <2 pp of oracle routing.Most errors occur between parallel and hybrid choices because Diamond tasks combine parallel and fan-in structure.
Loading 2602.16873v1…