Towards a Science of Scaling Agent Systems

1. Introduction

• Agentic tasks require sustained environmental interaction, iterative information gathering under partial observability, and feedback-driven strategy refinement, extending the Agentic Benchmark Checklist.Multi-agent evaluations predominantly use non-agentic tasks, potentially misleading judgments about collaboration’s value in interactive deployment.Source: §1 · p. 2
• Multi-agent exploration trades unified context for lossy communication, while sustained interaction introduces depth-dependent coordination overhead, divergent world states, and cascading execution errors.These dynamics differ from scaling on static benchmarks. Without quantitative guidance, practitioners rely on heuristics, while improving frontier-model capabilities further obscure collaboration’s added value.Source: §1 · pp. 2–3
• 260 matched-compute configurations compare five architectures across three LLM families and six agentic benchmarks while holding task prompts and tools constant.The architectures are Single Agent, Independent, Centralized, Decentralized, and Hybrid, instantiated using OpenAI, Google, and Anthropic models. The design addresses confounded architectural comparisons and examines coordination processes beyond final accuracy.Source: §1 · p. 3
• 87% of held-out task configurations receive correct architecture predictions, while cross-validated performance prediction achieves R^2=0.373 across six benchmarks.R^2 reaches 0.413 with a task-grounded capability metric. The mixed-effects model uses efficiency, error amplification, message density, and redundancy without dataset-specific parameters; its generalization claim is restricted to the reported held-out configurations.Source: §1 · p. 3
• β=−0.096 (p=0.002) characterizes the tool-coordination trade-off: tool-heavy tasks suffer multi-agent overhead, with efficiency penalties compounding as environmental complexity increases.Multi-agent coordination fragments the per-agent token budget, leaving insufficient capacity for complex tool orchestration.Source: §1 · pp. 3–4
• 45% single-agent accuracy marks the reported capability ceiling: above this baseline, additional agents yield negative returns (β=−0.236, p=0.004).Coordination costs exceed diminishing improvement potential. Higher Intelligence Index yields consistent linear returns only with architectural alignment; otherwise coordination overhead can negate foundational-model improvements.Source: §1 · pp. 3–4
• 17.2× trace-level error amplification in Independent systems contrasts with 4.4× under centralized coordination, whose validation bottlenecks intercept errors before aggregation.Unchecked individual mistakes cascade into final outputs, making verification structure a measurable determinant of collaborative reliability.Source: §1 · pp. 3–4
• +80.8% relative improvement in structured financial reasoning under centralized coordination contrasts with −70.0% degradation in sequential planning under independent coordination.Decentralized coordination improves dynamic web navigation by +9.2%, whereas all multi-agent variants degrade planning performance by −39% to −70%. Sequential constraint satisfaction suffers when coordination fragments reasoning capacity under fixed computational budgets.Source: §1 · pp. 3–4

Multi-Agent Systems (MAS) versus Single-Agent Systems (SAS)

• Single-agent systems have one reasoning locus, whereas multi-agent systems coordinate multiple LLM-backed agents through messages, shared memory, or orchestration [13] [14].Self-reflection does not constitute collaboration when it remains within one decision-making locus. Independent systems aggregate isolated outputs; Decentralized systems use peer exchange [27], Centralized systems use orchestrators [28], and Hybrid systems combine hierarchical and lateral communication [40].Source: §2 · pp. 4–5
• 15× more tokens are consumed by agents in Anthropic, while Gao et al. [16] find collaboration benefits diminish as base models improve.Gao et al. [16] report frontier models often outperform teams; Cemri et al. [15] identify 14 failure modes (Cohen’s Kappa=0.88). These findings challenge assumptions of uniform multi-agent superiority.Source: §2 · p. 5

Agentic Tasks and Benchmarks

• Agentic tasks require sustained environmental interaction, information gathering under partial observability, and feedback-driven strategy refinement, unlike single-shot inference benchmarks.The distinction separates interactive browsing, trading, software engineering, and planning from static evaluations of mathematics, knowledge, specification-complete coding, and comprehension.Source: §2 · p. 5
• 89% on HumanEval with five agents illustrates ensemble gains on non-agentic benchmarks [23], whereas genuine interaction introduces coordination overhead, divergent states, and cascading errors.Kapoor et al. [23] report only 34% world-state overlap after 10 interactions; coordination overhead grows with interaction depth. Voting can correct isolated errors without the sequential compounding that occurs in agentic settings.Source: §2 · p. 5
• The paper positions its contribution as systematically controlling agentic evaluation across coordination architectures to characterize how collaboration benefits scale with model capability.The study compares five coordination architectures and nine models from three LLM families, rather than introducing new definitions of agentic evaluation. Zhu et al. identify benchmark flaws causing 100% relative misestimation, reinforcing the importance of evaluation controls.Source: §2 · p. 5

Scaling Laws and Coordination Mechanisms

• Collaborative scaling exhibits logistic growth [42], unlike neural power-law scaling [54], but Wang et al. [1] find no significant universal collaborative scaling pattern.Chen et al. [55] distinguish compound inference scaling from single-model training. The absence of a universal collaborative pattern suggests domain-specific rather than general scaling laws.Source: §2 · pp. 5–6
• Prior work indicates that matching communication topology to task structure matters more than simply increasing team size.Zhang et al. [43] report superior performance at 6-45% cost using query-dependent configurations, while Dang et al. [40] attribute orchestration improvements to compact cyclic structures. Du et al. [27] and Smit et al. [58] show debate benefits depend on decomposability, task, and hyperparameters rather than reliably exceeding single-agent strategies.Source: §2 · p. 6

3.1. System Definition

• Building on multi-agent formalism [14], systems comprise agents, a shared environment, communication topology, and orchestration policy; agent count distinguishes single-agent from multi-agent systems.Agents combine a reasoning policy, tool-call actions, memory, and a history-to-action function; histories append action–observation pairs subject to context limits. Single-agent systems use one sequential loop with O(k) computation and memory, zero communication overhead, and limited decomposition or verification.Source: §3.1 · pp. 6–7
• Independent, Centralized, Decentralized, and Hybrid architectures isolate parallel exploration, hierarchical verification, peer information fusion, and combined hierarchical–lateral exchange, respectively.Independent aggregation concatenates outputs without cross-validation or voting; centralized orchestration introduces a control bottleneck. Decentralized agents communicate all-to-all, while Hybrid combines orchestration with limited peer communication.Source: §3.1 · p. 7
• Communication passes messages, whereas coordination directs activities; orchestration can govern output aggregation, decision overrides, memory persistence, and termination.Centralized coordination decomposes tasks and monitors progress, while decentralized debate intertwines information exchange with collective direction.Source: §3.1 · pp. 7–8
• Verification-equipped architectures contain errors more effectively than independent coordination under both task-level and trace-level error amplification metrics.Task-level amplification is E_MAS/E_SAS, where E = 1 − P and P is task success rate; values above 1 indicate net errors, while values below 1 indicate suppression. Trace-level amplification estimates additional computational work from coordination failures using execution-trace tokens.Source: §3.1 · p. 8

3.2. Agentic Tasks and Benchmarks

• Agentic benchmarks require sequential interdependence, partial observability, and adaptive strategy formation, targeting tasks where interaction meaningfully outperforms single-shot approaches.The task definition follows and extends Zhu et al.. Agents must acquire hidden information and update their strategies as observations arrive.Source: §3.2 · p. 8
• Agentic status depends on current model capabilities: benchmarks target tasks requiring multi-step environment interaction, rather than static reasoning solvable through direct prompting.GSM8K and MMLU lack the required conditions in the current setting; tasks considered agentic today may become single-shot solvable as models improve.Source: §3.2 · p. 9
• L > 3 action–observation trajectories, identical tool interfaces, parametric-knowledge controls, and normalization to the best single-agent baseline help isolate architectural effects.Within model families, evaluation emphasizes adaptive reasoning over memorized facts; cross-family comparisons account for knowledge differences through baseline normalization.Source: §3.2 · p. 9

4.1. Setup

• 260 configurations span six benchmarks, three LLM families, and five architectures, with matched maximum total iterations between single-agent and multi-agent systems.The architectures comprise SAS, Independent, Centralized, Decentralized, and Hybrid. Four benchmarks use 50–100 instances per configuration; SWE-bench Verified and Terminal-Bench use 20-instance subsets because Docker-based evaluation is costly.Source: §4.1 · pp. 10, 12

4.2. Main Results

• −0.3% mean MAS improvement over SAS across six benchmarks masks outcomes from −70.0% for PlanCraft Independent to +80.8% for Finance Centralized.The overall improvement has a 95% CI of [−58.7%, +77.2%], with σ = 37.5%, highlighting substantial heterogeneity.Source: Figure 2 · p. 11, §4.2 · p. 14
• +9.2% on BrowseComp-Plus and +5.6% on Workbench favor Decentralized MAS, while SWE-bench Verified and Terminal-Bench show limited or negative gains.All SWE-bench Verified MAS variants degrade by −2.1% to −14.9% versus SAS, consistent with most models’ single-agent baselines exceeding 45%. Terminal-Bench Independent gains +1.7%, while Centralized changes by −19.2%, suggesting limited orchestration benefits with two tools.Source: §4.2 · pp. 13–14
• Task decomposability, not complexity alone, governs coordination viability: parallel Finance Agent reasoning benefits, whereas sequential PlanCraft tasks lose reasoning capacity to communication.Under fixed token budgets, PlanCraft’s artificial decomposition adds redundant recipe research and inventory checks. Information-value convergence correlates strongly in Finance Agent (r = 0.71, p < 0.001), but weakly in PlanCraft (r = 0.18, p = 0.22).Source: §4.2 · pp. 13–15
• +164.3%, +127.5%, and +69.9% are Centralized MAS improvements over SAS on Finance Agent for Google, Anthropic, and OpenAI, respectively.No vendor dominates universally, and sequential PlanCraft constraints undermine multi-agent performance across vendors. Despite family-specific preferences, architecture-specific scaling slopes differ by at most Δmax = 0.023 across families, with CV < 0.02.Source: §4.1 · p. 12, §4.2 · p. 15

4.3. Scaling principles

• The scaling model combines capability, agent count, tool count, single-agent baseline, and measured coordination metrics through mechanistically motivated nonlinearities and interactions.Coordination metrics include overhead, message density, redundancy, efficiency, and trace-level error amplification. Quadratic capability and logarithmic count terms model nonlinear scaling, while interactions test baseline saturation and tool-related coordination costs.Source: §4 · p. 10, §4.1 · p. 13, §4.3 · pp. 16–17
• 0.413 cross-validated R^2 with ACI versus 0.373 with the Intelligence Index summarizes the regression’s predictive performance across 260 configurations.Five-fold validation held out configurations; the model outperformed architecture-label-only and intelligence-only alternatives without dataset-specific parameters. ACI measures each model’s mean single-agent performance across all six benchmarks.Source: §4.3 · pp. 16–17
• −0.096 is the efficiency–tools interaction coefficient, the largest interaction effect, indicating that tool-heavy tasks suffer disproportionately from multi-agent inefficiency.Its 95% CI is [−0.154, −0.037] with p = 0.002. Coordination efficiency falls from E_c = 0.466 for SAS to E_c = 0.074 for Hybrid or E_c = 0.234 for Independent, a reported 2–6× penalty.Source: §4.3 · p. 17
• Tool-rich environments amplify coordination inefficiencies, whereas higher model capability has a positive linear association with performance within the tested range.Standardized interactions cannot be interpreted by multiplying raw predictors: efficiency effects are negligible for T ≤ 4 (|ΔP| < 0.05), while architecture overhead ranges from 58% to 515% and its tool-count interaction is directional, not significant (p = 0.211). Intelligence has β = 0.126 (p = 0.008), with no significant quadratic effect (p = 0.977) over I ∈ [42, 71].Source: §4.3 · pp. 17–18
• R^2_CV = 0.373 and 87% correct held-out architecture selection support the full scaling model’s predictive utility beyond base predictors and capability-only selection.The full interaction model achieves AIC = −236.3; task-grounded ACI improves R^2_CV to 0.413 without finding reversals, while capability-only architecture selection achieves 54%. Illustrative task archetypes favor single-agent planning, centralized analysis, and decentralized tool-heavy execution.Source: Table 3 · p. 19, §4.3 · p. 19, §4.5 · p. 25

4.4. Coordination Efficiency, Error Dynamics, and Information Transfer

• T = 2.72 × (n + 0.5)^1.724 describes architecture-aggregated reasoning turns (R^2 = 0.974), with fixed-budget reasoning deteriorating beyond 3–4 agents.Topology still matters: three-agent Independent averages 11.4 turns versus Decentralized’s 26.1, while Hybrid requires 44.3 turns versus SAS’s 7.2. Coordination rankings remain consistent across domains (Kendall τ = 0.89), and larger-team turn counts are extrapolations rather than measurements.Source: §4.4 · pp. 21, 24
• 0.39 messages/turn approximately marks performance saturation among communicating multi-agent systems; redundancy offers marginal benefits that cannot compensate for architectural inefficiency.The approximately logarithmic relationship excludes SAS and Independent and is descriptive, not universal; redundancy’s interaction with agent count is weak (β = 0.024, p = 0.034) and context-dependent. Successful runs show 2.3% contradictory token mass versus 8.1% in failures, while redundancy above 0.50 correlates negatively with success (r = −0.136), with an observed optimum near 0.41.Source: §4.3 · pp. 18–19, §4.4 · pp. 21–22
• 22.7% average factual-error reduction accompanies orchestrator crosschecking or peer verification; Independent instead amplifies errors by 4.6% without inter-agent verification.Trace-error amplification is 17.2× baseline for Independent versus 4.4× for Centralized; Centralized reduces context omission to 8.3%, whereas Hybrid exhibits 26.4% numerical drift and 12.4% coordination failure. However, adjusted regression attributes architecture differences primarily to efficiency and overhead rather than error propagation itself; Independent averages 0.370 success versus SAS’s 0.466.Source: §4.3 · p. 18, §4.4 · pp. 21–23
• r = 0.71 links information gain to the MAS–SAS performance gap in structured domains, whereas BrowseComp-Plus shows weak, nonsignificant predictive power.Finance Agent successes average 1.4 bits of information gain versus 0.4 bits for failures. Reported gains reach +80.8% for Centralized on Finance Agent, compared with +9.2% for Decentralized on BrowseComp-Plus, where ambiguity limits verification.Source: §4.4 · p. 24
• SAS achieves 67.7 successes/1K tokens, exceeding Centralized’s 21.5, Decentralized’s 23.9, and Hybrid’s 13.6, with distinct deployment trade-offs across model families that remain empirical rather than mechanistically established.Under-coordination occurs below 100% overhead, the highest success–cost ratio occurs strictly between 200% and 300%, and Hybrid over-coordination above 400% shows reduced efficiency. OpenAI shows strongest Hybrid gains on structured tasks, Anthropic shows stable Centralized performance, and Google’s performance range across topologies is < 5%; marginal costs per 1% success gain are approximately $0.008 for OpenAI Hybrid, $0.024 for Anthropic Hybrid, and $0.012 across Google architectures.Source: §4.4 · pp. 23–24

4.5. Robustness and Sensitivity Analysis

• Capability saturation remains the most robust finding across cluster-robust inference and multiple-comparison correction, while several coordination effects remain directional.Dataset clustering inflates some standard errors by up to 2.9×; single_agent_baseline retains significance at p = 0.004. Separately applied Holm–Bonferroni correction retains log_tools, single_agent_baseline, and efficiency_x_tools, but does not resolve pseudoreplication.Source: §4.5 · p. 25
• 87% of held-out configurations have their optimal architecture correctly identified in within-domain cross-validation, despite limited absolute cross-domain prediction.Relative performance rankings between architectures are preserved.Source: §4.5 · p. 25
• 94% match rate across all 16 model×benchmark configurations on SWE-bench Verified and Terminal-Bench supports the ∼45% capability-saturation threshold.The binomial test yields p<0.001.Source: §4.5 · p. 25
• Leave-one-dataset-out cross-validation highlights difficulty predicting absolute success rates across structurally diverse domains using a single regression surface without dataset-specific parameters.Source: §4.5 · p. 25

Larger collectives

• Up to nine agents were explored preliminarily; superlinear communication overhead and declining coordination efficiency beyond moderate team sizes leave larger-scale benefits unresolved.Whether larger collectives develop spontaneous specialization or hierarchical self-organization, rather than being dominated by communication bottlenecks, remains open.Source: §5 · p. 25

Model heterogeneity

• 12.6 percentage points lower mean performance for centralized heterogeneous configurations versus strong-model homogeneous counterparts provides no evidence that model mixing bypasses capability saturation.Across 13 BrowseComp-Plus configurations mixing models within and across families, decentralized gains (+2.0 pp) largely reflect the stronger constituent model. The main study shares base architectures across agents; broader architectural diversity, domain specialization, complementary reasoning, and orchestration-versus-execution specialization remain research directions.Source: §5 · p. 26

Tool-intensive coordination

• Tool-heavy environments are a primary multi-agent coordination failure mode, with significant negative interactions between tool count and system efficiency.Proposed research directions include explicit tool-access scheduling, capability-aware task routing, and hierarchical tool delegation.Source: §5 · p. 26

Prompt optimization

• Identical prompts controlled experimental conditions but were not optimized for individual models or families, so architecture-specific prompt tuning may produce different scaling characteristics.This limitation reflects the known sensitivity of LLM behavior to prompt formulation.Source: §5 · p. 26

Benchmark coverage

• 20-instance subsets for SWE-bench Verified and Terminal-Bench produce typical bootstrap 95% confidence-interval widths of ±20 percentage points per cell, leaving individual pairwise comparisons underpowered.The other four benchmarks use 50–100 instances; intervals use 10,000 bootstrap resamples, while aggregate results across 8 models × 2 benchmarks retain a 94% match rate for the 45% threshold (p<0.001, binomial test). Coverage remains limited to six benchmarks, leaving embodied, multimodal, multi-user, long-horizon, and real-world feedback settings insufficiently tested.Source: §5 · pp. 26–27

Economic viability

• Token-based communication imposes latency and cost floors, while token consumption and latency grow substantially with agent count, often without proportional performance gains.Latent-space reasoning or activation sharing could alter these dynamics; sparse communication, early exits, and distilled coordinators could improve efficiency. Speculative parallel branches with post-hoc pruning may reduce wall-clock time at the expense of greater total compute.Source: §5 · pp. 26–27

Statistical inference

• 87% of held-out cases receive correct optimal-architecture selections under within-domain cross-validation, although predicting absolute success across diverse datasets remains difficult without dataset-specific parameters.With G=6 dataset clusters, conservative cluster-robust inference removes significance from several predictors significant under naive OLS. Both estimates are reported, and dataset-level predictors are treated as descriptive, directionally consistent patterns rather than conventionally confirmed effects.Source: §5 · p. 27

6. Conclusion

• 87% accuracy in selecting optimal architectures for held-out configurations was achieved by the predictive model.The model achieves R^2=0.373 across all six benchmarks and R^2=0.413 with a task-grounded capability metric.Source: §6 · p. 27
• ∼45% single-agent baselines mark the threshold beyond which coordination yields diminishing returns, the most robust scaling effect.Capability saturation remains significant under cluster-robust inference (p=0.004) and Holm–Bonferroni multiple-comparison correction (p_Holm=0.018).Source: §6 · p. 27
• +80.8% on Finance Agent versus −70.0% on PlanCraft illustrates that coordination benefits depend on task decomposability rather than team size.These performance changes show substantial variation with task structure.Source: §6 · p. 27
• 4.4× (centralized) to 17.2× (independent) trace-level error amplification reveals architecture-dependent differences in error propagation.This directional pattern appears consistently across all six benchmarks.Source: §6 · p. 27
• Tool-heavy tasks suffer from coordination overhead, a directional trade-off observed consistently across all six benchmarks.Source: §6 · p. 27
• 0.061 MAS-only MAE and 0.077 overall MAE indicate reasonably calibrated relative MAS performance predictions on held-out frontier models evaluated on BrowseComp-Plus.This provides preliminary evidence of transfer within BrowseComp-Plus, not full cross-domain generalization.Source: §6 · p. 27

Data Availability

• All six benchmark datasets are publicly available.BrowseComp-Plus [32], PlanCraft [21], and Workbench [34] each contain 100 instances; Finance-Agent [33] contains 50 instances.Source: Data Availability · p. 27
• 260 experimental configurations have per-instance results available in the code repository at etc/analysis/.Source: Data Availability · p. 27
• 20-instance subsets were selected using deterministic shuffling with seed 42 for SWE-bench Verified [20] and the first 20 instances for TerminalBench [25].The full datasets contain 500 instances for SWE-bench Verified [20] and 86 instances for TerminalBench [25].Source: Data Availability · p. 27

Code Availability

• The code repository contains the evaluation framework used in the study.Source: Code Availability · p. 28
• Repository documentation describes additional artifacts required for full reproduction.Source: Code Availability · p. 28
• The code repository includes the analysis scripts used in the study.Source: Code Availability · p. 28
• The code repository provides the configuration files used in the study.Source: Code Availability · p. 28
• The code repository includes the prompt templates used in the study.Source: Code Availability · p. 28
• The code repository contains representative sanitized execution traces used in the study.Source: Code Availability · p. 28

A. Model Intelligence Index

• The Artificial Analysis Intelligence Index provides a unified baseline capability measure independent of agentic mechanisms and multi-agent coordination.It spans reasoning, knowledge, mathematics, coding, instruction following, long-context reasoning, and agentic workflows. Consistent zero-shot instruction prompting, pass@1 scoring, robust equality checking, and reported confidence intervals below ±1% support its use.Source: A. Model Intelligence Index · p. 35
• Eight evaluation suites were independently reproduced using Artificial Analysis’s disclosed methodology to cover models lacking official Intelligence Index results.These were AA-LCR, HLE [63], MMLU-Pro [61], GPQA Diamond [62], AIME 2025, LiveCodeBench [65], SciCode [64], and IFBench [66], using disclosed prompts, scoring procedures, and evaluation environments.Source: A. Model Intelligence Index · p. 35
• Reconstructed scores follow Intelligence Index v3.0’s equal-weighting formulation, with incomplete reproductions marked as approximate estimates rather than officially certified values.Table 6 compiles official and reconstructed index values, constituent evaluation scores, and context-window, cost, throughput, and latency metadata.Source: A. Model Intelligence Index · p. 35, Table 6 · p. 36
• Reconstructed scores require caution because evaluations can be nondeterministic, some long-context estimates are upper-bound approximations, and private testing procedures cannot be fully reproduced.Upper-bound long-context approximations apply where models lack public API support for large-context evaluation; Artificial Analysis’s private test variants and additional filtering limit verification.Source: A. Model Intelligence Index · pp. 35–36

B. Out-of-Sample Validation

• 0.077 MAE across 15 held-out architecture-model combinations accompanies better MAS calibration (MAE = 0.061) than SAS calibration (MAE = 0.138).Held-out models GPT-5.2, Gemini-3.0 Pro, and Gemini-3.0 Flash share Intelligence Index = 75, approximately 5.6% beyond the training range of 42–71. SAS predictions systematically overestimate performance.Source: B. Out-of-Sample Validation · p. 36, Table 8 · p. 37
• 73% of finding-model pairs validate, with two additional partial validations; capability ceilings, Centralized-or-Decentralized optimality, and performance-limiting Hybrid overhead generalize across all three models.Independent MAS degradation validates only for GPT-5.2, not the Gemini models. The BrowseComp pattern of Decentralized outperforming Centralized also varies across model families.Source: B. Out-of-Sample Validation · pp. 36–37, Table 9 · p. 37
• +37.0% mean error for Hybrid accompanies incorrect optimal-architecture predictions for all three models; Centralized and Decentralized instead achieve superior performance.The equation predicts Hybrid performance of 0.560; linear extrapolation and unmodeled high-capability overhead motivate architecture-specific corrections. Centralized mean error is +3.0%, while Decentralized signed mean error is +8.8%.Source: B. Out-of-Sample Validation · pp. 37–38
• +38.9% and +41.2% MAS gains for Gemini-3.0 Pro and Gemini-3.0 Flash exceed GPT-5.2’s +6.7%, despite identical Intelligence Index = 75.Single-agent performance is 0.45 for GPT-5.2, 0.36 for Gemini-3.0 Pro, and 0.34 for Gemini-3.0 Flash, while best MAS performance converges at 0.48–0.50. This suggests MAS may compensate for single-agent limitations and Intelligence Index may not be directly comparable across vendors.Source: B. Out-of-Sample Validation · p. 37, Table 10 · p. 38

C. Domain Complexity

• Domain complexity is an ordinal score D∈[0, 1] capturing sequential interdependence and empirical difficulty to analyze coordination benefits versus prohibitive overhead.The score characterizes complexity across the evaluated benchmarks.Source: C. Domain Complexity · p. 38

C.1. Complexity Score Assignment

• Complexity assignment uses three normalized task properties: sequential interdependence, state-space complexity, and coordination-overhead sensitivity.These capture ordered reasoning requirements, dynamic state evolution, and observed multi-agent degradation relative to single-agent baselines. The final score is calibrated against observed MAS performance patterns across all configurations.Source: §C.1 · p. 38

C.2. Domain Characterisation

• Benchmark characterisation pairs complexity scores with task characteristics; illustrative properties include Finance Agent’s parallelizability, PlanCraft’s sequential constraints, and BrowseComp-Plus’s dynamic environments.Table 11 summarizes benchmark complexity scores and defining characteristics; its supplied caption contains no individual score values.Source: §C.1 · p. 38, §C.2 · p. 38, Table 11 · p. 39

C.3. Critical Threshold

• D≈0.40 marks the critical complexity threshold separating net-positive multi-agent returns below it from performance degradation above it.This suggests that domain-intrinsic properties constrain multi-agent suitability, rather than architectural sophistication alone.Source: §C.3 · p. 39
• Below the threshold, effective task decomposition and parallel reasoning enable multi-agent architectures to deliver net-positive returns.Source: §C.3 · p. 39
• Above the threshold, coordination overhead consumes computational resources otherwise allocated to reasoning, degrading performance.Source: §C.3 · p. 39

D. Datasets

• 50 finance questions in Finance Agent [33] and 100 browsing tasks in BrowseComp Plus [32] test domain analysis and multi-website synthesis, respectively.Finance Agent uses expert-provided evaluation rubrics, with questions typically requiring 15–30 minutes of expert time. BrowseComp Plus uses LLM-based comparison against ground-truth answers with confidence scoring.Source: D. Datasets · p. 39
• 100 business scenarios in WorkBench [34] test function-call workflows, while Plancraft [21] tests sequential Minecraft crafting plans.WorkBench covers analytics, calendars, email, project management, and customer relationship management, evaluating exact function-call sequences with tolerance for minor date variations. The plancraft-test subset requires dependency reasoning, resource management, and action ordering, with success determined by crafting within step limits.Source: D. Datasets · pp. 39–40
• 20-instance subsets assess GitHub issue resolution in SWE-bench Verified [20] and CLI task execution in Terminal-Bench [25].SWE-bench Verified uses a deterministic shuffle with seed 42 to select from 500 instances, provides seven tools, and requires patches that pass repository tests. Terminal-Bench uses the first 20 of 86 instances, provides two tools, and evaluates objective success in Docker environments under varying time limits.Source: D. Datasets · p. 40

E.1. Technical Infrastructure

• LiteLLM standardizes model access across providers, while LangChain supports agent orchestration, tool integration, conversation management, and structured prompting.Provider-specific APIs connect GPT, Gemini, and Claude models. API key rotation handles rate limits and quotas, while automatic conversation-history truncation manages context limits.Source: §E.1 · p. 40
• Dataset-specific environments expose structured tools through LangChain’s BaseTool interface and dynamically bind them to models using function calling when available.Tool ecosystems include Tavily web search, Python REPL execution, mathematical operations, and task-completion markers, with structured input schemas and execution methods.Source: §E.1 · p. 41

E.2. Agent Configuration

• 10 iterations cap single-agent runs; multi-agent configurations use synthesis, orchestration, debate, or hybrid coordination with architecture-specific execution limits.Independent systems use three agents with synthesis-only coordination; centralized systems use three sub-agents and one orchestrator for at most five rounds with three iterations per agent per round; decentralized systems use three agents over three debate rounds with three iterations per round; hybrids combine orchestration with limited peer communication. Heterogeneous configurations support different models for orchestrators and workers, including model diversity among decentralized workers.Source: §E.2 · p. 41

E.3. Prompt Compilation System

• Prompt compilation merges agent prompts with shared dataset templates to deliver consistent instructions across architectures while preserving task specificity.Named YAML templates combine base prompts and role-specific extensions, using variable interpolation and conditional selection based on agent type and conversation state. Dataset templates supply instructions and examples, while instances supply problem descriptions, context, and constraints.Source: §E.3 · p. 41

E.4. Evaluation Methodology

• 50 Finance Agent instances and 100 each from BrowseComp Plus, WorkBench, and Plancraft balance evaluation cost with statistical significance.Instance selection aims to represent task types and difficulty levels within each benchmark.Source: §E.4 · p. 41
• 20 instances each from SWE-bench Verified and Terminal-Bench limit the computational cost of Docker-based evaluation environments.SWE-bench Verified uses a deterministic shuffle with seed 42 from 500 instances; Terminal-Bench uses the first N from 86. Bootstrap 95% confidence intervals are reported in Table 16.Source: §E.4 · p. 41
• Identical tool interfaces and observation structures across architectures, together with frozen model weights without fine-tuning, isolate architectural effects from external feedback and model optimization.Source: §E.4 · p. 41
• Consistent context truncation policies, API rate limiting, and retry mechanisms standardize context management and ensure fair resource allocation.Source: §E.4 · p. 41

E.5. Information Gain Computation

• Information gain ΔI measures coordination’s reduction in task uncertainty using Bayesian posterior variance reduction between initial reasoning traces and final aggregated outputs.The task success indicator Y is binary.Source: §E.5 · p. 42
• 10 reasoning traces per state at temperature 0.7 estimate empirical variance of predicted success probabilities before and after coordination.The mean predicted success probability across samples is denoted by p̂(s).Source: §E.5 · p. 42

• 95% bootstrap confidence intervals accompany resolution rates for both benchmarks, using 20 instances and 10,000 resamples.Source: Table 16 · p. 43
• SWE-bench Verified shows limited multi-agent gains when single-agent baseline performance is high, consistent with the capability-saturation threshold.Source: Figure 6 · p. 44
• Terminal-Bench shows mixed multi-agent results despite lower single-agent baselines, with low tool count limiting coordination benefits.Source: Figure 6 · p. 44