The Coordination Tax and the Heterogeneity Dividend: Break-Even Conditions for Multi-Agent LLM Pipelines

• Single-task, single-model benchmarks can conflate decomposition, protocol overhead, and model assignment, omitting the pipeline’s potential savings from cheaper role-specific models.The paper notes that such comparisons can report the combined cost as if it were the protocol cost.Source: §1 · pp. 1–2
• The runtime question matters especially for voice agents because coordination overhead delays the reply within the conversational turn.Source: §1 · p. 1
• A closed-form model compares decomposition overhead with savings from routing work to cheaper models, deriving a cost threshold and feasibility bound.The paper also identifies a latency condition and misrouting term.Source: §1 · p. 2
• The study separates decomposition, protocol, and model assignment across six conditions, including a cheap-model call and a router, and evaluates 135 tasks in two difficulty tiers.It also measures OFP’s per-handoff runtime overhead directly inside the convener.Source: §1 · p. 2

2.1 Agent Interoperability Protocols

• OFP standardises the Conversation Envelope, Dialog Events, and floor-management events—requestFloor, grantFloor, revokeFloor, and yieldFloor—so agents built on different stacks can take turns; protocol-level floor management makes mid-task reassignment expressible.Source: §2.1 · p. 2

2.2 Cost-Aware Serving: Cascades and Routers

• Single-call cascades and routers make model choice the optimization variable, offering cheaper serving without decomposition or inter-agent protocols. [3] [20]Source: §2.2 · p. 3

2.3 Multi-Agent LLM Frameworks and Their Evaluation

• Prior role-structured pipeline evaluations often prioritize quality, while cost and latency may be secondary; cost-controlled evaluation motivates reporting an accuracy-cost frontier. [23] [16] [17]Earlier work also instrumented an OFP-mediated pipeline to make the protocol boundary measurable.Source: §2.3 · p. 3

2.4 The Gap

• The identified gap is a controlled design separating decomposition, protocol, and model assignment while directly measuring handoffs and including cheap-call and router baselines.Comparisons across protocols and fault-recovery, fan-out, and discovery properties remain future work.Source: §2.4 · p. 4

3.1 Notation

• For a pipeline of k stages, latency and cost aggregate model time and token charges across stages, with a protocol tax for each handoff.The baseline is a single strong-model call, and the per-handoff tax is measured by subtracting the receiving model’s call time from the timed handoff.Source: §3.1 · p. 4

3.2 Three Derived Quantities

• The model defines α as pipeline output-token volume relative to a single call, f as the share of pipeline output tokens generated by the cheap model, and r as the cheap-to-strong output-price ratio.These quantities depend on the deployment, routing policy, and provider price list, respectively.Source: §3.2 · p. 4

3.3 The Cost Break-Even Condition

• Decomposition is cheaper when α[(1 − f) + fr] < 1, equivalently when f exceeds the break-even threshold f⋆.The model interprets α − 1 as the decomposition tax and αf(1 − r) as the heterogeneity dividend; decomposition pays when the dividend exceeds the tax.Source: §3.3 · pp. 4–5

3.4 A Feasibility Bound

• At r = 0.025, the feasibility ceiling is a 40-fold token-volume inflation; beyond α = 1/r, no routing policy can make decomposition cheaper.At any practical decomposition overhead, routing pays only when a clear majority of generated tokens can be assigned to the cheap model.Source: §3.4 · pp. 5–6
• Table 1 gives the minimum routable fraction by decomposition inflation α and price ratio r, with dashes marking infeasible cases where α ≥ 1/r.Figure 1 shows the threshold curves steepest near α = 1.Source: Table 1 · p. 5, Figure 1 · p. 5

3.5 Including Input Tokens

• Including input costs makes decomposition overhead worse in long-context regimes because later stages must receive the context again, inflating input volume with pipeline depth.Equation (3) is the ρ → 0 special case of the input-and-output formulation.Source: §3.5 · p. 6

3.6 The Latency Condition

• The pipeline is faster only when routed work exceeds a threshold that accounts for generation throughput and fixed overhead; feasibility requires αw + θ < 1.The fixed overhead includes extra prefills and protocol tax, which routing to a faster model does not remove.Source: §3.6 · p. 6

3.7 Separating Price from Volume

• Token volume can change with model assignment, so α must be measured under the evaluated assignment and heterogeneous cost ratios should separate volume and price factors.A cheaper model may write less or reason more, making the volume change a behavioral difference rather than a contractual discount.Source: §3.7 · pp. 6–7

3.8 Misrouting and Retries

• Misroutes incur both a wasted cheap call and a retry, tightening the routing threshold; the heterogeneity dividend vanishes at q = (1 − r)/(κ − r).With a price ratio of one half and an unrefunded retry, roughly one misroute in three erases the advantage.Source: §3.8 · p. 7

4.1 Conditions

• Six conditions isolate decomposition, protocol use, and model assignment, while the cheap single call and router provide economic baselines.B versus A tests decomposition, C versus B tests the protocol, and D versus C tests model assignment.Source: §4.1 · p. 7, Figure 2 · p. 8

4.2 Tasks

• The evaluation uses 85 eligible ClassEval tasks and 50 HumanEval tasks, with generated code counted as solved only when every test passes.ClassEval tasks requiring more than the Python standard library were excluded; the HumanEval sample was drawn with a fixed seed.Source: §4.2 · pp. 7–8

4.3 Models, Prices and Reasoning

• Claude Opus 5.5 is the strong model and Claude Haiku 5.5 the cheap model, with both priced at a 0.025 ratio and reasoning effort fixed explicitly.Calls use medium reasoning effort except the router, which uses low effort.Source: §4.3 · p. 8

4.4 Measuring the Protocol

• Protocol overhead is measured within the convener by subtracting reported model-call time from each timed handoff; loopback HTTP makes this a lower-bound setting.Agents run as separate processes on one host, and process startup is excluded from handoff timing.Source: §4.4 · p. 9

4.5 Execution and Analysis

• Each task-condition pair is repeated three times, and condition comparisons use task-level averages, paired tests, and bootstrap confidence intervals.The hypotheses and criteria were fixed before data collection; the study also tests predictions about protocol cost, model assignment, economics, and class-task quality.Source: §4.5 · p. 9

5.1 The Protocol’s Cost

• Across 2,430 handoffs in C and D, the protocol took a median 7.2 ms per handoff, or 0.11% of C’s median wall time.C cost 1.01× as much as B; the mean per-task wall-time difference was +0.29 s, with p = 0.15.Source: §5.1 · p. 10

5.2 Accuracy and Cost

• B and C cost 3.20× and 3.22× as much as A, respectively, while their pass@1 differences from A were −0.5 and −1.5 points.The paired comparisons cover all 135 tasks; Table 3 reports confidence intervals and tests.Source: Table 3 · p. 11, §5.1 · p. 11, §5.2 · p. 11
• In the class tier, no condition exceeded A’s 90.2% pass@1; every pipeline was dominated by A, while in the function tier each condition solved at least 98.7% of runs.In the class tier, B and C solved no more than A at about three times the cost, and D solved fewer at 1.28× A’s cost.Source: §5.2 · p. 11

5.3 The Model’s Quantities

• D cost 0.39× as much as C but had a pass@1 difference of −6.7 points; its observed cost was 1.27× A, versus a 1.02× forecast.Under D’s model assignment, output and input volume inflated by 4.10× and 5.13× relative to A, and the forecast error was −19%.Source: §5.3 · p. 11, §5.2 · p. 11

5.4 Routing

• An oracle choosing between the two single calls per task reached 94.3% pass@1 at $0.0098 per solved task, versus R’s 71.4% at $0.0046.R routed 98.0% of function-level runs and 86.7% of class-level runs to the cheap model; its routing call accounted for 15.1% of its cost.Source: §5.4 · p. 11

5.5 Latency

• D took 2.60× as long as A per task and 0.80× as long as C.Within conditions, output length and wall time correlated at r = 0.91–0.98.Source: §5.5 · p. 12

5.6 Failures

• Almost every unsolved run executed and failed at least one test; only five runs did not execute, with no timeouts.No run hit the 16,000-token output limit, and one model call was declined by a safety classifier.Source: §5.6 · p. 12, Table 4 · p. 12

6 Discussion

• 7.2 ms per handoff adds about 20.5 ms across three handoffs, negligible beside model calls lasting tens of seconds.The C-against-B latency comparison has a 95% interval from -0.04 to +0.62 s.Source: §6 · p. 12
• D costs 1.27× A after its token-volume increase, although the forecast was 1.02×; its volume factor is 1.32 and its price factor 0.30.Haiku 5.5 reasoning is 37.6% of D’s output tokens against 21.9% for C; the measured routable fraction f = 0.74 exceeds f⋆ = 0.68, yet D costs more than A.Source: §6 · p. 12
• 94.3% of runs are solved by an oracle choosing between the two single calls, at $0.0098 per solved task versus A’s $0.0198.The oracle's cost per solved task is half of A's at A's accuracy.Source: §6 · p. 13
• On class-level tasks, C’s pass@1 differs from A’s by -2.4 points (95% CI -6.3 to +1.2), despite costing 3.22× as much and taking 3.24× as long.The discussion leaves open whether a reviewer has useful work on under-specified or longer-horizon tasks.Source: §6 · p. 13
• For voice agents, decomposition remains a latency concern: the uniform pipeline takes 3.24× as long as a single call, and the mixed pipeline 2.60×.The discussion identifies the latency condition, rather than the cost condition, as binding for voice agents.Source: §6 · p. 13
• The cost results address only one half of the interoperability case; capabilities such as discovery, fault tolerance, audit trails, and cross-organizational composition remain unmeasured.These capability claims are presented as requiring measurement rather than assertion.Source: §6 · p. 13

7 Threats to Validity

• The 135 tasks come from two public benchmarks likely seen during training, and the two tiers support tier-level—not narrower task-type—comparisons.The authors state contamination threatens absolute pass rates more than paired differences between conditions.Source: §7 · p. 13
• Repeated runs measure sampling noise, but the task—not the run—is the unit of inference.The run count describes measurement effort, not sample size.Source: §7 · p. 13
• Provider-side latency is not absolutely reproducible: randomised interleaving limits drift confounding, but measurements came from one run over two days.Hosted-model latency also depends on load conditions outside experimental control.Source: §7 · p. 13
• The price ratio r can change without notice, so any particular price list is an instantiation of the model, not a finding.The model is expressed in terms of r.Source: §7 · p. 13
• The measured τ is for one OFP Python implementation on one host and is a lower bound for deployments crossing hosts, networks, or organisations.In B, C, and D, the reviewer’s output is graded; in D, the cheap model has the last word on every artefact.Source: §7 · pp. 13–14
• Both models come from one vendor and all tasks are code generation, so protocol effects cannot be separated from vendor-specific serving behavior or generalized here to other task families.The authors note that non-executable quality assessment may rely on LLM judges with known biases.Source: §7 · p. 14

8 Future Work

• Future studies could compare OFP with A2A, measure protocol costs across hosts and networks, test fault recovery, fan-out, and discovery, and evaluate harder tasks, another provider, a trained router, and length-controlled ablations.These extensions target limits of the current one-protocol, one-host study and its unquantified capability claims.Source: §8 · p. 14

9 Reproducibility

• The released MIT-licensed materials include the harness, pinned task-dataset hashes, raw run log, discarded runs, and analysis script for every number, table, and figure in Sections 5 and 6.The run log records per-call model, inputs, outputs, reasoning tokens, cost, model-call time, and protocol time; dry-run mode exercises the harness without model calls.Source: §9 · p. 14

10 Conclusion

• Decomposition can be cheaper only when the routable fraction exceeds the explicit threshold; no routing policy suffices once α reaches 1/r.The model is protocol-agnostic; only the per-handoff tax must be measured anew.Source: §10 · p. 14
• About three times the single call’s cost and time, decomposition brought no accuracy gain; moving roles to a cheap model recovered some cost but lost accuracy.A cheap model that reasons at length raises token volume, which affects the break-even threshold.Source: §10 · pp. 14–15
• 7.2 ms per handoff, the protocol’s measured contribution, added nothing to the bill.Source: §10 · p. 14
• Cheap-model roles can inflate token volume enough to make the break-even forecast miss, despite recovering part of the cost.Source: §10 · p. 15