Source-linked AI summary
SquidAgent: Parallelize Wisely, Coordinate Efficiently
Yexiong Lin, Shanshan Ye, Yu Yao, Zhen Fang, Bo Han, Tongliang Liu
TL;DR
Parallel LLM-agent systems can lose expected speedups when context rediscovery and output reconciliation offset concurrency, while duration estimates are unreliable. SquidAgent schedules each dependency layer using predicted output-token costs and reduces coordination overhead; across nine tasks it improves throughput and wall time over stated baselines.
Problem
Parallel multi-agent systems can miss expected speedups or run slower than sequential execution, and lack a principled criterion for when parallelization is worthwhile.
Source: §1 · p. 2Method
SquidAgent applies a layer-wise token-cost criterion and reduces coordination overhead through inherited orchestrator context and upfront shared conventions.
Source: §1 · p. 3Results
Across nine evaluation tasks, SquidAgent achieves 2.6× average wall-time speedup and 2.2× average throughput improvement over Claude Code, and 2.0× average throughput improvement over the strongest multi-agent baseline.
Source: §1 · p. 3, §5 · p. 10Takeaways & Limitations
Cost-aware layer scheduling offers a practical principle for efficient multi-agent LLM execution, with coordination costs included in the parallelization decision.
Source: §1 · p. 3, §5 · p. 10SquidAgent uses a fixed safety margin, omits tool or API latency from its token proxy, and makes only one serial-or-parallel decision per layer.
Source: §5 · p. 10
Abstract
from arXiv · showhide
LLM-based agents solve complex multi-step tasks, but sequential execution incurs substantial latency. In principle, parallelizing work across multiple agents should yield near-linear speedups. Yet existing parallel multi-agent systems often run slower than a single-agent baseline. We attribute this gap to two hidden costs that parallel execution incurs but a serial agent avoids. First, there is a re-exploration cost: redundant effort spent by parallel workers reconstructing context that the orchestrator already possesses, such as prior decisions, that would otherwise be inherited implicitly in a serial execution. Second, there is an alignment cost: the overhead required to reconcile inconsistencies across independently generated outputs. We thus derive a principled decision criterion: a layer should be parallelized only when its critical-path cost, plus re-exploration and alignment overheads, is lower than the corresponding serial cost. While this criterion is naturally expressed in wall-clock time, we observe that LLMs are poorly calibrated when asked to estimate task duration. To address this, we instead measure cost in predicted output tokens, which we empirically find LLMs can estimate substantially more reliably than wall-clock time. Building on this token-based criterion, we propose SquidAgent. It estimates all token budgets in a single planning step, forks each worker directly from the orchestrator's session to eliminate re-exploration cost, and replaces post-hoc reconciliation with a pre-generated shared convention block that converts alignment into a bounded upfront cost. A deterministic scheduler then applies the criterion layer by layer. Empirically, SquidAgent achieves a 2.2$\times$ mean throughput improvement and a 2.6$\times$ mean wall-time speedup over Claude Code, and a 2.0$\times$ throughput improvement over the strongest multi-agent baseline.
1 Introduction
The introduction asks when parallel execution benefits LLM-agent systems, arguing that its coordination costs can erase concurrency gains. SquidAgent uses a token-cost criterion and coordination-reduction mechanisms, with reported throughput and wall-time improvements over baselines.
1 Introduction
- Parallel systems often miss expected speedups because workers redundantly rediscover context and reconcile inconsistent outputs, costs that can outweigh concurrency gains.The figure illustrates naive parallel execution exceeding the sequential baseline; implicit dependencies include shared conventions and global design choices.Source: §1 · p. 2, Figure 1 · p. 2
- The criterion compares summed subtask costs with the slowest worker’s cost plus re-exploration and alignment, using predicted output tokens because duration estimates are unreliable.It is applied before execution, layer by layer across the task DAG.Source: §1 · p. 3
- SquidAgent estimates token budgets in one planning pass, forks workers from orchestrator context, and prepares shared conventions to reduce coordination overhead.A deterministic scheduler chooses serial or parallel execution by applying the criterion to each dependency layer.Source: §1 · p. 3
- 2.6× average wall-time speedup and 2.2× average throughput improvement over Claude Code, plus 2.0× average throughput improvement over the strongest multi-agent baseline.Across nine evaluation tasks, SquidAgent also obtains the best overall task completion quality.Source: §1 · p. 3
2 Related Work
Related work spans sequential, parallel, and flexible multi-agent execution, alongside graph-based task representations. The paper distinguishes its focus on explicit cost-aware decisions about whether to parallelize.
2 Related Work
- Sequential pipelines preserve global consistency but sacrifice speedup; parallel systems dispatch independent subtasks concurrently but typically use fixed rules rather than cost-benefit criteria.Flexible agent frameworks often leave execution policy to developers.Source: §2 · pp. 3–4
- Flexible composition frameworks leave execution policy to developers, while debate systems target answer quality through critique and consensus, orthogonal to execution scheduling.The paper studies coordination of workers during inference.Source: §2 · p. 4
- Graph task representations expose explicit dependencies and synchronization layers; existing adaptive schedulers still lack an explicit serial-versus-parallel cost criterion.SquidAgent uses dependency layers as the unit for deciding whether parallelization is worthwhile.Source: §2 · p. 4
3 Method
SquidAgent compares serial and parallel layer costs, including parallelism’s re-exploration and alignment overheads, and parallelizes only when its estimate favors parallel execution. It operationalizes this criterion with output-token budgets, inherited context, upfront conventions, and deterministic layer-wise scheduling.
3.1 Setting and notation
- The orchestrator represents a request as a dependency DAG, groups eligible tasks into topological layers, and estimates task and layer costs.Tasks in a layer have dependencies satisfied by earlier layers.Source: §3.1 · p. 4
3.2 Cost decomposition and parallelization criterion
- A layer is parallelized only when its critical-path execution plus re-exploration and alignment costs is lower than serial execution.The savings from replacing a serial sum with a parallel maximum must exceed the additional overheads.Source: §3.2 · p. 5
- SquidAgent estimates output-token budgets instead of execution time, a proxy suited primarily to generation-dominated workflows that excludes tool, retrieval, and external API latency.For a fixed model and decoding configuration, output tokens are produced at a roughly stable rate, and their length is better calibrated than runtime estimates.Source: §3.2 · p. 5
- The safety margin limits decisions vulnerable to estimation error, though it can forgo modest parallel gains.If estimation error is bounded by ε, then ρk > α + ε guarantees a parallel decision, while ρk < α − ε guarantees a serial one.Source: §3.2 · p. 6
3.3 The SquidAgent framework
- In one planning response, the orchestrator estimates each subtask’s output-token cost and each dependency layer’s alignment cost.Per-layer alignment estimates allow for different convention-planning needs and require no extra LLM call.Source: §3.3 · p. 7
- Workers inherit the orchestrator’s context, while layer-specific conventions move alignment work upfront and make its cost estimable.Context inheritance reduces re-exploration to near zero; conventions specify shared notation, interfaces, formatting, and cross-reference requirements.Source: §3.2 · p. 6, §3.3 · p. 7
- A deterministic scheduler applies the estimated cost criterion to each layer, choosing parallel execution only when the ratio exceeds safety margin α.Otherwise, the layer runs serially.Source: §3.3 · p. 7
4 Experiments
The experiments compare SquidAgent with seven baselines across nine tasks and evaluate throughput, output quality, component ablations, token-cost prediction, and scheduler sensitivity. They report higher throughput and quality alongside analyses of the scheduling components and margin choice.
4.1 Setup
- The evaluation uses nine tasks spanning code generation, technical writing, and structured planning, compares seven baselines, and gives methods the same task specification.The protocol sets SquidAgent’s α to 1.9 and reports wall time and deliverable throughput, with output quality evaluated separately.Source: §4 · p. 7, §4.1 · pp. 7–8
4.2 Throughput results
- 2.2× mean throughput (17.2 to 38.1 words/s) over Claude Code and 2.0× (19.2 to 38.1 words/s) over AgentConductor make SquidAgent the throughput leader across the nine tasks.Improvements are most evident on large tasks; deliverables are comparable or larger in size than baselines.Source: Table 1 · p. 8, §4.2 · p. 9
4.6 Scheduler sensitivity analysis
- On MathRef, α ≤1.8 parallelizes Layer 1 at 34–41 words/s, while α ≥2.0 serializes it at 45–51 words/s; the default is α = 1.9.The DAG and token estimates are held fixed; the transition occurs between α = 1.8 and α = 2.0.Source: Figure 2 · p. 9, §4.6 · p. 10
4.3 Quality evaluation
- 98.2±2.1% overall, including 100% on 5 of 9 tasks, gives SquidAgent the highest rubric score; Claude Code scores 97.1±3.1%.Rubrics assess task compliance, domain-specific correctness, and overall polish.Source: §4.3 · p. 9
4.4 Component-wise ablation
- Removing any of the three components lowers throughput; removing scheduling decreases it 32.1%, and the components’ effects are not additive.The ablation removes scheduling, session forking, or convention planning in turn, leaving the other two enabled.Source: §4.4 · p. 9, Table 3 · p. 10
4.5 Output-token versus wall-clock prediction
- Across 59 subtasks, output-token predictions correlate more strongly with observations than wall-clock predictions under fixed DAG, model, backend, and prompt structure.Each quantity was predicted five times per subtask, and median predictions were compared with recorded execution values.Source: §4.5 · p. 10
5 Conclusion
SquidAgent applies a token-cost criterion to choose serial or parallel execution by dependency layer and reduces coordination costs through context inheritance and upfront convention planning. Across nine evaluation tasks, it improves mean throughput over Claude Code and the strongest multi-agent baseline, while retaining the stated limitations of its safety margin, token proxy, and layer-level decisions.
5 Conclusion
- SquidAgent uses token-cost scheduling to choose serial or parallel execution by layer, with context inheritance and upfront convention planning to reduce coordination costs.The conclusion describes this as a cost-aware scheduler for multi-agent LLM systems.Source: §5 · p. 10
- Across nine evaluation tasks, mean throughput improves 2.2× over Claude Code and 2.0× over the strongest multi-agent baseline, while quality remains competitive.Source: §5 · p. 10
- The fixed safety margin does not adapt to token-estimate uncertainty, the token proxy omits tool or API latency, and each layer receives one serial-or-parallel decision.Source: §5 · p. 10
A Evaluation Tasks
The evaluation suite contains nine tasks grouped by output scale and designed to expose different coordination requirements. The tasks range from software systems and document collections to tightly cross-referenced writing and API documentation.
A Evaluation Tasks
- The nine tasks span large software or document collections and smaller multi-part outputs with consistency constraints; every method receives the same prompt without extra task-specific guidance.Source: Table 5 · p. 17, A Evaluation Tasks · p. 17
PixelCraft; ShopFlow
- PixelCraft specifies a Pygame survival game, while ShopFlow requires a full-stack store spanning product, account, order, admin, and frontend features.Source: PixelCraft · p. 17, ShopFlow · pp. 17–18
CompressKit; ArcadeBox
- CompressKit requires multiple from-scratch compression algorithms with round-trip checks; ArcadeBox requires eight browser games with persistent high scores.Source: CompressKit · p. 18, ArcadeBox · p. 18
SlideKit; ClimateAnalysis
- SlideKit links four presentations to summaries and an index with consistent notation; ClimateAnalysis links four analysis scripts to reports whose numbers match script outputs.Source: SlideKit · p. 18, ClimateAnalysis · p. 19
LinAlgBook; MathRef
- LinAlgBook emphasizes mathematical notation consistency across six tutorial files; MathRef requires consistent symbols, theorem numbering, and cross-references across chapters and appendices.Source: LinAlgBook · p. 19, MathRef · p. 19
CloudDocs
- CloudDocs pairs five module-level API references with three cross-cutting guides that must match endpoint, error-code, and schema names.Source: CloudDocs · pp. 19–20
B Additional experiment results
The additional results tables report wall time and final deliverable output across nine evaluation tasks, with throughput defined as deliverable output divided by wall time.
B Additional experiment results
- Throughput in the main text is calculated as deliverable output divided by wall time.The output count includes only final user-facing files.Source: B Additional experiment results · p. 20
- Table 6 reports wall time in seconds across nine evaluation tasks, with lower values indicating better performance.The supplied passage does not provide the table's cell values.Source: Table 6 · p. 20
- Table 7 reports final deliverable output in words, excluding intermediate artifacts such as logs, planning documents, and build outputs.The supplied passage does not provide the table's cell values.Source: Table 7 · p. 20
C Scheduler analysis
The scheduler makes parallel-versus-serial decisions separately for each dependency layer. The examples show parallel execution for independent work and serial execution for tightly coupled layers.
C Scheduler analysis
- On MathRef, the scheduler parallelizes six heavy chapters (ρ0 = 5.09) but serializes four appendices (ρ1 = 1.45) that must cross-reference exact theorem numbers, notation, and definitions.Table 8 summarizes scheduler decisions on multi-layer DAG tasks using a per-layer alignment cost and a threshold of α = 1.9.Source: C Scheduler analysis · p. 20, Table 8 · p. 21
- On CloudDocs, five independent API module references are parallelized (ρ0 = 3.31), while three guides cross-referencing endpoint paths and error codes are serialized (ρ1 = 1.64).Source: C Scheduler analysis · p. 21
- On ClimateAnalysis, independent scripts are parallelized, while a correlation script depending on prior outputs and final reports citing exact analysis values are serialized.The reports have an estimated alignment cost of 5000 tokens and a token-cost ratio of ρ2 = 0.93.Source: C Scheduler analysis · p. 21
D Full prompts
- The appendix provides the actual orchestrator and worker system prompts used in SquidAgent's fork-and-bypass mode.Source: D Full prompts · p. 21
D.1 Scheduler invocation
- The scheduler runs inside create_plan_branch, parses the task DAG, applies the per-layer token-cost inequality, and returns a deterministic schedule that the orchestrator cannot override.Source: §D.1 · p. 21
D.2 Orchestrator prompt
- The orchestrator prompt requires task output-token estimates and per-layer convention-token estimates, and directs it to inspect the returned schedule and dispatch parallel tasks to workers.For parallel layers, the prompt requires shared conventions before worker launch; serial tasks are written directly by the orchestrator.Source: §D.2 · pp. 21, 23
D.3 Worker prompt
- The worker prompt assigns each worker one task in an isolated worktree, requires progressive commits, and forbids workers from merging their task branches.Source: §D.3 · pp. 24–25
E Robustness to ratio-estimation error
The robustness analysis bounds when ratio-estimation error can alter the scheduler's threshold decision. Its safety margin protects against erroneous parallelization while potentially foregoing modest parallel gains.
E Robustness to ratio-estimation error
- If the true ratio ρ exceeds α + ε, parallel execution is guaranteed; if ρ is below α − ε, serial execution is guaranteed.Estimation error can change the threshold decision only in the intervening region.Source: E Robustness to ratio-estimation error · p. 25
- When ε < α − 1, every layer with ρ ≤ 1 is guaranteed to run serially.Source: E Robustness to ratio-estimation error · p. 25
- The safety margin asymmetrically guards against parallelizing layers whose serial execution is cheaper, but may forgo a modest parallel gain.Source: E Robustness to ratio-estimation error · p. 25
- With an exact estimate, layers with 1 < ρ ≤ α are deliberately serialized; under bounded error, a truly parallel-favoring layer can be serialized only when 1 < ρ ≤ α + ε.Source: E Robustness to ratio-estimation error · p. 25
F.1 Prediction correlations and rank reversals
The analysis compares predicted and observed subtask orderings using rank-based measures and finds that token predictions preserve ordering better than wall-clock predictions in the evaluated workflows.
F.1 Prediction correlations and rank reversals
- The analysis compares predicted and observed subtask orderings using Kendall’s τb and the fraction of inverted pairs, with paired bootstrap confidence intervals clustered by evaluation task.Source: §F.1 · p. 26, Table 9 · p. 26
- Wall-clock predictions had a 27-percentage-point higher inversion rate than token predictions in the rank-ordering comparison, whose inverted-pair fractions excluded ties and whose uncertainty used a paired bootstrap clustered by task.Source: §F.1 · p. 26, Table 9 · p. 26
- The observed rank reversals cannot be repaired by a constant multiplicative correction or strictly increasing recalibration, supporting the scoped finding that wall-clock estimates mis-rank durations in the evaluated workflows.Source: §F.1 · p. 26
F.2 Safety-margin stability across nine tasks
Repeated planning tests how stable SquidAgent’s layer decisions are across nine tasks. Tightly coupled layers have lower estimated ratios than loosely coupled layers, creating a stable safety-margin interval that includes the default α = 1.9.
F.2 Safety-margin stability across nine tasks
- The study repeats planning three times for each of the nine tasks and records the estimated ratio for each dependency layer.These repeated-planning results are distinct from the original executed-plan examples in Appendix C.Source: §F.2 · p. 26
- A layer’s decision changes only when the safety margin α crosses its estimated ratio.The scheduler parallelizes a layer exactly when its estimated ratio exceeds α.Source: §F.2 · p. 26
- Across 42 layer instances, tightly coupled layers have lower estimated ratios than loosely coupled layers, yielding a stable safety-margin interval that includes α = 1.9.Table 10 summarizes the mean and range across three planning runs; this experiment tests decision stability, not throughput.Source: Table 10 · p. 26, §F.2 · p. 26
G Transfer to external Flow tasks
On three practical Flow tasks, SquidAgent reduces mean wall time and raises mean throughput while maintaining comparable task-completion quality. The evaluation provides preliminary evidence of transfer beyond the author-designed suite.
G Transfer to external Flow tasks
- The transfer evaluation covers conference website design, game development, and LaTeX Beamer generation from Flow.These are three practical tasks used to assess transfer beyond the designed benchmarks.Source: G Transfer to external Flow tasks · p. 26
- Task-specific automated rubrics assess executability, required components, and task-specific correctness.SquidAgent is compared with Claude Code.Source: G Transfer to external Flow tasks · p. 27
- 1.48× speedup: SquidAgent reduces mean wall time from 386.3 to 260.3 seconds while raising mean throughput from 11.6 to 15.4 words/s and maintaining comparable task-completion quality.The tasks are relatively lightweight and offer limited opportunities for parallelism.Source: G Transfer to external Flow tasks · p. 27, Table 11 · p. 27
H Limitations and future work
The section outlines limitations in token-cost estimates, external-latency coverage, and layer-level decisions, along with adaptive, trustworthy, and cross-domain extensions; it also specifies consistency, scope, and polish requirements for module and guide documents.
H Limitations and future work
- Task and alignment token estimates can be off by 2–5×, while a global margin cannot adapt to each layer’s prediction confidence.Future work could use confidence-aware margins or calibrate token estimates.Source: H Limitations and future work · p. 27
- Predicted output tokens mainly proxy generation-heavy workflows and miss variable tool, retrieval, environment, and API latency that context inheritance cannot fully eliminate.Future work could add environment-specific latency models or online runtime measurements.Source: H Limitations and future work · p. 27
- Layer-level scheduling gives every task in a DAG layer the same decision, though a <3000-token threshold serializes small tasks; finer per-task choices may improve efficiency.The current threshold only partially addresses layers mixing large and small tasks.Source: H Limitations and future work · p. 27
- The work coordinates existing LLM agents and introduces no model capabilities, training procedures, or datasets; it anticipates no direct negative societal impacts beyond those inherent in LLMs.Source: H Limitations and future work · p. 27
- Future work could recalibrate costs during model adaptation [Huang et al., 2025, Cai and Sugiyama, 2026, Li et al., 2026a, Wang et al., 2026a] and investigate worker dependencies and error propagation using causal methods [Li et al., 2025, Li and Liu, 2025, Li et al., 2026b; Lin et al., 2026b,a; Zhu et al., 2026c].Because the current DAG represents execution dependencies rather than identified causal effects, these extensions require additional validation.Source: H Limitations and future work · p. 27
- Future work could include quality checks, execution safeguards, and privacy and memory controls in scheduling, accounting for their computational overhead [Cai et al., 2025, Chen et al., 2026b, Huang et al., 2026e; Chen et al., 2026a; Wu et al., 2026; Xiang et al., 2026, Huang et al., 2026b; Huang et al., 2026c; Zhang et al., 2026b].Source: H Limitations and future work · pp. 27–28
- Cross-domain evaluation could test dialogue, selling, image and game generation [Gao et al., 2025, 2024, Su et al., 2026; Chen et al., 2025; Huang et al., 2026a], clinical and driving workflows [Zhu et al., 2026a, Hu, 2026, Zhu et al., 2025, 2026b, Huang et al., 2026f], and security and financial prediction [Wang et al., 2026c, Ma et al., 2026, You et al., 2026].Evaluations should jointly examine latency, quality, and robustness rather than assume gains transfer from current tasks.Source: H Limitations and future work · p. 28
CloudDocs
- Documentation should use consistent endpoint, schema, error-code, and authentication formats, with guide references matching exact module paths and methods.The error reference should include each error’s status, code, description, affected endpoints, and resolution steps.Source: CloudDocs · p. 37
- Tier 3 covers quality and polish, including consistent heading levels, code blocks, and table styles across documents.Source: CloudDocs · p. 37
- Module and guide documents have specified word-count ranges and minimum endpoint counts, while the migration guide must cover renamed endpoints, schema changes, deprecated fields, and new required parameters.Source: CloudDocs · p. 37