Scaling Small Agents Through Strategy Auctions

• The smallest agent reaches ∼90% of the largest agent’s pass@1 on simplest tasks but only ∼21% on the most complex, showing performance fails to scale with complexity.Complexity is operationalized as expert human solution time, ranging from a few seconds to one hour, across deep search and coding tasks.Source: §1 · p. 2
• SALE has agents bid with strategic plans, scores bids by predicted cost and value, selects a winner, and lets cheaper agents refine plans using outcomes from prior auctions.This couples per-task routing with test-time self-improvement, increasing smaller agents’ share of work where possible.Source: §1 · pp. 2–3
• SALE beats the largest agent’s pass@1 (+3.5% deep search; +2.7% coding), offloads workload (−65% deep search; −40% coding), and cuts spend (−42% deep search; −25% coding).These gains come with only a negligible increase in inference tokens.Source: §1 · pp. 2–3
• Task-description routers either underperform the largest agent or fail to meaningfully reduce spend, while SALE’s auction memory leads the smallest agents to be selected increasingly often.The paper argues that complex trajectories decouple task inputs from downstream success, making strategic planning a more reliable routing signal.Source: §1 · p. 3
• HST-Bench1 pairs agentic tasks with human solution times as a proxy for complexity, enabling fine-grained evaluation of how agent performance scales.Source: §1 · pp. 2–3

Agent Performance Under Task Complexity

• Where Kwa et al. (2025) study a human-time-based 50%-success horizon and Sinha et al. (2026) examine controlled multi-step task degradation, this work analyzes real-world deep-search and coding workloads at system level.It shifts from isolated model behavior to marketplace allocation across heterogeneous agents.Source: §2 · p. 4

Multi-Agent Routing

• SALE uses lightweight, partially predictive routing from short strategy bids rather than full solutions, with auction feedback and shared memory refining later bids.Prior non-predictive routing runs multiple models, while predictive routers require separate learned policies and typically lack test-time adaptation; plan quality has been linked to downstream task success (Sun et al., 2024; Kang et al., 2025; Xiong et al., 2025b).Source: §2 · p. 4

Memory-Driven Adaptation

• SALE stores bidding strategies and auction outcomes—wins and losses—rather than answers, execution traces, or user histories, using feedback to refine bids and reallocate work.This contrasts with memory systems that extract reusable routines from successful trajectories (Cao et al., 2026; Wang et al., 2025e) or maintain structured records of interactions and user knowledge (Salama et al., 2025; Wang and Chen, 2025; Xu et al., 2025).Source: §2 · p. 4

Agent Systems as Virtual Economies

• SALE treats agents as freelancers in a marketplace, awarding jobs through a centralized scoring rule that weighs bid price and quality and allocates work and learning opportunities.This builds on work exploring auction-based coordination, virtual economies, and direct agent transactions (Shah et al., 2025; Jiang et al., 2025; Yang et al., 2025b; Tomasev et al., 2025; Rothschild et al., 2026).Source: §2 · p. 4

3 Experimental Setup

• All experiments use the Agent Research Environment (ARE), a standardized platform for consistent benchmarking and comparison of agent behavior (Froger et al., 2025).The evaluation covers deep search and coding, domains selected to represent extended reasoning and multi-step planning.Source: §3 · p. 4, §3.1 · p. 5

3.1 Data

• The 753-task HST-Bench spans deep-search and coding datasets, with complexity defined by expert human solution time and divided into five geometric bins.Deep search includes SimpleQA (Wei et al., 2024), PopQA (Mallen et al., 2023), HotpotQA (Yang et al., 2018), GAIA (Mialon et al., 2024), and part of Humanity’s Last Exam (Phan et al., 2025); coding includes MBPP (Austin et al., 2021), LeetCode, and custom multiple-choice questions. Three expert annotators produced human solution-time estimates with Krippendorff’s α = 0.86; the bins range from up to 6 seconds to 60 minutes.Source: §3.1 · p. 5

3.2 Models

• Experiments use Qwen3 models at 4B ($0.05), 8B ($0.09), 14B ($0.16), and 32B ($0.36) parameters, with prices per million tokens.The effective prices assume a 4:1 average input-to-output token ratio, and all models use greedy decoding.Source: §3.2 · p. 5

4 Agent Performance vs. Task Complexity

• At τ(t) ≤0.1, cheapest agents reach about 87% (deep search) and 92% (coding) of the most expensive agents’ pass@1, but at τ(t) ≤60 only 25% and 17%, respectively.Performance is pass@1 scored via LLM-as-a-judge against ground-truth answers.Source: §4 · p. 5
• As τ(t) rises, token use grows across models; larger agents do not consistently shorten traces, and any shorter-trace savings are too small to offset higher per-token costs.Larger agents therefore do not generally achieve more token-efficient solutions on complex workloads.Source: §4 · p. 6
• Because cheap agents suffice at low τ(t) while complex tasks appear to require expensive agents, universal use wastes resources, motivating dynamic task-to-agent allocation.The goal is to balance resource efficiency with capability by assigning tasks to the right agent.Source: §4 · p. 6

5 Strategy Auctions

• SALE turns each agent’s task-specific strategic plan into a bid, enabling selection based on strategic intent rather than the task description alone.Plans are conditioned on the task and environment and encode intended approaches, including decomposition and tool selection.Source: §5 · p. 6
• Agent cost uses model price and strategy length, while value combines strategy entropy with self-and-peer jury scores.Strategy length is motivated as a proxy for trace cost and execution risk (Goebel and Zips, 2025; Xiong et al., 2025a).Source: §5 · p. 6
• Ablations find that excluding self-evaluation or reducing jury size degrades performance.Source: §5 · p. 7
• SALE learns scoring weights by minimizing worst-case cost-minus-value across training tasks, then routes each new task to the agent with the lowest score.The min–max objective is intended to guard against any training task receiving a disproportionately poor assignment.Source: §5 · p. 7
• After auctions, SALE stores proposed strategies and outcomes in shared memory, allowing only agents cheaper than the provisional winner to submit refined bids when needed.Retrieved past strategy pairs are presented contrastively to guide refinement; an improved bid can replace the provisional winner, otherwise that winner is retained.Source: §5 · pp. 7–8
• Under 1% of average inference cost comes from the auction phase, which adds only a few hundred tokens compared with final traces of tens of thousands to millions.Source: §5 · p. 8

Evaluation Setup

• SALE metrics average five independent random task-order permutations because its auction memory evolves online, whereas single-model baselines use one run; all runs use greedy decoding.Tasks are run in randomized order across the full HST-Bench test set and partitioned into complexity bins only for analysis.Source: §6 · p. 8

Comparison with Existing Routers

• Against existing routers, SALE is more consistent: CARROT reduces cost by 22% on deep search and 14% on coding but underperforms SALE on both metrics.At highest coding complexity, WTP reaches 6.3% pass@1 versus 22.8% for the best single agent; TO-Router stays close to the single-agent baseline, while FrugalGPT spend rises to 0.61$/Mt versus 0.36$/Mt for the best agent.Source: §6 · pp. 9–10

Main Results

• 36–53% lower deep-search cost and 17–50% lower coding cost accompany SALE’s maintained or improved pass@1 versus the best single agent across all complexity levels, advancing the Pareto frontier.The reported reductions are in price per million tokens; improvements are statistically significant across random orderings.Source: §6 · pp. 9–10, Table 1 · p. 9

Ablation

• Without memory, SALE’s single-auction strategic bidding matches or improves the best agent’s average pass@1 while reducing cost across every complexity bin; memory further improves the trade-off.Past auction outcomes can reduce cost at similar accuracy or jointly improve pass@1 and cost; further ablations find that every cost–value term contributes and jury diversity provides a regularizing effect.Source: Table 1 · p. 9, §6 · p. 10

7.1 Agent Allocation

• Smaller agents handle substantial shares of SALE’s workload, including nearly 30% of the highest-complexity deep-search tasks.For deep search, the 32B agent handles 20–44% of tasks across bins, while the 14B agent handles 29–55% and the 4B and 8B agents together handle approximately 12–37%. For coding, the 4B and 8B agents together handle 7–32% of queries across bins; the 32B agent is used for only 22% of the easiest coding tasks.Source: §7.1 · p. 10

7.2 Agent Contributions via Shapley Values

• Without memory, the 32B agent has the highest Shapley values across complexity bins and domains, but memory lowers its contribution while smaller agents generally gain.The analysis attributes the 32B agent’s contribution partly to jury scoring even when smaller agents are selected for final inference; gains for 4B and 8B agents are especially notable on higher-complexity coding tasks.Source: §7.2 · p. 11

7.3 Smallest Agent Selection Over Time

• The smallest agent’s cumulative selection rate rises from 3.7% to 11.1% for deep search and from 1.4% to 5.3% for coding as auction history accumulates.The coding increase is nearly fourfold, with most of the rise concentrated in approximately the first 150 tasks; the authors attribute the increases to increasingly relevant feedback from the growing memory bank.Source: §7.3 · pp. 11–12, Figure 5 · p. 11

7.4 Qualitative Analysis of Refined Strategies

• Memory-guided refinement reuses patterns from winning bids, producing more structured, task-aligned strategies for both search and coding.Search strategies more often specify tools and arguments, constrain searches to reputable sources, and cross-check ambiguity; coding strategies more often specify function names and arguments, align with the requested output, and test provided and additional edge cases.Source: §7.4 · p. 12

7.5 Complementary Failure Modes

• The largest agent’s failures often reflect overconfidence, while smaller agents more often use tools and explicit checks, creating complementary failure modes visible in bids before execution.The analysis notes that this complementarity does not make smaller agents generally more accurate: the largest remains superior in aggregate, especially at higher complexities.Source: §7.5 · p. 12

Conclusion

• Smaller agents perform comparably to larger ones on simple deep-search and coding instances but fall substantially behind as task complexity increases.The conclusion argues that small agents alone are insufficient for complex workloads, while always using the largest model overlooks efficiency opportunities.Source: §8 · p. 12
• Across deep-search and coding tasks, SALE improves pass@1 over the strongest single agent while reducing cost and shifting substantial workload away from the largest model.These gains persist on the most complex tasks, as smaller agents adaptively take on more work over time.Source: §8 · pp. 12–13
• SALE has heterogeneous agents bid with short strategic plans, scores bids using a cost–value objective, and refines them through shared auction memory.It operates at test time on off-the-shelf models without training a separate router, with negligible additional inference overhead beyond executing the final trajectory.Source: §8 · p. 12
• SALE treats capability as an emergent property of systems that allocate work, price compute, and let agents learn from one another and adapt their strategies.The conclusion presents coordination and division of labor as another lever for progress alongside scaling individual models.Source: §8 · p. 13

Domain scope

• Experiments cover only deep search and coding, so the findings may not generalize to other agentic applications.These domains exercise complementary capabilities and offer objective, automatable evaluation, but the authors suggest testing SALE on tasks such as data analysis or long-form report writing.Source: Limitations · p. 13

Model scope

• Experiments use Qwen3 models from 4B to 32B, leaving behavior across model families and substantially larger frontier models less thoroughly evaluated.The authors chose a controlled size ladder to isolate scale effects; preliminary Appendix M results show analogous complexity-sensitive allocations for Gemma 3 and sensible allocations when Llama 3.1 and Ministral 3 join a Qwen3 pool.Source: Limitations · p. 13

Auction memory

• The auction memory bank grows linearly with task count, and retrieval at much larger scales, across model-version upgrades, or under task-distribution shift was not studied.The authors recommend approximate nearest-neighbor indexing for larger memories and utility-based retention to prune stale or low-value records; both can be added without changing the auction mechanism.Source: Limitations · p. 13

Cost accounting

• Cost accounting covers language-model tokens but does not explicitly price tool calls, which may matter in deployments where tool costs are non-negligible.The cost term can incorporate strategy-derived tool features or a blended cost-per-step metric; Appendix D.5 preliminarily demonstrates shifting workload away from agents generating the most tool calls when a tool-latency penalty is added.Source: Limitations · p. 13

Broader Impacts

• The work examines how task complexity affects language-model-agent effectiveness and proposes coordination for allocating work across heterogeneous models.These are the contribution and framing stated for the broader impacts discussion.Source: Broader Impacts · p. 14
• SALE’s marketplace metaphor describes allocation among software agents, not human workers or policies about employment.Bids, competition, and learning from feedback motivate a mechanism-design perspective on multi-agent systems.Source: Broader Impacts · p. 14
• Approximately 52% less reliance on the largest agent and 35% lower overall inference cost are presented as efficiency gains that may support more sustainable AI deployment.The passage links these reductions to more efficient computational-resource use and says routing simpler tasks to smaller models can avoid degrading performance.Source: Broader Impacts · p. 14
• More efficient agentic systems could enable beneficial or harmful applications at greater scale, but the paper identifies no unique risks beyond those of underlying models and frameworks.The passage encourages practitioners to consider deployment contexts carefully.Source: Broader Impacts · p. 14

A Dataset

• HST-Bench is described through its source-dataset composition, complexity bins, and human solution-time annotation protocol.The appendix identifies these as the dataset construction details covered.Source: A Dataset · p. 21

A.1 Data Composition

• HST-Bench combines seven open-source benchmarks with Coding-MCQ, validates sampled test instances, and assigns them to five complexity bins using average human solution times.The source mix shifts from factual QA and Coding-MCQ at lower complexity toward multi-source search and harder coding tasks at higher complexity.Source: §A.1 · p. 21

A.2 Data Annotation

• Human solution times came from at least three independent, proficient annotators per task, with outlier filtering and high inter-annotator agreement.Agreement was CCI = 0.83 (95% CI [0.81, 0.85]) and Krippendorff’s α = 0.86 (95% CI [0.84, 0.87], p < 0.001); LeetCode Hard timings instead use published estimates.Source: §A.2 · p. 22

A.3 Annotators’ Guidelines

• Annotator guidelines standardize timing and correctness procedures, allowing web search for deep search and code execution in a Python shell for coding tasks.Annotators were instructed to solve tasks quickly without AI assistants and to report stopwatch completion times.Source: §A.3 · pp. 22–24

A.4 Coding-MCQ Examples

• Coding-MCQ examples illustrate short, low-complexity questions on core programming concepts, such as Python syntax and string operations.The dataset was designed to populate the lowest-complexity coding bin.Source: §A.4 · p. 25

B Estimated Cost of Running Models

• The estimated Qwen3 cost model scales token prices linearly with parameter count and has mean absolute deviation within 6% of advertised provider averages.It uses a 4:1 input-to-output token weighting; latency and throughput are outside the model’s scope.Source: B Estimated Cost of Running Models · p. 26, Table 4 · p. 26

C Environment, Prompts and Hyperparameters

• Experiments use the standardized, tool-augmented Agent Research Environment with its default configuration unless otherwise noted.The paper describes model hyperparameters, prompts, and implementation details for reproducibility.Source: C Environment, Prompts and… · p. 27

C.1 Model Hyperparameters

• All agents share a decoding configuration, and experiments ran on NVIDIA A100 and H100 GPU clusters with 40–80 GB of HBM per accelerator.Table 5 summarizes the decoding and batching hyperparameters.Source: §C.1 · p. 27, Table 5 · p. 27

C.2 Environment Hyperparameters

• ARE episodes end when the time or iteration budget is exhausted, and evaluation reports pass@1 for a single final solution.Search outputs are matched against ground truth, while coding solutions are evaluated for functional equivalence.Source: §C.2 · p. 27, Table 6 · p. 27
• For coding tasks, agents perform strategic planning without explicit fact extraction, while Table 7 summarizes each domain’s available tools and Python execution environment.The passage refers to the original ARE paper for exact tool interfaces, observation formats, and error handling.Source: §C.2 · p. 28

C.3 Retrieval Hyperparameters

• At each episode, agents retrieve winning and losing strategies from the eight most similar past tasks, a practical trade-off between coverage and context or latency overhead.The authors follow established practice in setting k = 8 (Dai et al., 2023; Wang et al., 2024; Rashid and Hakak, 2025).Source: §C.3 · p. 28

C.4 Prompts

• The judge scores plans on a 0–5 Likert scale; coding omits fact extraction, and refinement uses prior winning and losing plans as contrastive examples.The 0–5 scale had higher correlation with human scores in preliminary experiments; the refinement template is similar to Alazraki et al. (2025).Source: §C.4 · p. 28

D.1 Cost and Value Function Design

• Spearman’s ρ = 0.39 and Pearson’s r = 0.36 (both p < 0.001) link strategy length to realized trajectory length, and including |s_t,i| improves development set accuracy.Removing strategy length from the cost function degrades performance in ablations.Source: §D.1 · p. 33
• Normalized per-token entropy signals strategy information content, while the full judge jury, including self-judgment, performs best on validation and removing any judge reduces accuracy.Learned weights calibrate each judge’s influence.Source: §D.1 · p. 33

D.2 Min-Max Formulation

• The min-max scoring weights are learned with a mixed-integer linear program, whose big-M constraints activate the cost-minus-value score only for the selected agent.M = 104 exceeds the largest observed absolute cost-minus-value score by at least two orders of magnitude.Source: §D.2 · p. 33

D.3 Implementation Details

• Entropy receives the highest learned weight in both domains, while the relative importance of judge models differs between deep search and coding.Smaller-model judgments receive higher weight for deep search; for coding, the 32B agent contributes most to the value estimate.Source: §D.3 · p. 34

D.4 Alternative Formulations

• Z-score normalization lowers accuracy and disproportionately selects cheaper models, with some learned value weights becoming negative and penalizing quality signals.The authors attribute this to losing the natural scale differences among cost, entropy, and judge scores; unconstrained, jointly learned weights calibrate these scales without manual normalization or thresholds.Source: §D.4 · p. 34

D.5 Tool-Latency Cost Extension

• As β grows, the 32B agent’s overall share drops from 23% to 11% (a 52% relative reduction), while the 14B agent rises from 18% to 28%.For deep search, the 32B share drops from 20% to 8%; the MILP converges to optimal at every β. The extension adds a tool-call latency penalty to cost while leaving the value function unchanged.Source: Table 9 · p. 35, §D.5 · p. 35

E.1 Algorithm; E.2 Contrastive Example Selection

• The algorithm collects initial bids, selects a provisional winner, and lets cheaper agents refine strategies using retrieved winning and losing examples from auction memory.Auction selection supplies the contrastive signal without an external success evaluator; winning strategies balance cost and value and tend to come from smaller models with competitive value, making them feasible examples for cheaper agents.Source: §E.1 · p. 36, §E.2 · p. 36

E.3 Refinement Patterns

• Structural elements of refined winning strategies recur consistently across search and coding tasks.Table 10 reports the prevalence of patterns in selected refined plans, and notes that patterns are not mutually exclusive.Source: §E.3 · p. 37, Table 10 · p. 37

E.4 Representative Examples; E.4.1 Deep Search Examples

• In deep-search examples, refinement adds clearer tool and argument specifications, trusted sources, and intermediate cross-checks to the strategies.The unrefined strategies are rejected and the refined strategies selected, yielding successful downstream answers.Source: §E.4 · p. 37, §E.4.1 · pp. 37–39

E.4 Representative Examples; E.4.2 Coding Examples

• In coding examples, refined strategies clarify function signatures and the requested artifact, and add systematic testing and edge-case coverage.The examples cover parentheses validation, magic-square testing, comb sort, and summing common divisors.Source: §E.4 · p. 37, §E.4.2 · pp. 39–41

E.5 Refinement and Auction Overhead

• 669 additional tokens per task for deep search and 1,042 for coding leave token overhead under 1% of total consumption; auction overhead is approximately 1% of wall time.The wall-clock comparison is against 5–10 minutes on average to execute the final agentic trace; the auction-phase measurement includes refinement on 100 stratified HST-Bench tasks.Source: §E.5 · p. 41, Table 11 · p. 41

F Significance Tests

• Memory-based methods were evaluated across five random test-set permutations, while baselines without memory used one run; significance used two-tailed one-sample t-tests and bootstrap confidence intervals.The bootstrap intervals use 10,000 resamples, comparing sale’s five runs against each baseline’s single-run reference value.Source: F Significance Tests · p. 42, Table 13 · p. 42
• sale significantly reduces $/Mt versus the best single agent, carrot, TO-Router, and FrugalGPT across all conditions; versus wtp, it is cheaper on deep search but costlier on coding.On coding, wtp's lower cost comes with substantially degraded performance.Source: F Significance Tests · p. 42
• In coding bins, pass@1 was nonsignificant for FrugalGPT at τ(t) ≤0.1 (p = .208), best single agent and TO-Router at ≤60 (p ≈.057), and carrot at ≤0.5 (p = .339).The corresponding $/Mt comparisons remained strongly significant for FrugalGPT and for best single agent and TO-Router; carrot’s marginal pass@1 advantage over sale was not significant.Source: F Significance Tests · p. 42
• On the full test set, sale has higher pass@1 than all baselines across both task types (all |t| > 5, p < .005), confirmed by bootstrap confidence intervals.Table 13 reports bootstrap 95% confidence intervals from 10,000 resamples.Source: Table 12 · p. 42, Table 13 · p. 42, F Significance Tests · p. 42

G.1 Baseline Implementation Details

• WTPRouter uses 50 cosine-distance nearest neighbors over sentence embeddings to predict model performance, then selects by a tuned willingness-to-pay trade-off.The reported configuration uses all-MiniLM-L6-v2 embeddings; the willingness-to-pay parameter is tuned on deep-search and coding training sets.Source: §G.1 · p. 43
• CARROT freezes a RoBERTa backbone and trains a classification head to predict candidate-model success probabilities.The development-set choice is µ = 0; larger µ values routed almost exclusively to the smallest agent, hurting performance without meaningful cost savings.Source: §G.1 · p. 43
• TO-Router predicts model-performance scores with a fine-tuned BERT encoder, while FrugalGPT combines correctness scoring with cascade optimization (Stripelis et al., 2024; Chen et al., 2024).TO-Router uses soft labels and inverse class-frequency weighting; FrugalGPT optimizes model ordering and acceptance thresholds.Source: §G.1 · p. 44

G.2 Pareto Frontier vs. Baseline Routers

• SALE shifts the deep-search Pareto frontier outward at every τ and improves coding trade-offs in all bins except τ ≤0.5, where CARROT is comparable.WTP can achieve lower coding costs, but only with substantial accuracy degradation.Source: §G.2 · pp. 44–45

H.1 Oracle Router

• The oracle selects the smallest agent that answers correctly, defaults to the smallest if all fail, and therefore gives a lower bound on spend.When many tasks are unsolved, its cost can be artificially low because choosing an agent cannot change accuracy; those low-accuracy-bin costs are not representative operating points.Source: §H.1 · p. 46, Table 18 · p. 46
• At τ(t) ≤0.1, oracle–SALE pass@1 is 97.5 vs. 91.3 for deep search and 98.8 vs. 98.3 for coding.At τ(t) ≤60, the gap widens: deep search is 25.0 vs. 16.3 and coding is 34.2 vs. 26.1. The oracle–best-single-agent gap also grows with complexity, peaking in medium–high bins.Source: §H.1 · p. 46, Table 18 · p. 46
• SALE’s oracle accuracy gap is 10.6 points on deep search (67.3 vs. 77.9) and 7.3 points on coding (61.1 vs. 68.4).Baseline gaps are 14.1–16.9 points on deep search and 10.0–18.3 on coding. SALE’s deep-search cost is 0.21 versus the oracle’s 0.07, lower than any baseline; WTP costs $0.11 on coding but has an 18.3-point accuracy gap.Source: Table 18 · p. 46, §H.1 · p. 46

H.2 Routing Diagnostics

• The routing audit compares SALE’s single-run selections with an oracle and classifies decisions as correct, over-escalation, under-escalation, or unavoidable.Over-escalation means selecting a larger, more expensive agent than necessary; under-escalation means a smaller agent fails when a larger one would succeed; unavoidable means no agent succeeds.Source: §H.2 · p. 47
• In deep search (n = 354), 65.1% of 14B and 61.0% of 32B selections could have been handled by 4B, while 4B selections were correct 54.0% overall.The row-normalized confusion matrix compares SALE’s selections with the oracle router.Source: §H.2 · p. 47, Figure 7 · p. 48
• In coding (n = 399), 55.2% of 14B and 31.5% of 32B selections could have used 4B, with another 25.0% of 32B selections mapping to 8B; 4B diagonal accuracy was 69.6%.The row-normalized confusion matrix compares SALE’s selections with the oracle router.Source: §H.2 · p. 47, Figure 8 · p. 48
• At τ ≤0.1 in coding, 4B routing is 100% correct and every 14B/32B selection could use 4B; at τ ≤60, unavoidable outcomes reach 50–78%, with 4B under-escalation at 25%.Unlike deep search, hard coding tasks retain variation in required agent capability.Source: §H.2 · p. 47, Figure 8 · p. 48
• Across both domains, SALE’s main improvement opportunity is reducing low-complexity over-escalation; its conservative routing preserves accuracy, and under-escalation remains rare.When SALE does route to 4B, diagonal accuracy is 54.0% for deep search and 69.6% for coding, suggesting it identifies easy tasks but triggers 4B too infrequently.Source: §H.2 · p. 47

I.1 Cost–Value Function Ablations

• Removing any cost–value term lowers pass@1 on both task types; all removals raise deep-search cost, while price and entropy removals slightly lower coding cost at accuracy’s expense.Without jury scoring, deep-search pass@1 falls by 3.6 on average and by 7.6 at τ ≤12.5; coding is most sensitive to price and strategy length at τ ≤2.5 (−4.2 and −5.5, respectively). At τ ≤0.1 for deep search, removing entropy produces the lowest cost but sacrifices accuracy.Source: §I.1 · p. 49

I.2 Jury Ablations

• Replacing the full jury with a single judge reduces overall Pass@1 from 67.3 to 65.0–66.4 for deep search and from 61.1 to 59.0–60.5 for coding, while raising deep-search cost from 0.21 to 0.25–0.32 $/Mt.No single judge leads on both task types: the 8B judge has the highest deep-search accuracy (66.4), while the 14B leads on coding (60.5).Source: Table 20 · p. 50, §I.2 · p. 50
• Removing any jury member lowers pass@1; excluding the 4B judge causes the largest coding drop (−2.9) and raises deep-search cost from 0.21 to 0.28 $/Mt.Removing the 14B judge has the smallest deep-search impact, with accuracy falling to 66.9 from 67.3.Source: §I.2 · p. 50, Table 21 · p. 51
• Removing peer-judgment lowers coding pass@1 to 48.7, whereas removing self-judgment leaves SALE near the best single-agent baseline but below full SALE on both task types.Self-judgment alone can match or exceed the best single agent on simple tasks, but deep-search pass@1 falls to 5.6 versus 12.5 at τ ≤60; removing peer-judgment lowers deep-search cost to 0.07 from 0.21 $/Mt.Source: §I.2 · p. 50, Table 22 · p. 51

I.3 Memory Ablations

• Enabling memory lowers the 32B agent’s allocation share from 35% to roughly 17% and raises refined-plan jury scores by about 0.3 points; a 10-task TTL window performs comparably to a 99-task window.Increasing memory size beyond M > 0 yields only small further improvements within seed-level noise (σ = 1.7–2.6 percentage points); the 10-task window has a 21% 32B share versus 18% at W = 99.Source: Table 23 · p. 52, Table 24 · p. 52, §I.3 · p. 52

J Jury Validation

• SALE validates jury scores through two complementary analyses: alignment with blind expert annotations of plan quality and predictiveness of downstream task success.The alignment analysis uses blind expert annotations on a stratified subset of 100 HST-Bench plans.Source: J Jury Validation · p. 53, §J.1 · p. 53

J.1 Alignment with Human Judgments

• Jury scores correlate strongly with expert human plan-quality ratings across complexity bins (ρ ∈[0.74, 0.82]), with no meaningful degradation as task complexity increases.The Spearman rank correlations compare average jury scores with human scores within each complexity bin.Source: Table 25 · p. 53, §J.1 · p. 53

J.2 Predictiveness of Task Success

• A point-biserial correlation of 0.36 (p < 0.01) indicates jury scores predict task accuracy and falls in the good-discrimination range (Ebel and Frisbie, 1991).The authors describe this as converging evidence, alongside human-alignment results, that jury scores reflect plan quality and predict downstream success.Source: §J.2 · p. 53

K Agent Selection Over Time

• Refined strategies were generated on 92% of deep-search and 93% of coding samples by 4B, 91% and 92% by 8B, and 62% and 77% by 14B.Cheaper agents have more opportunities to refine because an agent enters memory-based refinement only when a more expensive agent provisionally wins.Source: K Agent Selection Over Time · p. 54
• As auction memory accumulates, the 4B selection share rises across domains, 8B rises for deep search but stays relatively flat for coding, and 14B drifts downward.The cumulative curves are more volatile early because they average over fewer tasks; they smooth as more tasks accumulate.Source: K Agent Selection Over Time · p. 54, Figure 9 · p. 54

L Qualitative Performance Comparison

• The qualitative section presents test-set examples in which smaller agents succeed while 32B fails, using strategies rather than final answers to illustrate signs of failure.The authors note that full traces span tens of thousands to millions of tokens and describe telltale strategic signs including task misunderstanding, excessive complexity, and hallucinated details.Source: L Qualitative Performance Comparison · p. 55

L.1 Performance Comparison on Deep Search Tasks

• In deep search, 32B sometimes skips tool use and answers from parametric memory, or over-engineers straightforward questions with tangential or excessive queries.The passage gives examples of an answer appearing in the initial strategy, pursuing a musician’s date of death, and issuing sequential queries that lead away from the answer.Source: §L.1 · p. 55
• The 32B agent can also let partial or related parametric knowledge interfere with search, conflating a legal case or introducing ambiguity between a series and its sequel.Source: §L.1 · p. 55

L.2 Performance Comparison on Coding Tasks

• The 32B agent over-engineers simple coding tasks, ignores constraints and verification, and misses edge cases, producing errors that simpler approaches avoid.Examples include planning recursion for fixed two-level nesting, proposing an unavailable library, skipping Python-shell verification, using incorrect shortcuts, and mishandling the no-duplicate case.Source: §L.2 · p. 58
• For the LCM connected-components problem, the solution filters and sorts values, uses disjoint-set union to connect valid pairs, and is described as efficiently handling constraints and counting components correctly.The described approach processes multiples and checks pairs whose LCM is within the threshold; values above the threshold count separately.Source: §L.2 · p. 61
• The first-duplicate coding solution uses O(n) time and O(n) space by tracking seen elements while iterating through the array.It returns the first repeated element encountered.Source: §L.2 · p. 65

M.1 Auction Transfer to Gemma 3

• 80% of work on the hardest coding tasks went to Gemma 27B, while easier tasks were allocated more evenly across agent sizes.The Gemma-only auction converged and produced complexity-sensitive allocations matching those observed for Qwen3.Source: §M.1 · p. 66
• Across size-matched Gemma 3 and Qwen3 agents, strategy-length distributions had Wasserstein distance as low as 9 tokens on average for the largest models, jury scores correlated at Spearman ρ ∈[0.45, 0.76] (all p < 0.001), and entropy at Spearman ρ ∈[0.46, 0.60] (all p < 0.001).Despite different absolute entropy scales, the MILP compensates using its learned weight wh; the two families’ juries converge on the same task-level quality signal.Source: §M.1 · p. 66

M.2 Heterogeneous Agent Pool

• The frozen Qwen3 auction weights gave Llama 3.1 8B and Ministral 3 8B non-trivial workload shares that decreased with task complexity.The added agents were bidders only; the Qwen3 jury and learned MILP weights were unchanged, and no retraining was performed.Source: §M.2 · p. 66, Table 26 · p. 66
• Llama jury mean 3.48 vs Qwen3 8B 3.56, and per-task cost-minus-value scores correlated with Qwen3 8B rankings (Spearman ρ = 0.40, p < 0.01).Together with the Gemma 3 results, these findings indicate that SALE’s strategy-text, entropy, and peer-jury signals are not family-specific and support coordination across architectures and training recipes.Source: §M.2 · p. 66