PaperlayerView exact revision on arXiv

Source-linked AI summary

SpecEvo: Speculative Evolution with Large Language Models for Cost-Efficient Scientific Discovery

Ha Minh Hieu, Tongyao Zhu, Do Xuan Long, Min-Yen Kan

arXiv:2610.28706v1cs.AI

TL;DR

LLM-driven evolutionary search can be costly and may underuse evidence from failed, redundant, or saturated trials. SpecEvo coordinates lightweight exploration and frontier-model interventions through continual consolidation of search evidence. Across 176 tasks, it matches or outperforms strong baselines, with lower API expenditure under a three-hour wall-clock budget.

  • Problem

    Many evolutionary-search systems use one backbone for candidate generation and higher-level decisions, leaving routine frontier-model proposals costly and lightweight proposals without reusable guidance liable to revisit strategies or errors.

    Source: §1 · pp. 1–2

  • Method

    SpecEvo coordinates parallel lightweight Speculators and an Advisor with a sparsely invoked frontier Navigator, using accumulated search evidence and stagnation to guide proposals and interventions.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    Across 176 scientific and algorithmic discovery tasks, SpecEvo matches or outperforms strong baselines under matched budgets, with 50%–80% lower API expenditure under the three-hour wall-clock budget.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The results support using accumulated search evidence to guide lightweight exploration and frontier-model intervention across scientific and algorithmic discovery tasks.

    Source: §5 · p. 10

    Wall-clock efficiency depends partly on available parallelism, and the reported cost reductions primarily reflect LLM API expenditure in the specified computing environment.

    Source: A LIMITATIONS · p. 15

Abstract

from arXiv · show

LLM-driven evolutionary search is a promising paradigm for scientific discovery, but existing methods remain costly and slow, often applying expensive reasoning uniformly while underusing evidence from failed, redundant, and saturated trials. We introduce Speculative Evolution (SpecEvo), a multi-LLM framework that coordinates lightweight and frontier models through continual consolidation of search evidence. Lightweight models power both parallel Speculators, which generate and refine candidate programs, and an Advisor, which distills accumulated successes, failures, and saturated directions into persistent guidance for subsequent exploration. At sparse checkpoints, a frontier-model Navigator inspects the evolving population and intervenes through synthesis, surgical refinement, or reframing according to search stagnation. This speculate-then-consolidate workflow delegates routine exploration and evidence consolidation to inexpensive models while reserving frontier reasoning for strategic interventions whose benefits extend across many subsequent low-cost evaluations. Across 176 tasks spanning four scientific and algorithmic discovery domains, SpecEvo matches or outperforms frontier-model baselines while reducing API cost by up to 80%, achieves up to 1.5x greater evaluation efficiency than cost-efficient baselines on equation discovery, and improves over the strongest CO-Bench baseline by 3.8%. These results support trajectory-conditioned model coordination as an effective approach to cost-efficient scientific discovery.

1 INTRODUCTION

LLM-driven evolutionary search can produce strong discoveries, but single-backbone systems face a cost–quality trade-off and may revisit similar strategies without reusable guidance. SpecEvo coordinates lightweight exploration and frontier-model intervention using accumulated search evidence.

1 INTRODUCTION

  • Single-backbone systems can make routine proposals costly with frontier models, while lightweight models may reduce quality or repeat strategies and implementation errors without reusable guidance.The introduction frames accumulated successes, failures, and saturation as evidence that could inform future lightweight proposals and frontier intervention.Source: §1 · pp. 1–2
  • SpecEvo combines parallel lightweight Speculators, a lightweight Advisor, and a sparsely invoked frontier-model Navigator.The Advisor consolidates successes, failures, and saturated directions; Navigator interventions use stagnation signals to select among synthesis, refinement, and reframing.Source: §1 · p. 2
  • The evaluation spans 176 tasks across mathematical discovery, systems optimization, combinatorial optimization, and scientific equation discovery.It combines matched-budget comparisons with model-allocation studies, component ablations, and analyses of search dynamics.Source: §1 · p. 2

2 RELATED WORK

Prior evolutionary-search work has focused on preserving alternatives and using evaluation feedback to improve search decisions. Cost-efficient model-allocation methods address cheap–strong coordination, while SpecEvo continually consolidates search evidence to guide exploration and intervention.

LLM-Guided Evolutionary Search

  • Evolutionary-search systems preserve alternatives through quality-diversity archives or strategy-level memory, and use evaluation feedback to improve search decisions.OpenEvolve and SeaEvo retain alternatives; GEPA, AdaEvolve, and EvoX use feedback in different ways.Source: §2 · pp. 2–3

Cost-Efficient LLM Allocation

  • Cost-efficient allocation methods route between cheap and strong models, but SCOPE uses fixed module assignments and LEVI ties frontier use to predefined roles.RelayEvolve makes a one-time handoff, ending lightweight exploration without consolidating discarded or exhausted trajectories into reusable guidance.Source: §2 · p. 3
  • SpecEvo continually consolidates accumulated evidence to guide both lightweight exploration and state-dependent frontier-model intervention.This contrasts with allocation approaches whose decisions do not adapt to evolving search state or broader progress and saturation evidence.Source: §2 · p. 3

3 METHODOLOGY

SpecEvo applies asymmetric computation to search trajectories: inexpensive exploration supplies evidence, while periodic consolidation turns that evidence into guidance and strategic proposals. Its workflow maintains an archive of diverse candidates and coordinates continuous lightweight exploration with slower Advisor and Navigator updates.

3.1 MOTIVATION: FROM SPECULATIVE DECODING TO SPECULATIVE EVOLUTION

  • The StC workflow uses inexpensive exploration to gather evidence and periodic consolidation to produce guidance and strategic proposals for later exploration.After the prescribed budget, the highest-scoring program in the final archive is returned.Source: §3.1 · p. 3
  • The Navigator generates five sequential seed programs, and the Speculator pool expands each executable seed into 20 lightweight variants for the initial archive.The archive uses a MAP-Elites-style design, preserving competitive programs across structural–semantic niches rather than only global fitness ranks.Source: §3.1 · pp. 3–4

3.2 THE SPECULATE-THEN-CONSOLIDATE WORKFLOW

  • The archive refits its PCA basis and normalization every 50 completed evaluations, then re-anchors niche sites to the evolving population.Candidates fill open slots or replace a niche incumbent only when they strictly improve its fitness; refitting effects are reported in Table 3.Source: §3.2 · p. 4
  • Stagnation rises as global-best improvements or archive admissions become infrequent, broadening Speculator parent sampling from leading elites toward lower-ranked archive regions.The signal reaches one after 100 evaluations without a global-best improvement or 20 without an archive admission; rank selection avoids dependence on task-specific score scales.Source: §3.2 · p. 5
  • At fixed checkpoints, the Navigator uses synthesis during steady progress, surgical correction under moderate stagnation, and reframing under stronger stagnation.It sees complete code for selected incumbents plus summaries from five additional niches; a valid intervention seeds five nearby Speculator variants.Source: §3.2 · p. 5
  • The Advisor periodically distills working niches, saturated directions, and recurring execution failures into revisable guidance of at most 500 words.Its guidance is included independently in each Speculator prompt with probability 0.35, leaving most proposals without explicit Advisor guidance.Source: §3.2 · pp. 5–6

4 EXPERIMENTS

Across the evaluated domains and budgets, SpecEvo performs strongly while using less API cost than frontier-heavy baselines. Its results and ablations point to trajectory-conditioned coordination and complementary exploration and consolidation mechanisms.

4 EXPERIMENTS

  • Across 176 tasks in mathematics, systems optimization, combinatorial optimization, and equation discovery, the experiments compare evolutionary baselines and heterogeneous model-allocation strategies.The evaluation includes matched-budget comparisons, controlled model-allocation studies, component ablations, and search-dynamics analyses.Source: §4 · p. 6

4.1 MAIN RESULTS

  • 3.8% higher than the strongest competitor on aggregate CO-Bench, SPECEVO also leads most mathematical settings, all systems settings, and over 50% of problems.Figure 4 presents the CO-Bench performance comparison.Source: §4.1 · pp. 6–7, Figure 4 · p. 7
  • Approximately 50%–80% lower API cost than frontier-heavy baselines accompanies SPECEVO’s strong results across domains and tested model configurations.Figure 5 shows stronger solutions reached earlier at lower API cost; under $10, SPECEVO beats the strongest baseline, with a 5.12% gain on LLM-SQL.Source: §4.1 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7
  • On LSR-Synth, competing methods require approximately 43%–65% more evaluations to match SPECEVO’s performance at evaluation 500.The comparison uses an identical 500-evaluation budget, and SPECEVO achieves the lowest NMSE across all eight domain–split settings.Source: §4.1 · p. 7, Table 2 · p. 9
  • Under 500 model calls, SPECEVO ranks first on most tasks and never below second, using the frontier model for 2.4% of calls versus RelayEvolve’s 89.0%.The table caption reports the best performance on five of six tasks.Source: §4.1 · p. 7, Table 1 · p. 8
  • SPECEVO maintains higher population diversity than its single-prompt ablation and LEVI, while its stagnation rate is 17.5% versus LEVI’s 22.4%.Frontier-model contribution is 15.5% for SPECEVO versus 10.2% for LEVI, counting direct frontier outputs and lightweight offspring with frontier-generated parents.Source: §4.1 · p. 8, Figure 7 · p. 8, Figure 8 · p. 8

4.2 ABLATION STUDIES

  • Specialized Speculator prompts produce the largest exploration-side benefit, while random Navigator modes and fixed Advisor guidance underperform adaptive, search-conditioned feedback.Fixed advice can perform worse than removing the Advisor on Heilbronn Triangle.Source: §4.2 · p. 9, Table 3 · p. 9
  • On Circle Packing Rectangle and EPLB, Navigator interventions shift search strategies, while Advisor instructions help subsequent variants avoid recurring implementation errors.The case studies describe REFRAME and SURGICAL interventions followed by low-cost Speculator refinement.Source: §4.2 · p. 9, Figure 10 · p. 9

5 CONCLUSION

SPECEVO coordinates inexpensive exploration and evidence consolidation with sparse frontier-model reasoning, matching or outperforming strong baselines across 176 discovery tasks while reducing API expenditure under a three-hour budget.

5 CONCLUSION

  • 50%–80% lower API expenditure under the three-hour wall-clock budget accompanies performance that matches or exceeds strong baselines under matched wall-clock, API-cost, and evaluation budgets.The evaluation spans 176 scientific and algorithmic discovery tasks.Source: §5 · p. 10
  • Parallel lightweight Speculators explore, an Advisor distills successes, failures, and saturated directions into persistent guidance, and a frontier-model Navigator intervenes at fixed checkpoints.Navigator interventions depend on stagnation.Source: §5 · p. 10
  • The reported gains across backbone configurations and matched evaluation budgets support using accumulated search evidence to guide sparse frontier interventions and subsequent lightweight refinement.Source: §5 · p. 10

AI USE STATEMENT

The statement describes how generative AI was used in the research and writing, and outlines the supplementary material’s coverage of methods, evaluations, and analyses.

AI USE STATEMENT

  • Language models generate, refine, and analyze candidate programs in SPECEVO and its baselines, while text-embedding-3-small supports archive descriptors and the diversity metric.GPT-4o judges mathematical equivalence to ground truth for LSR-Synth under the LLM-SRBench protocol.Source: AI USE STATEMENT · p. 10
  • Generative AI tools assisted manuscript editing, clarity, readability, and rephrasing; the authors reviewed and revised the text and take responsibility for the final content.Source: AI USE STATEMENT · p. 10

Appendix

  • The appendix documents limitations, related work, baselines, algorithms, metrics, benchmarks, detailed results, prompts, and analyses of programs and Advisor guidance.Appendix I includes analyses of recurring failure modes and algorithmic mechanisms.Source: Appendix · p. 14

A LIMITATIONS

The limitations concern computing infrastructure, evaluator costs, and model-generated descriptions; the related-work discussion contrasts SPECEVO’s persistent division of labor with existing resource-allocation approaches.

A LIMITATIONS

  • Wall-clock efficiency depends partly on Speculator parallelism, and reported API savings reflect the specified computing environment rather than total computational cost.The experiments use four concurrent Speculators; worker count, CPU capacity, memory bandwidth, network latency, and API rate limits can change candidate throughput.Source: A LIMITATIONS · p. 15
  • The evaluated benchmarks have relatively lightweight CPU evaluators, so fixed evaluation counts do not equal fixed compute budgets when evaluation costs vary.GPU training, large-scale inference, expensive simulation, or real-world experiments may multiply resource use and unsuccessful-trial costs as workers increase.Source: A LIMITATIONS · p. 15
  • Model-generated strategy descriptions may misrepresent code and distort semantic distances, affecting niche assignment, parent selection, and population summaries; sensitivity has not been systematically quantified.The authors identify controlled description perturbations and code-grounded verification or execution-derived descriptors as future work.Source: A LIMITATIONS · p. 15

B EXTENDED RELATED WORK

  • SPECEVO assigns candidate generation, experience consolidation, and strategic intervention to persistent roles for models with different costs.The discussion distinguishes this coordination from systems that primarily organize strategy families or evolve search strategies.Source: B EXTENDED RELATED WORK · pp. 15–16
  • Prior resource-allocation methods address model selection, search effort, handoff, or mutation roles, but the discussion identifies limits in persistent roles and continual trajectory-wide evidence consolidation.Examples include mutation-level selection, one-time cheap-to-strong transfer, and paradigm shifts without continual aggregation for subsequent small-model search.Source: B EXTENDED RELATED WORK · p. 16

C BASELINE DETAILS

The appendix specifies common baseline protocols and controls, then details SPECEVO’s archive, asynchronous workflow, and Navigator context. These mechanisms define how candidates are compared, represented, and routed during search.

C BASELINE DETAILS

  • All baselines use native search logic and released implementations with shared task specifications and evaluators; wall-clock tests share hardware, with fixed API-cost and evaluation-budget comparisons as complements.For baseline implementations with native parallel search, four workers match SPECEVO.Source: C BASELINE DETAILS · p. 16
  • API-cost and wall-clock accounting include auxiliary LLM calls, while each evaluated candidate consumes an evaluation even if it fails parsing, violates constraints, or times out.Each method returns its highest-scoring valid program before its stopping condition.Source: C BASELINE DETAILS · pp. 16–17
  • Baseline implementations retain distinct search mechanisms, including island archives, Pareto-aware reflection, adaptive island allocation, strategy evolution, role-aware routing, and a terminal frontier handoff.The descriptions cover OpenEvolve, GEPA, AdaEvolve, EvoX, LEVI, and RelayEvolve, respectively.Source: C BASELINE DETAILS · p. 17
  • Four controlled allocation strategies share an asynchronous backend and differ only in model assignment: all-cheap, fixed-switch, random, and UCB1 bandit.Bandit reward is non-negative global-best improvement; invalid or non-improving proposals receive zero reward.Source: C BASELINE DETAILS · p. 17

D.1 BEHAVIORAL DESCRIPTOR CONSTRUCTION

  • Archive descriptors combine eight AST-derived structural statistics with a semantic embedding of the strategy description, projected onto eight PCA directions and standardized by block.The structural statistics include AST depth, cyclomatic complexity, loop counts and nesting, and counts of conditionals, functions, comprehensions, and calls.Source: §D.1 · p. 18
  • The archive admits at most 50 incumbents, one per niche; before the first refit each admitted program opens a niche, and later candidates compete against the nearest incumbent.Refits every 50 evaluations rebuild descriptor geometry while preserving archive slots and niche statistics.Source: §D.1 · pp. 18–19

D.2 THE COMPLETE SPECEVO WORKFLOW

  • After frontier-seeded initialization and lightweight diversification, asynchronous Speculators generate candidates while Navigator, Advisor, and archive-refit operations follow independent evaluation schedules.Navigator interventions run as non-blocking background tasks; worker sampling uses stagnation-dependent parent selection and specialized variation prompts.Source: §D.2 · p. 19

D.3 NAVIGATOR INTERVENTION

  • The Navigator ranks niche elites, using full-code anchors for precise reasoning and description-only inspirations for broader mechanisms; mode-specific context supports synthesis, repair, or reframing.Anchors and inspirations are disjoint within each invocation.Source: §D.3 · pp. 19–20

D.4 ADVISOR INTERVENTION

The Advisor distinguishes improving niches from saturated ones, combines those signals with persistent failure memory, and distills compact guidance for later Speculator generations.

D.4 ADVISOR INTERVENTION

  • At each checkpoint, the Advisor tracks niche activity and incumbent movement to distinguish improving search from high-effort saturation without a task-specific improvement threshold.Activity includes unsuccessful evaluations; an active niche has incumbent movement, while a saturated niche has activity but no movement.Source: §D.4 · p. 21
  • The Advisor selects up to four highest-scoring niches with incumbent movement and up to three active niches without movement for complementary population views.The second set captures the most active saturated niches.Source: §D.4 · pp. 21–22
  • The two population views separate current solution quality from recent progress, so a high-scoring niche may be stalled while a lower-scoring niche may reveal a newly productive direction..Source: §D.4 · p. 22
  • Failure memory retains normalized error signatures throughout the run and supplies the Advisor with the eight most frequent signatures, breaking ties by recency.The prompt is bounded while earlier failures remain in the underlying history.Source: §D.4 · p. 22
  • The Advisor revises a compact note with WORKING mechanisms, SATURATED directions, and AVOID anti-patterns, using population views, recurrent failures, and previous advice.Algorithm 3 summarizes the intervention from extracting niche-level signals and failures through distilling guidance for subsequent generations.Source: §D.4 · pp. 20, 22
  • Guidance reaches each subsequent Speculator independently with probability ρA = 0.35, preserving a substantial fraction of independent exploration.The note is constrained to fewer than 500 words.Source: §D.4 · p. 22

E.1 AGGREGATE METRICS USED IN FIGURE 3

This section defines within-family aggregate performance and cost measures, then specifies diagnostics for population diversity, stagnation, and frontier-attributed improvements. It also describes the mathematical and systems optimization benchmarks used in evaluation.

E.1 AGGREGATE METRICS USED IN FIGURE 3

  • Within each benchmark family, aggregate performance and API cost are normalized against the strongest displayed mean, with tasks weighted equally despite different native scales.100% performance means matching the strongest displayed mean on every task; higher normalized performance and lower normalized cost are preferable, while detailed-table Best scores are not used in this aggregate.Source: §E.1 · p. 22

E.2 CODE DIVERSITY INDEX (CDI)

  • The Code Diversity Index is the Shannon entropy of normalized positive edge lengths in a minimum spanning tree over canonicalized program embeddings.Higher CDI means separation is distributed across more MST edges rather than dominated by one bridge; uniform rescaling of embedding distances leaves it unchanged.Source: §E.2 · p. 23

E.3 STAGNATION RATE

  • Stagnation rate measures the share of the full evaluation budget consumed by the run’s single longest global-best plateau.The plateau convention includes evaluations before the first new best and after the last new best.Source: §E.3 · p. 23

E.4 NEW BESTS TRACED TO THE FRONTIER MODEL

  • FrontierTrace is the fraction of new-best events directly produced by the frontier model or reached through lightweight refinement of frontier-generated programs.Initialization is excluded; the metric is missing when there are no new-best events and should be read with final performance and API cost because it counts events, not their magnitude.Source: §E.4 · pp. 23–24

F BENCHMARK DETAILS

  • Unless otherwise stated, candidates are executable programs evaluated by task-specific evaluators, and methods see only scalar scores and diagnostic feedback—not hidden test instances or ground-truth solutions..Source: F BENCHMARK DETAILS · p. 24

F.1 MATHEMATICAL OPTIMIZATION BENCHMARKS

  • The mathematical optimization evaluation uses seven executable tasks covering geometric packing, extremal point configurations, and signal processing.Each candidate constructs or refines a feasible solution, scored by a higher-is-better scalar objective.Source: §F.1 · p. 24

F.2 SYSTEMS OPTIMIZATION BENCHMARKS

  • The systems evaluation uses four real-world tasks from the AI-Driven Research for Systems benchmark suite, with every reported evaluator score oriented so larger is better.Some underlying objectives are minimization problems, but their evaluators convert them to higher-is-better scores.Source: §F.2 · p. 24

F.3 CO-BENCH

CO-Bench assesses algorithm-design agents across 36 real-world optimization problems, while LSR-Synth tests scientific equation discovery across four domains. The section also reports API spending and token use under a fixed three-hour budget.

F.3 CO-BENCH

  • CO-Bench contains 36 real-world optimization problems in eight categories, and its overall score is the unweighted mean of problem scores averaged across test instances.Under a three-hour wall-clock budget, at most 50 instances per problem are split 7:3 into training and test sets; hidden test instances and evaluator code are unavailable during search.Source: §F.3 · pp. 24–25
  • CO-Bench scores feasible solutions with task-specific higher-is-better normalization; failed, timed-out, or infeasible runs receive zero, while scores above 1 can exceed a recorded best-known reference.The benchmark retains special handling for zero-reference instances.Source: §F.3 · pp. 24–25

F.4 LSR-SYNTH

  • LSR-Synth combines known and novel synthetic terms to evaluate equation discovery across chemistry, biology, physics, and material science, using all 129 problems.Final evaluation reports NMSE on both in-distribution test ranges and out-of-distribution ranges to assess extrapolation.Source: §F.4 · pp. 25–26

G.1 BREAKDOWN OF API COST AND TOKEN USAGE

  • SpecEvo can consume more total tokens than several baselines while costing less because routine search is assigned to lightweight models and frontier calls are infrequent.The cited prices are $10.00 per million GPT-5 output tokens versus $0.30 for Qwen3-30B-A3B-Instruct-2507.Source: §G.1 · p. 27
  • Under the fixed three-hour budget, SpecEvo reduces API cost by approximately 49.9%–70.7% across four benchmark–backbone settings when LEVI is excluded.LEVI is treated separately because it also uses a lightweight model for routine search and reserves frontier calls for occasional paradigm shifts.Source: §G.1 · p. 27

G.2 DETAILED RESULTS FOR MATHEMATICAL DISCOVERY AND SYSTEMS OPTIMIZATION TASKS

Across mathematical and systems optimization, SpecEvo combines strong benchmark performance with lower API expenditure and robustness to frontier-backbone changes. On CO-Bench, it leads in multiple categories and wins more individual problems than it loses in every baseline comparison.

G.2 DETAILED RESULTS FOR MATHEMATICAL DISCOVERY AND SYSTEMS OPTIMIZATION TASKS

  • Across both frontier backbones, SpecEvo has the strongest or tied-best single-run result on six of seven mathematical tasks and leading or tied mean performance on five.Its advantage is most consistent on Heilbronn and MinMax; Signal Processing is the main exception, with noisier evaluation and competitive conventional baselines.Source: §G.2 · p. 28, Table 12 · p. 29
  • At least one SpecEvo configuration achieves best or tied-best mean performance on every systems task under both GPT-5 and Kimi-K2, with no single lightweight backbone dominating.Qwen2.5-7B reaches the performance ceiling on PRISM; across the closed loop, different inexpensive models carry out high-volume refinement and population-level evidence signals when redirection is needed.Source: §G.2 · pp. 28–29, Table 13 · p. 30
  • 34%–85% lower API expenditure on mathematical discovery and 51%–71% on systems optimization versus four frontier-heavy baselines, while SpecEvo also stays close to task ceilings when the frontier backbone changes.It generally improves on LEVI at comparable cost; the authors attribute the remaining quality difference mainly to integration of scarce frontier reasoning and accumulated evidence, rather than the model price gap alone.Source: §G.2 · p. 29, Table 12 · p. 29, Table 13 · p. 30

G.3 DETAILED RESULTS FOR EACH CO-BENCH PROBLEM CATEGORY

  • SpecEvo has the highest category-level score in five CO-Bench categories and ranks first on 20/36 problems, with Assignment nearly saturated and Facility Location more mixed.The five categories are Packing, Cutting, Scheduling, Routing, and Tree; the largest category advantages are on Cutting and Tree.Source: §G.3 · p. 30, Figure 14 · p. 30, Table 14 · p. 31
  • +0.0332 against LEVI to +0.0500 against OpenEvolve are the positive mean paired score differences across baselines, and SpecEvo wins more problems than it loses in all five comparisons.Bootstrap confidence intervals for the mean difference are positive for four of five baselines; AdaEvolve is the exception.Source: §G.3 · p. 30, Table 15 · p. 31, Figure 15 · p. 32

G.4 DETAILED RESULTS ON LSR-SYNTH

On LSR-Synth, SpecEvo is competitive across numerical and symbolic metrics, with paired task-level results favoring it over every baseline under ID and OOD evaluation. The supplied appendix passages also describe the Speculator, Navigator, and Advisor prompts and qualitative analyses of two search runs.

G.4 DETAILED RESULTS ON LSR-SYNTH

  • Numeric precision uses a 0.1 relative-error tolerance at every test point, while symbolic accuracy uses GPT-4o to judge expression equivalence after removing parameters and constants.NMSE and numeric precision are reported separately for ID and OOD; symbolic accuracy is reported once per domain.Source: §G.4 · p. 32
  • SPECEVO achieves the lowest NMSE in six of eight domain–split settings, leads numeric precision in chemistry, biology, and physics, and has the highest or tied symbolic accuracy in every domain.The NMSE result includes three of four OOD settings; biology is the exception, where LLM-SR has lower NMSE on both splits.Source: §G.4 · p. 32, Table 16 · p. 33
  • Against every baseline, SPECEVO wins 95–109 of 129 ID problems and 81–100 of 129 OOD problems, with all paired-difference confidence intervals above zero.The comparison covers four baselines; Figure 16 presents per-task score differences and Table 17 summarizes paired comparisons.Source: §G.4 · pp. 33–34, Figure 16 · p. 34, Table 17 · p. 34

H.1 SPECULATOR

  • The lightweight Speculator generates and refines variants using six operators, while requiring accepted candidates to provide executable code and an embeddable strategy description.Its shared prompt can include parent programs, archive inspirations, and optional Advisor lessons.Source: §H.1 · p. 35

H.2 NAVIGATOR

  • The frontier-model Navigator uses synthesis, surgical improvement, or reframing, selected according to search stagnation.Its prompts call for combining mechanisms, making one local change, or proposing a fundamentally different algorithmic family.Source: §H.2 · pp. 36–38

H.3 ADVISOR

  • The Advisor turns niche-level statistics and accumulated failures into short guidance for future mutation attempts, identifying working and saturated strategies and concrete errors to avoid.Its prompt draws on current search state, improving and saturated niches, accumulated failure knowledge, and previous advice.Source: §H.3 · p. 38

I QUALITATIVE ANALYSIS

  • The qualitative appendix examines one search run each for Circle Packing Rectangle and EPLB, tracing candidate rejection, algorithmic mechanisms, and Advisor guidance.It records every candidate, its producing role and model, execution status, and evaluator message.Source: I QUALITATIVE ANALYSIS · p. 38

I.1 CIRCLE PACKING RECTANGLE

The qualitative analyses show distinct dominant failure modes in the two tasks and describe how search mechanisms and accumulated failure guidance shape later candidates. On Circle Packing Rectangle, the final record combines mechanisms already present in the population; on EPLB, it comes from a lightweight-refined Navigator proposal.

I.1 CIRCLE PACKING RECTANGLE

  • Overlapping circles are the most common Circle Packing Rectangle failure, often caused by radius inflation or rescaling without rechecking distances; wrong output shape is the next most common class.The reported overlaps are not near misses, so relaxing the tolerance would accept almost none of them.Source: §I.1 · p. 39
  • The Circle Packing run’s rejection rate drops from initialization to the evolutionary phase, where Speculators receive Advisor notes; RelayEvolve rejection rates are roughly constant across run quarters.The passage notes that RelayEvolve components do not carry earlier failure information forward.Source: §I.1 · p. 39
  • Circle Packing failures differ by model tier: strong-model programs account for nearly all timeouts, while lightweight-model programs account for most indexing and shape errors.When call counts are comparable, the two tiers have similar rejection and overlap rates.Source: §I.1 · p. 39
  • Circle Packing programs commonly start from hexagonal layouts, refine overlaps, repair perimeter and radius constraints, and retain the best feasible configuration under a wall-clock guard.The described final best program combines damped pairwise separation, nearest-neighbor radius growth, backtracking enlargement, perimeter rescaling, and a final shrink pass.Source: §I.1 · pp. 39–40, Table 20 · p. 40

I.2 EPLB

  • On EPLB, indexing errors dominate: capacity arithmetic can produce out-of-bounds slot or replica indices even when the allocation policy is sound.Missing priority-queue imports and unmodded hash products that overflow integer ranges are also prominent failure causes.Source: §I.2 · p. 41, Table 21 · p. 41
  • EPLB rejection rates decrease through the run as accumulated Advisor guidance addresses missing imports, overflowing constants, and invalid slot indices.Those mechanical faults are largely absent later, while the same algorithmic ideas remain in use.Source: §I.2 · pp. 41, 43
  • EPLB programs allocate replicas using marginal-gain rules or load-threshold search, place them by round-robin or least-loaded-GPU strategies, and enforce capacity with repair and budget checks.The strongest programs combine threshold search with marginal-gain correction.Source: §I.2 · pp. 41–42, Table 22 · p. 42
  • The final EPLB record combines threshold-based replica allocation, marginal-gain correction, and hierarchical-capacity-aware placement in a lightweight structural crossover refined from a Navigator proposal.Later REFRAME interventions score below the incumbent, but the archive retains those programs as parents to increase diversity under stagnation.Source: §I.2 · p. 42

Related papers

Loading 2610.28706v1…