PaperlayerView exact revision on arXiv

Source-linked AI summary

Initialization Improves LLM-Driven Discovery

Mansi Sakarvadia, Marco Ciccone, Colin Raffel

arXiv:2610.00707v1cs.LGcs.AIcs.CL

TL;DR

LLM discovery harnesses iteratively refine past discoveries, but success varies with harness design, task, and model. The paper develops Modular and proposes parallel initialization of sequential optimization; high-performing initialization consistently improves discovery performance, including by 2–21% for state-of-the-art harnesses.

  • Problem

    The paper examines how discovery harnesses’ initial populations relate to downstream discovery quality, given that harness performance varies across tasks and LLMs.

    Source: §1 · p. 1

  • Method

    The authors develop 12 Modular harnesses and initialize sequential harnesses by generating a parallel pool, curating high-performing candidates, and then beginning sequential optimization.

    Source: §1 · p. 1, §5 · p. 7

  • Results

    Across task, LLM, and harness settings, initialization consistently improves discovery performance; state-of-the-art harnesses gain 2–21%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Early-iterate performance can predict downstream discovery success, supporting the use of high-performing initial populations.

    Source: §1 · p. 2

    Exploration does not guarantee recovery from poor initial discoveries, and unnecessary exploration may hurt initially high-performing trajectories.

    Source: §4 · p. 7

Abstract

from arXiv · show

Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optimize an objective. In this work, we study the relationship between the population of previous iterates and eventual discovery success. We generalize past work on harness design to develop a suite of 12 harnesses called 'Modular' and characterize their performance across 5 diverse discovery tasks, finding that discovery success is brittle and sensitive to harness design. We uncover mode collapse, characterized by a dramatic drop in the diversity of iterates, as a common failure mode. We find that popular state-of-the-art harnesses and diversity-inducing harness interventions, which aim to prolong this collapse, yield inconsistent gains. Our results instead uncover that the performance of early discoveries is predictive of eventual success. We therefore propose a universally applicable intervention that performs an initial stage of parallel exploration in order to initialize subsequent iterative optimization. Our method provides consistent gains across many harnesses and target applications, confirming the importance of initialization in LLM-driven discovery.

1 Introduction

LLM discovery harnesses manage a population of past discoveries that the model iteratively refines, but performance varies across harness designs, tasks, and models. The paper studies how discovery populations evolve and proposes initializing sequential optimization with a high-performing population.

1 Introduction

  • Across five tasks, parallel prompting beats some sequential harnesses, but top-performing harnesses differ, making harnessed discovery sensitive to task and LLM.Parallel prompting is compared with sequential harnesses under controlled inference-time compute.Source: §1 · p. 1
  • Harnesses show diminished discovery diversity over time, while diversity-oriented harnesses and interventions yield task- and LLM-dependent gains and do not consistently recover from poor initial discoveries.Early-iterate performance can predict downstream discovery success.Source: §1 · p. 2
  • Initialization consistently increases state-of-the-art harness discovery performance by 2–21% across tasks, LLMs, and harnesses.Source: §1 · p. 2

2 Background on LLMs as Optimizers

This section frames LLM-driven discovery as iterative optimization and describes the harnesses, metrics, and tasks used to study discovery trajectories. It examines general-purpose harness design and compares sequential with parallel discovery.

2.1 Sequential Discovery Harnesses

  • A discovery harness samples prior solutions, prompts an LLM to mutate them, evaluates new discoveries, adds qualifying results, and periodically prunes the active population.Modular exposes mutation and parent-sampling choices in 12 harnesses; the study also considers three popular state-of-the-art harnesses.Source: §2.1 · pp. 2–3

2.2 Discovery Metrics

  • Discovery trajectories are compared by their maximum task-specific score, token expenditure, and semantic diversity.Diversity is one minus the average pairwise cosine similarity of discovery embeddings; token expenditure is used because evaluations can require differing inference compute.Source: §2.2 · p. 3

2.3 Discovery tasks

  • The study repeats experiments across multiple LLMs and five optimization tasks, reporting a subset in the main text and full results in Appendix C.The tasks include cloud-transfer cost, deadline-driven scheduling, circle packing, traveling salesperson, and GSM8K prompt optimization.Source: §2.3 · p. 3
  • The task suite covers cloud-transfer cost, spot-instance job scheduling, packing 26 circles, routing among 100 cities, and GSM8K prompt optimization.The tasks respectively seek an algorithm, an arrangement, a shortest tour, or an instruction prompt that improves downstream performance.Source: §2.3 · p. 3
  • Sequential compute generally outperforms parallel discovery, although the best harness type varies by task and LLM.Both approaches use the same initial seed solutions.Source: Figure 2 · p. 4

3 Parallel vs. Sequential Discovery

The section tests whether sequential refinement outperforms independent parallel discovery. Across task–LLM settings, sequential refinement sometimes wins, but the best harness varies by pairing, and sequential trajectories lose diversity.

3 Parallel vs. Sequential Discovery

  • Sequential harnesses update the LLM’s context with recent discoveries, whereas parallel discovery independently mutates the same initial seed without changing context.The comparison uses the same initial seed iterate and token budget in both settings.Source: §3 · p. 4
  • Multiple Modular harnesses outperform parallel discovery on each task, but the winning variants vary by task–LLM pairing, making sequential discovery sensitive to harness choices.The comparison holds the initial seed iterate and token budget constant.Source: §3 · p. 4
  • Parallel schemes generally produce more diverse discovery populations, while sequential trajectories show declining candidate diversity over time.Sequential diversity is measured over a window; parallel diversity is computed over the full discovery set.Source: §3 · p. 4, Figure 3 · p. 5
  • Despite mode collapse, sequential refinement can outperform parallel discovery when a harness finds a high-performing mode.The paper hypothesizes that the in-context prior is reinforced during sequential discovery but absent in parallel discovery.Source: §3 · p. 5

4 Can Exploration Enable Better Discovery?

The section tests whether greater exploration improves discovery, comparing diversity-oriented harnesses and interventions with Modular baselines. It finds that greater diversity does not reliably improve scores, while early discovery quality predicts downstream performance.

4 Can Exploration Enable Better Discovery?

  • Both state-of-the-art harnesses and Modular harnesses with diversity interventions generally produce more diverse populations than baseline Modular harnesses.Table 2 reports average diversity across harnesses.Source: Table 2 · p. 5
  • State-of-the-art harnesses do not consistently outperform Modular harnesses, and the best-performing set varies across task and LLM settings.Table 4 compares SE, OA, and OE with Modular harnesses.Source: Table 4 · p. 6
  • Diversity interventions increase discovery diversity but have variable effects on quality, sometimes harming performance and varying by task and hyperparameter.The interventions include cosine-similarity thresholds, an LLM judge, and deduplication.Source: §4 · p. 6, Table 3 · p. 6
  • Initialized sequential discovery can outperform purely sequential and purely parallel discovery when seeded with a high-performing population.Figure 5 describes allocating part of the discovery budget to initialization; its dotted line indicates a predicted ideal ratio.Source: Figure 5 · p. 7
  • Exploration may help trajectories with poor initial candidates, but does not guarantee recovery and may hurt trajectories that already start strong.Distinguishing these scenarios is described as challenging.Source: §4 · p. 7
  • High-scoring CloudCast trajectories begin with greater diversity but eventually experience mode collapse.Figure 6 stratifies Modular trajectories by maximum score and plots windowed diversity over time.Source: Figure 6 · p. 7
  • The median score of the first 15 discoveries is associated with the trajectory’s maximum score, and high-performing harnesses tend to start with stronger populations.The relationship is shown across discovery harnesses, but no specific harness consistently performed well.Source: Figure 4 · p. 6, §4 · p. 7

5 The Importance of Initialization

The paper allocates part of a discovery budget to parallel exploration, then warm-starts sequential optimization with selected high-performing candidates. This initialization improves discovery relative to purely sequential or parallel approaches and also benefits state-of-the-art harnesses.

5 The Importance of Initialization

  • The method spends a fraction of the total budget on parallel discovery, then greedily selects a high-performing initial population for sequential refinement.Cosine-similarity filtering is used during selection; if unique candidates run out, remaining slots are filled with deduplicated high performers.Source: §5 · pp. 7–8
  • Across the 12 Modular variants, parallel initialization uniformly outperforms fully serial and fully parallel optimization in average maximum score.The experiments use five initialization budgets and an initial population size of 15.Source: §5 · p. 8
  • Initialized sequential discovery consistently outperforms purely sequential and purely parallel discovery when enough serial compute remains.Across the reported initialization settings, minimal diversity filtering generally performs well, while substantially more diverse initial populations have lower average performance.Source: §5 · p. 8
  • Detected convergence in parallel discovery consistently coincides with the highest-performing initialization budgets and can guide when to switch to sequential optimization.The paper suggests batched parallel discovery to enable early convergence detection.Source: §5 · p. 8
  • Post-initialization improves downstream discovery, with discovery scores increasing by 2%-21%; this suggests explicitly initializing a high-performing trajectory is beneficial because harnesses alone do not consistently do so.Source: §5 · p. 9

6 Related Work

Related work describes declining diversity in LLM outputs and strategies for using parametric knowledge and inference-time compute. This paper connects those themes to iterative discovery harnesses and the quality of initial discoveries.

6 Related Work

  • Prior work reports reduced LLM response diversity after RL-based posttraining and proposes prompting strategies to address it.The passage also notes that earlier outputs can influence motifs in later outputs during iterative language modeling.Source: §6 · p. 9
  • Prior research finds that prompting can use parametric knowledge and that additional inference-time compute can improve output quality.This work examines how discovery harnesses manage that compute and how initial discovery quality mediates its usefulness.Source: §6 · p. 9

7 Conclusion

Early discovery quality predicts eventual performance, and warm-starting harnesses with curated high-performing populations consistently improves their performance. The conclusion also identifies harness-design sensitivity and uncharacterized interactions between parametric and in-context knowledge.

7 Conclusion

  • Warm-starting discovery harnesses with high-performing initial populations consistently improves their performance.The proposed method is described as task-, LLM-, and harness-agnostic.Source: §7 · p. 9
  • The best-performing discovery trajectories can be predicted by their initial discovery quality across a wide range of task and LLM settings.Source: §7 · p. 9
  • Sequential refinement is beneficial but sensitive to harness design, while diversity-oriented choices do not predictably yield more performant discoveries.Source: §7 · p. 9
  • Future work should quantify parametric knowledge and examine how model priors interact with task-specific in-context priors.The current parallel-discovery analysis does not update in-context knowledge and relies on parametric knowledge.Source: §7 · p. 9

AI use statement

The authors report using generative AI for editing experimental software, modifying figures, formatting tables, rephrasing sentences, and editing code, while reviewing AI-assisted work.

AI use statement

  • Generative AI tools were used to edit experimental software, modify scientific figures, format LaTeX tables, rephrase sentences, and edit software code.Source: AI use statement · p. 10
  • The authors state that generative AI was not used for the listed research activities, including hypothesis development, methodology design, implementation of methods, or interpretation of results.Source: AI use statement · p. 10
  • One author reviewed LLM-generated code for correctness, and AI-refined figures were compared against human-generated originals.Source: AI use statement · p. 10

Reproducibility statement

The paper provides code and supporting task materials for reproducing its experiments, and describes a study spanning 15 discovery harnesses.

Reproducibility statement

  • The code base, harness details, parallel-discovery scheme, task instructions, initial seeds, and task code bases are identified as reproduction resources.Source: Reproducibility statement · p. 10
  • Experimental figures and tables report 95% confidence intervals, and main-text Table 1 results are replicated across additional task–LLM pairings in Appendix C.Source: Reproducibility statement · p. 10

A Discovery Harnesses

A.1 Modular Harnesses

The appendix describes the Modular harness design, novelty evaluation, state-of-the-art baselines, parallel discovery, and task configurations.

A.1 Modular Harnesses

  • Modular harnesses minimally modify OPRO (Yang et al., 2024), varying mutation and parent-sampling strategies and pruning the lowest-scoring discovery when the population exceeds 15.Four mutation strategies and three sampling strategies yield 12 variants.Source: §A.1 · p. 14, Table 5 · p. 14

A.2 LLM-Judge as a Novelty Evaluator

  • The LLM novelty evaluator judges whether candidates differ meaningfully, using task-specific prompts and disregarding trivial changes.The prompts cover coding and non-coding tasks.Source: §A.2 · pp. 15–16

A.3 State-of-the-art Harnesses

  • The state-of-the-art comparison includes OpenEvolve, ShinkaEvolve (Lange et al., 2026), and OA Agrawal et al. (2026a), instantiated with recommended defaults.ShinkaEvolve uses Gemini 3.5 Flash as its LLM-judge and a cosine-similarity threshold of 0.95.Source: §A.3 · p. 16

A.4 Parallel Discovery

  • Parallel discovery independently mutates the initial seed without updating in-context knowledge; K Context is generally more performant than Reflect in this setup.DE and GA require at least two seed discoveries, whereas K Context and Reflect can start from one.Source: §A.4 · p. 16

B Discovery Tasks

B.1 Reward Hacking on CloudCast Task

The appendix documents reward hacking in CloudCast and describes a convergence criterion used to estimate initialization budgets, noting that the criterion predicted an ideal budget in most tested settings.

B.1 Reward Hacking on CloudCast Task

  • For Gemini 3.5 Flash on CloudCast, reward hacking produced scores above 0.015, while genuine discoveries typically hit a ceiling near 0.01.The authors applied a post-hoc max-score filter of 0.015 to CloudCast discoveries.Source: §B.1 · p. 18
  • The example CloudCast discovery searches the function call stack for simulator or evaluator functions and tries to use them as a local evaluation oracle.The listing excerpt identifies function names containing sim, eval, or runner.Source: §B.1 · p. 18

B.2 Convergence Criterion

  • The trajectory is declared converged when its best-known candidate has not improved by threshold δ within N% of the discovery budget.The thresholds were generally at least an order of magnitude below the maximum known task score, and two orders of magnitude below it for Can’t Be Late, CloudCast, and TSP.Source: §B.2 · p. 19
  • Convergence predicted the ideal initialization budget in most settings, though the authors present the analysis as a proof of concept.The task-specific δ and budget-specific N choices were made by non-subject-matter experts; the authors suggest expert-curated pairs for future work.Source: §B.2 · p. 19

C Extended Results

The extended results compare sequential and parallel discovery, examine diversity interventions, and report initialization and population-curation findings. They also include mutation-prompt implementations used by the harnesses.

C Extended Results

  • The extended results repeat the main-text experiments over additional LLMs.The appendix also compares sequential and parallel discovery and provides expanded results on diversity and initialization.Source: C Extended Results · p. 20

C.1 Parallel vs. Sequential Discovery

  • Some form of sequential compute generally outperforms parallel discovery, although the best harness type varies by task and LLM.Sequential trajectories also show diminished diversity over time, whereas parallel schemes discover more diverse candidates.Source: Figure 9 · p. 20, Figure 10 · p. 21

C.2 Can Exploration Enable Better Discovery?

  • Harnesses and interventions that increase diversity do not predictably produce more performant discoveries; the best choice varies across task and LLM settings.State-of-the-art harnesses and diversity interventions generally achieve more diverse discoveries than baseline Modular harnesses.Source: Table 9 · p. 22, Table 10 · p. 24

C.3 The Importance of Initialization

  • Greedy initialization with minimal similarity filtering (s = 0.98) typically outperforms purely greedy populations, while initialized sequential discovery can outperform purely sequential or parallel discovery.The predicted initialization budget coincided with the most performant budget in 9/13 scenarios.Source: Table 11 · p. 25, Figure 12 · p. 26

D Mutation Prompts

  • The mutation-prompt appendix describes prompt variants that use past solutions, scores, and execution traces to guide candidate revisions; Reflect and the listed evolutionary mutators use two LLM calls.Reflect separates diagnosis from mutation, while the Differential Evolution and Genetic Algorithm listings also separate mutation or crossover from discovery extraction.Source: D Mutation Prompts · pp. 27–29

E Task Objectives

The task objectives span deadline-aware cloud scheduling, multi-cloud routing, tour and circle-packing optimization, and prompt optimization. Initial seeds range from task-specific heuristics and structured solutions to random paths and a placeholder prompt.

E Task Objectives

  • Can’t Be Late minimizes cost while meeting a hard deadline, choosing between cheap, preemptible spot instances and more reliable on-demand instances.The objective also weighs waiting for spot availability against using on-demand instances.Source: E Task Objectives · p. 29
  • CloudCast seeks low-cost multi-cloud broadcast routes that balance cost and throughput across destinations and data partitions.Each destination must have a valid path for every partition.Source: E Task Objectives · p. 29
  • The TSP task favors shorter tours that traverse every point exactly once, while circle packing maximizes the radii of 26 nonoverlapping circles inside a unit square.Source: E Task Objectives · pp. 29–30
  • Prompt optimization asks for a prefix that improves a downstream LLM’s score on the GSM8K benchmark.Source: E Task Objectives · p. 30

F Initial Seed Discoveries

Loading 2610.00707v1…