PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

Young-Jun Lee, Seungone Kim, Minki Kang, Alistair Cheong Liang Chuen, Zerui Chen, Seungho Han, Taehee Jung, Dongyeop Kang

arXiv:2606.29082v1cs.CLcs.LG

TL;DR

Prior evolutionary-search methods do not establish whether LLMs can internalize and reuse discovery capability across tasks. EFT trains open-source LLMs on evolutionary trajectories from Finch Collection, yielding cross-task gains and strong results when combined with test-time RL.

  • Problem

    Existing methods construct discovery capability during each search rather than internalizing reusable strategies in the model.

  • Method

    EFT converts evolutionary search trajectories into supervised training data and fine-tunes 2B–9B open-source LLMs using Finch Collection across 371 tasks.

  • Results

    Across 22 held-out tasks, EFT improves discovery generalization; with test-time RL, Finch achieves best performance on two circle-packing tests and improves Erdős performance by +3.2%.

  • Takeaways & Limitations

    EFT functions as mid-training that strengthens discovery behavior before deployment and can complement test-time RL.

  • Takeaways & Limitations

    The study uses only the OpenEvolve scaffold and demonstrates positive synergy with test-time RL only on mathematical tasks.

Abstract

from arXiv · show

Would experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture? Large Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K-trajectory dataset spanning 10 domains and 371 optimization tasks, and fine-tune open-source LLMs from 2B to 9B parameters. Empirically, EFT confers cross-task generalization: across 22 held-out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test-time RL, our model matches state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem. EFT thus serves as a "practice phase" for general-purpose discovery agents that do not solve new problems from scratch.

1. Introduction

Existing evolutionary-search methods leave discovery capability in task-specific scaffolds or single-task updates. Evolution Fine-Tuning instead trains open-source LLMs on evolutionary trajectories so they can reuse discovery strategies across optimization tasks.

  • Optimization tasks score candidate solutions against objectives, including mathematical conjectures, GPU kernel design, and scientific-law discovery.
  • Existing methods either keep the LLM fixed while scaffolds drive search or update model weights only within a single target-task search loop.
  • Figure 1 associates EFT with improved Erdős minimum-overlap discovery and cross-domain strategy transfer on NP-hard competitive programming tasks.
  • EFT treats evolutionary search trajectories as supervision, aiming to internalize how to iteratively improve solutions across optimization tasks.
  • Finch Collection contains 156K trajectories from 10 domains and 371 tasks, supporting fine-tuning of Finch-2B, Finch-4B, Finch-8B, and Finch-9B.
  • Across 22 held-out tasks, Finch outperforms base counterparts and reaches performance comparable to best-known solutions from a proprietary model despite its smaller open-source backbone.

2. Preliminaries

The paper formulates optimization as iterative candidate generation and evaluation within an evolutionary scaffold. Scaffolds combine prompting, mutation, evaluation, and population management, while Finch Collection covers 371 tasks grouped across optimization domains.

  • An optimization task starts with candidate x_0 and generates candidates through T iterations, forming the set X = {x_0, ..., x_T}.
  • The evolutionary scaffold uses a mutation operator to generate each candidate from the previous solution, task instruction, search history, and feedback.
  • A discovery is a candidate that improves on the previous best-known solution within the search budget, with the inequality direction depending on maximization or minimization.
  • Finch Collection organizes 371 optimization tasks into groups spanning mathematical discovery, programming, heuristics, numerical optimization, symbolic regression, GPU kernels, and biological denoising.
  • Evolutionary scaffolds construct prompts, generate candidates by edits or rewrites, evaluate them, and store eligible candidates in a population database.

3. Evolution Fine-Tuning

Evolution Fine-Tuning transfers evolutionary discovery behavior into smaller open-source LLMs by training on filtered search trajectories from diverse optimization tasks. The resulting Finch Collection spans 371 tasks and supports models in learning both improvement patterns and mutation strategies across domains.

  • Seed Task Collection: 371 seed tasks from 10 benchmarks cover mathematical discovery, competitive programming, heuristic and numerical optimization, symbolic regression, GPU kernels, constructive search, and biological denoising.Tasks are selected for nontrivial search, no known ground-truth answer, and deterministic evaluators producing continuous or comparable scores.
  • Trajectory Characteristics: The collection balances diff-based editing and full rewriting, exposing models to exploitation-oriented refinement and broader exploration.The two mutation strategies represent 50.3% diff-based edit and 49.7% full rewrite trajectories.
  • Trajectory Collection and Filtering: 172,997 raw trajectories are filtered to retain 156,731 trajectories, approximately 156K, across 371 tasks.Filtering removes unreliable feedback, unrecoverable or breakage cases, and candidate solutions with systematic errors.
  • Evolution Fine-Tuning: EFT converts evolutionary search trajectories into supervised examples so models learn to map an evolutionary state to a plausible next mutation.Each instance includes the task instruction, parent solution, selected history, previous scores, and evaluator artifacts, with the teacher-generated candidate as the target.
  • Trajectory Characteristics: Improvement labels divide trajectories into 61,802 Imp, 30,130 NC, and 64,799 Reg cases, with Imp trajectories primarily used for evolution fine-tuning.The filtering and training choices aim to avoid imitating non-improving behavior while retaining contrast between promising and regressive solutions for KTO.

4. Experiments

Experiments evaluate ℱinch on 22 tasks across five domains, comparing it with base-model and learning-scaffold baselines. EFT improves cross-task discovery across model scales, preference learning adds gains, and test-time RL benefits from EFT while smaller models remain below the strongest proprietary system.

  • Evaluation: Evaluation covers five domains and 22 tasks, using task-specific metrics and maximum scores achieved within T iterations.The evaluation tasks are designed to be largely disjoint from those used to train ℱinch.
  • Baselines: ℱinch is evaluated as a mutation operator within OpenEvolve against base models and a learning-scaffold baseline.The primary comparison measures relative improvement over the initial program score.
  • EFT performance: +10.31% relative gain is achieved by ℱinch-9B, compared with +3.40% for ℱinch-4B, while ℱinch-4B reaches 0.386460 on Erdos versus Qwen3-8B’s 0.403585.The largest task-level improvements are +290.59% on ahc058 and +74.30% on Transaction.
  • Offline RL: Offline RL with KTO consistently improves ℱinch’s discovery capability, and ℱinch-8B surpasses the best human score on AC1 and AC2.KTO jointly uses improving and regressing trajectories to teach the model to distinguish good from bad solutions.
  • Test-time RL: ℱinch achieves the best performance on two circle-packing tests and improves Erdős performance by +3.2% after test-time RL.It still underperforms the original TTT-Discover using GPT-OSS-120B, indicating a remaining frontier-discovery gap for smaller models.

5. Analysis

Analysis finds that broader trajectory collections improve transfer across tasks and that ℱinch can reuse domain-specific patterns in new optimization settings. A Convolve2D case study illustrates this transfer through a library change aimed at computational efficiency.

  • Scaling: Increasing the number of training tasks produces a clear positive scaling trend on AC2, CP (n=26), and PRISM.The authors report that larger task collections, more trajectories, and greater model capacity may further increase gains.
  • Transfer: ℱinch-9B improves Convolve2D computational efficiency by replacing a scipy implementation with jax.The authors hypothesize that this behavior reflects exposure to many jax trajectories in ℱinch Collection.
  • Transfer: ℱinch Collection supports transferable discovery patterns across domains by combining varied task experience with multiple improvement types.Table 7 examines how improvement type affects EFT, while the case study connects library knowledge to cross-domain transfer.

6. Related Work

Related work organizes LLM-driven discovery around evolutionary search scaffolds and benchmarks for long-horizon optimization. Existing benchmarks span GPU kernels, numerical routines, programming-contest algorithms, and scientific discovery.

  • LLM-driven Evolutionary Search Scaffolds: LLM-driven evolutionary scaffolds are categorized into search-based and learning-based approaches that differ in candidate archival, parent selection, and parameter updates.AlphaEvolve is cited as a search-based example using MAP-Elites with island populations.
  • Benchmarks for Optimization and Scientific Discovery: Optimization benchmarks cover GPU kernels, numerical routines, algorithm engineering, and scientific discovery under long-horizon problem-solving settings.Examples include KernelBench, GSO, AlgoTune, ALE-Bench, FrontierCS, and AutoLab.

7. Conclusion

The paper concludes that EFT transfers discovery behavior from evolutionary trajectories into open-weight models and supports cross-task generalization. It also identifies limits in scaffold diversity, test-time RL coverage, modality, interaction length, and potential misuse.

  • Limitations: Evaluations use only the OpenEvolve scaffold, so mixed-scaffold trajectories are proposed to improve generalization to stronger scaffolds such as EvoX.This is identified as a future direction rather than an established result.
  • Limitations: Positive synergy with test-time RL is demonstrated only on mathematical tasks, leaving kernel engineering and other practical tasks untested.The authors call for evaluation on diverse and realistic tasks.
  • Limitations: The current framework is language-only and single-turn, motivating extensions to multimodal scientific discovery and multi-turn evolution.The proposed extensions would let models interpret visual observations and reason over previously explored lineages.
  • Implications: EFT may reduce search costs and enable local discovery pipelines, but the capabilities may also be misused for harmful objectives or reward hacking.The paper also cautions against relying on automatically generated discoveries without sufficient verification.

B.3. Distinguishing Optimization from Reasoning and Agentic Tasks

Optimization tasks differ from reasoning and agentic tasks because they are open-ended searches over abstract solution spaces, requiring iterative refinement and continuous evaluation rather than single-pass correctness.

  • Optimization tasks seek improved or novel solutions without known ground-truth answers, so success is measured by relative improvement over prior best-known results.
  • Unlike agentic tasks, optimization searches span larger and more abstract spaces, such as programs, algorithms, and mathematical constructions.
  • Optimization tasks involve longer time horizons and iterative refinement, whereas agentic tasks execute action sequences against predefined acceptance criteria.
  • Reasoning and agentic tasks typically use binary success signals, while optimization uses deterministic continuous metrics that measure and accumulate partial progress.
  • These properties position language models as mutation operators that repeatedly propose candidate solutions for evolutionary evaluation and improvement.

C.1. Systematic Error Breakdown Analysis

This section reports a breakdown of system-level errors filtered from the evolutionary search trajectories.

  • 6,321 trajectories were included in the system-level error breakdown, with counts and percentages calculated over that total.

C.2. Benchmark Licenses

The benchmark tasks were sourced from publicly available resources under permissive open-source licenses, with specific attribution and permission arrangements for CC BY-ND material.

  • The seed tasks come from publicly available benchmarks released under licenses including MIT, Apache 2.0, and CC-BY 4.0.
  • The authors limit use of these resources to non-commercial academic research and state that their use complies with the respective license terms.
  • Table 10 summarizes the license terms for the benchmark repositories and associated datasets.
  • ALE-Bench tasks derive from a CC BY-ND 4.0 dataset, and the authors received AtCoder permission to publicly release the resulting evolutionary trajectories.

D. Additional Implementation Details

The additional implementation details describe nanodiscover’s five-stage search-and-training pipeline and the compute and timeout settings used in the experiments.

  • nanodiscover reproduces TTT-Discover without the Tinker API and organizes each search epoch into five pipeline stages.The stages are parent sampling, child generation, child evaluation, archive updating, and test-time training.
  • Ray Data LLM orchestrates vLLM for child generation, while DeepSpeed supports test-time training.
  • Unless otherwise noted, nanodiscover hyperparameters were matched as closely as possible to the TTT-Discover setup.
  • All runs lasted 50 epochs on one node with 4 GPUs and 96 logical CPU cores.
  • The Erdős experiment used a 1000-second prompted budget and an 1100-second timeout, while both circle-packing experiments used 530-second timeouts without informing the model of the evaluation budget.

E. Full List of Optimization Tasks in ℱinch Collection

The Finch Collection task list covers Mathematical Discovery, Numerical Algorithm Optimization, and Heuristic Optimization, with descriptions spanning diverse computational and combinatorial objectives.

  • Mathematical Discovery: Mathematical Discovery contains 14 tasks, including counting Riemann zeta zeros and generalized eigenvalue or eigenvector problems.The listed mathematical tasks also include graph and optimization-related computations such as connected components and channel capacity.
  • Numerical Algorithm Optimization: Numerical Algorithm Optimization contains 46 tasks targeting speedups or implementations for linear algebra, graph algorithms, transforms, integration, geometry, and optimization.Examples include Cholesky factorization, Dijkstra shortest paths, FFTs, Delaunay triangulation, and earth mover’s distance.
  • Heuristic Optimization: Heuristic Optimization contains 35 tasks involving routing, scheduling, graph design, packing, placement, and other constrained search problems.Examples include food-delivery tours, online Steiner-MST, maze walking, loop construction, repair scheduling, and antenna placement.
  • Heuristic Optimization: The heuristic task descriptions also include grid-based cleaning, interactive polyomino detection, hall reservation, stamp pressing, dump-truck leveling, and seed-crossing problems.These tasks specify objectives such as minimizing dirt, queries, or penalties and maximizing sums, yields, or catches.
Loading 2606.29082v1…