PaperlayerView exact revision on arXiv

Source-linked AI summary

GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms

Valentin Khrulkov, Andrey Galichin, Denis Bashkirov, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Andrey Kuznetsov, Ivan Oseledets

arXiv:2511.17592v1cs.NEcs.AIcs.LG

TL;DR

Published descriptions of successful LLM-guided evolutionary methods leave implementation choices unspecified, complicating reproducibility and further research. GigaEvo provides an open-source framework with modular evolutionary search and LLM-driven mutation; it reports reproducing prior results and improving on a prior bin-packing result.

  • Problem

    AlphaEvolve achieved strong results, but its high-level description leaves many design and implementation choices unspecified, motivating reproducibility studies.

    Source: §1 · p. 1

  • Method

    GigaEvo is an open-source framework combining MAP-Elites search, asynchronous DAG evaluation, and LLM-driven mutation with program insights and bidirectional lineage context.

    Source: §2 · p. 2, §5 · p. 9

  • Results

    GigaEvo matched FunSearch’s uniform-distribution result and reduced Weibull excess bin usage from 0.68% to 0.55%, while recovering a nearly identical AlphaEvolve solution.

    Source: §4.1 · p. 5, §4.3 · p. 8

  • Takeaways & Limitations

    The reproduced AlphaEvolve results with an alternative architecture suggest that quality-diversity search, rich lineage-based contexts, and LLM-driven mutation are robust to implementation variations.

    Source: §5 · p. 9

    The framework currently focuses on single-file Python programs; support for multi-file projects and other programming languages remains future work.

    Source: §5 · p. 9

Abstract

from arXiv · show

Recent advances in LLM-guided evolutionary computation, particularly AlphaEvolve (Novikov et al., 2025; Georgiev et al., 2025), have demonstrated remarkable success in discovering novel mathematical constructions and solving challenging optimization problems. However, the high-level descriptions in published work leave many implementation details unspecified, hindering reproducibility and further research. In this report we present GigaEvo, an extensible open-source framework that enables researchers to study and experiment with hybrid LLM-evolution approaches inspired by AlphaEvolve. Our system provides modular implementations of key components: MAP-Elites quality-diversity algorithms, asynchronous DAG-based evaluation pipelines, LLM-driven mutation operators with insight generation and bidirectional lineage tracking, and flexible multi-island evolutionary strategies. In order to assess reproducibility and validate our implementation we evaluate GigaEvo on challenging problems from the AlphaEvolve paper: Heilbronn triangle placement, circle packing in squares, and high-dimensional kissing numbers. The framework emphasizes modularity, concurrency, and ease of experimentation, enabling rapid prototyping through declarative configuration. We provide detailed descriptions of system architecture, implementation decisions, and experimental methodology to support further research in LLM driven evolutionary methods. The GigaEvo framework and all experimental code are available at https://github.com/AIRI-Institute/gigaevo-core.

1 INTRODUCTION

AlphaEvolve demonstrated strong results in LLM-guided evolutionary computation, but its high-level description leaves implementation choices unspecified. GigaEvo is an open-source implementation intended to support reproducibility studies and further research.

1 INTRODUCTION

  • AlphaEvolve combined LLM code generation with evolutionary computation and achieved state-of-the-art results on challenging algorithmic and mathematical problems (Novikov et al., 2025).Its published description leaves many design and implementation choices unspecified.Source: §1 · p. 1
  • GigaEvo is an extensible open-source implementation designed to support reproducibility studies and further research.The authors evaluate it on AlphaEvolve benchmarks and practical tasks across domains.Source: §1 · p. 1
  • The framework emphasizes modularity, concurrency, and ease of experimentation for exploring hybrid LLM-evolution approaches.The experimental code is made available online.Source: §1 · p. 1

2 SYSTEM DESIGN

GigaEvo combines Redis-backed program storage with asynchronous evaluation, MAP-Elites search, and LLM-driven mutation. Its configurable pipelines and problem interface support concurrent experimentation and adding tasks without changing core system code.

2 SYSTEM DESIGN

  • The architecture connects Redis storage, a DAG execution engine, a MAP-Elites evolution engine, and an LLM mutation operator through a shared storage layer.The components support concurrent evolutionary search and single- or multi-island configurations.Source: Figure 1 · p. 2
  • Each Program stores code, lifecycle state, metrics, lineage, and DAG-stage outputs in Redis with optimistic concurrency control.Atomic counters support concurrent runs sharing an archive and realtime lineage tracking.Source: §2 · p. 2
  • The asyncio DAG engine runs programs concurrently and executes independent stages in parallel, with data-flow and execution-order dependencies.It supports cascading validation, scheduling, caching, and automatic stage skipping based on preconditions.Source: §2 · p. 2
  • MAP-Elites archives programs by fitness and validity, replacing a cell’s elite when a new program is better and sampling elites for mutation.Multi-island configurations can use distinct behavior spaces, such as fitness versus complexity, and periodically exchange top programs.Source: §2 · p. 3
  • The mutation agent builds prompts from task descriptions, parent code, metrics, insights, and lineage, then generates offspring by rewrite or diff.Rewrite mode prompts the model to localize changes; multi-model routing can assign distinct models to different stages.Source: §2 · p. 3
  • Problem directories specify task instructions, metrics, validation, and seed programs, while Hydra configuration composes experiments from separate components.The modular design lets users add problems by creating the required directory without modifying core code; validators return metrics including a validity indicator.Source: §2 · pp. 3–4

3 EXPERIMENTAL SETUP

GigaEvo enriches mutation prompts with individual program insights and bidirectional lineage analyses. A context-assembly stage brings code, metrics, and these analyses together for use during mutation.

3 EXPERIMENTAL SETUP

  • InsightsStage prompts an LLM with the task, program code, metrics, and execution exceptions to produce categorized program insights.Categories include algorithmic or structural type, beneficial or harmful effect, and severity; an example flags missing inter-point distance constraints as harmful.Source: §3 · p. 4
  • Lineage analysis examines parent-to-current transitions and selected descendants’ stored analyses to track improvements in both directions.Collectors select ancestors and descendants for detailed analysis, with configurable strategies such as selecting highest-fitness programs.Source: §3 · p. 4
  • MutationContextStage aggregates code, metrics, insights, and ancestor- and descendant-based lineage analyses into program metadata for mutation prompts.The context includes strategies that succeeded previously and improvements achieved by descendants.Source: §3 · p. 4

4 BENCHMARK PROBLEMS

GigaEvo is evaluated on four challenging optimization problems from prior work, with additional tests in bin packing and prompt/agent evolution. The illustrated DAG validates and evaluates programs, generates insights and lineage analyses, and assembles mutation context.

4.1 HEILBRONN TRIANGLE PROBLEM.

  • At n=11, GigaEvo reached 0.0364 minimum triangle area, 10^-4 below AlphaEvolve’s 0.0365, with a visually near-identical configuration.The authors suggest additional computational budget for fine-tuning simulated-annealing parameters could close the small difference.Source: §4.1 · p. 5, Figure 3 · p. 6
  • For circle packing, GigaEvo achieved 2.63598 versus AlphaEvolve’s 2.635 at n=26, and 2.939 versus the previous state of the art’s 2.937 at n=32.The task jointly optimizes disk positions and radii subject to non-overlap and square-boundary constraints.Source: §4.1 · pp. 5–6

4.2 KISSING NUMBERS IN HIGH DIMENSIONS.

  • In dimension 12, Gemini-2.5-Flash (Comanici et al., 2025) recovered the established lower bound of 840, but GigaEvo did not improve it; Qwen plateaued around 500.Candidates are integer-vector sets on a common shell, checked using exact integer arithmetic and exhaustive pairwise-distance validation.Source: §4.2 · p. 7

4.3 BIN PACKING

  • For Weibull arrivals, GigaEvo reduced excess bin usage from FunSearch’s 0.68% to 0.55%, and replicated FunSearch’s state-of-the-art result for uniform arrivals.In the reported ablation, the 7B model drove exploration while the 70B model primarily accelerated convergence.Source: §4.3 · p. 8

4.4 PROMPT AND AGENT EVOLUTION

  • On test data, the evolved prompt reached 0.783 AUC versus the baseline prompt’s 0.670 AUC; the agent reached 0.803 AUC.The agent combines predicted probability with precomputed Qwen embedding similarities through a calibrated sigmoid function and requires one LLM call.Source: §4.4 · pp. 8–9

5 CONCLUSIONS

GigaEvo packages modular LLM-driven evolutionary components for reproducibility studies and rapid prototyping. The report notes design insights, limits of current experiments and program scope, and its open-source availability.

5 CONCLUSIONS

  • GigaEvo provides modular MAP-Elites, asynchronous DAG-based evaluation, LLM mutation with bidirectional lineage tracking, and flexible configuration for reproducibility and rapid prototyping.The framework is intended to support research on LLM-driven evolutionary computation.Source: §5 · p. 9
  • Rewrite-based mutation was more robust than diff-based approaches for open-source models, while lineage analysis enriches mutation context and Hydra configuration reduces development iteration time.The report also describes heterogeneous LLM routing, including Qwen for geometry and Gemini for discrete optimization.Source: §5 · p. 9
  • Single-island experiments showed no clear benefit from multi-island MAP-Elites, and the framework remains focused on single-file Python programs.The authors call for further investigation of behavior-space design and future extension to multi-file projects and other languages.Source: §5 · p. 9
  • Reproducing AlphaEvolve results with an alternative architecture suggests the core principles are robust to implementation variations.The report names quality-diversity search, rich program contexts via lineage analysis, and LLM-driven mutation as those principles.Source: §5 · p. 9
  • The framework and all experimental code are available as open source.The report gives the repository address as https://github.com/AIRI-Institute/gigaevo-core.Source: §5 · p. 9

A.1 TASK DESCRIPTION

The passage introduces the task description supplied to the LLM during evolution.

A.1 TASK DESCRIPTION

  • The LLM receives a task description during the evolutionary process.The passage does not specify the contents of that description.Source: §A.1 · p. 11

TASK DEFINITION – HEILBRONN TRIANGLE PROBLEM (11 POINTS)

The task is to place 11 distinct points inside a unit-area equilateral triangle and maximize the smallest area among the triangles they form. The described formulation requires containment and non-collinearity, with a target minimum area of at least 0.0365.

TASK DEFINITION – HEILBRONN TRIANGLE PROBLEM (11 POINTS)

FAILURE MODES

The section identifies geometric and numerical failure modes for the placement task and lists helper routines and output requirements.

FAILURE MODES

  • Failure modes include clustering, nearly collinear triplets, duplicate or out-of-boundary points, incorrect triangle geometry, and precision errors that cause false collinearity detection.Source: FAILURE MODES · p. 11

HELPER FUNCTIONS

  • Helper functions provide the unit triangle vertices, the minimum triangle area for coordinates, and an inside-triangle check.The visible descriptions specify tuple-of-three vertex arrays and a Boolean containment result.Source: HELPER FUNCTIONS · p. 11
  • The entrypoint must return an (11, 2) NumPy array of points inside the unit-area equilateral triangle; random methods should fix their seed.The triangle is flat-bottomed with its apex at the origin.Source: HELPER FUNCTIONS · p. 11

A.2 EVOLVED SOLUTION CODE

The evolved Heilbronn-triangle program combines repeated initialization strategies with adaptive optimization and refinement. Its reported fitness is 0.036354 at generation 20.

A.2 EVOLVED SOLUTION CODE

  • Fitness 0.036354 at generation 20 is reported for the evolved Heilbronn-triangle program.The program uses Halton-sequence sampling, adaptive simulated annealing, critical-triangle resolution, and fitness-adaptive boundary distribution.Source: §A.2 · p. 12
  • 25 restarts initialize three points using Halton sampling or, periodically, a fitness-adaptive boundary distribution.Boundary initialization is selected every fifth restart when best_min_area is below 0.035.Source: §A.2 · pp. 12–13
  • Boundary points are distributed across the triangle’s three edges, with the total count adapted to best_min_area and small perpendicular perturbations.The total is bounded from 5 to 8 points; edge counts divide that total among AB, BC, and CA.Source: §A.2 · pp. 12–13
  • Halton coordinates are transformed into points inside the unit triangle, while later insertions are checked against a duplication threshold.The first point is generated from two Halton values; a later point must be more than twice the initial duplication threshold from the first.Source: §A.2 · p. 13
  • The code applies simulated annealing, critical-triangle resolution, and final refinement after initialization.The full 443-line implementation also includes fallback strategies; the supplied code excerpt does not detail those optimization steps.Source: §A.2 · p. 14

B PROMPT AND AGENT EVOLUTION RESULTS

The evolved prompt combines rule-comment and example similarities with step-by-step reasoning to produce a violation probability. The agent pairs an LLM estimate with a similarity-based heuristic and combines them with fallback handling.

B.1 FINAL PROMPT

  • The prompt supplies rule-comment similarity and average similarities to violating and non-violating examples.It also includes two examples of each class.Source: §B.1 · p. 14
  • The prompt instructs the model to weigh rule-comment similarity most, then positive-example similarity, while reducing the score for high negative-example similarity.It asks for a probability between 0.00 and 1.00, rounded to two decimal places.Source: §B.1 · p. 14
  • The prompt requests step-by-step comparison of the comment with the rule and examples, followed by a final probability in the format “Answer: X.XX”.Similarity scores are interpreted as high at >=0.7, moderate at 0.4-0.7, and low at <=0.4.Source: §B.1 · p. 14

B.2 FINAL AGENT

  • The agent’s heuristic applies a calibrated sigmoid to a weighted combination of rule similarity and the positive-minus-negative example similarity gap.The stated combination is 5.0 times the example-similarity gap plus 2.0 times rule similarity.Source: §B.2 · pp. 15–16
  • The agent makes an LLM call and combines its extracted probability with the heuristic at weights 0.70 and 0.30, respectively.If extraction fails, the implementation falls back to the heuristic; the final probability is clamped to [0,1].Source: §B.2 · pp. 15–16
  • The agent parses rule and example similarity fields from the sample, using 0.0 defaults when fields are absent.The implementation catches value and type errors during parsing.Source: §B.2 · p. 15
  • The probability extractor reads a numeric value after “Answer:” and converts values above 1.0 from percentages before clamping them to [0,1].The prompt requests the same Answer format.Source: §B.1 · p. 14, §B.2 · p. 16
Loading 2511.17592v1…