Source-linked AI summary
LLM-Guided Program Evolution for Circle Packing: Breaking 10 Packomania Records for $28
Wes Sander
TL;DR
The paper asks whether LLM-guided algorithm evolution can be reproduced cheaply and simply outside large research-lab infrastructure. It presents Discovery Loop, which iteratively replaces a seed solver using scoreboard- and history-guided proposals verified independently. On Packomania, it improved 10 target values by 2.4%–5.4% over prior records at a total LLM cost of $27.72, while analyzing plateau detection for cheaper stopping.
Problem
The paper examines how much of the AlphaEvolve paradigm can be reproduced with a single LLM, a single machine, and a budget under $30.
Method
Discovery Loop uses an LLM to propose complete replacement solvers guided by the current solver, scoreboard, and idea history, retaining improvements after independent verification.
Results
10 of 12 Packomania targets for N ∈{101, . . . , 114} improved by 2.4% to 5.4% over prior records, with a total LLM cost of $27.72.
Takeaways & Limitations
The results provide practical guidance for stopping automated search and support the accessibility of LLM-guided evolution to individual researchers.
Takeaways & Limitations
The system has not been demonstrated on problems with expensive evaluation or fundamentally different search spaces, and MIPLIB code generation failed 75% of the time.
Abstract
from arXiv · showhide
We present Discovery Loop, a lightweight system that uses a large language model (LLM) to iteratively evolve optimization algorithms. Starting from a simple seed solver, the LLM proposes algorithmic improvements guided by a scoreboard of results and a history of prior ideas. Each candidate is evaluated against an independent verifier; improvements are kept and failures discarded. Applied to the Packomania circle-packing benchmark (csqv: maximize the sum of radii of N variable-radius circles in the unit square), the system improved the best known solutions for 10 values of N in the range 101-114, with gains of 2.4%-5.4% over prior records, all within 15 iterations and at a total LLM cost of $27.72. These results have been independently accepted by Packomania. We describe the method, analyze cost-efficiency dynamics including an adaptive plateau-detection mechanism, and discuss implications for democratizing automated scientific discovery.
1 Introduction
Discovery Loop asks whether LLM-guided algorithm evolution can reproduce part of AlphaEvolve’s paradigm using one LLM, one machine, and under $30. On Packomania, it produced independently accepted improvements at low cost.
- Discovery Loop uses an LLM to propose complete replacement solvers from the current solver, scoreboard, and idea history.Candidates are evaluated against benchmark targets, and improvements are retained while failures are discarded.
- 10 of 12 Packomania targets for N ∈{101, . . . , 114} improved by 2.4% to 5.4% over prior records.The results were submitted to and accepted by Packomania.
- $27.72 was the total LLM cost for the reported run.The run used approximately eight hours overnight on a consumer PC.
- The paper contributes a minimal, reproducible implementation of LLM-guided program evolution.The implementation is open source.
- The work reports new best-known solutions for 10 Packomania csqv instances, independently verified and accepted.The instances span N = 101–114.
- A plateau-detection analysis indicates that stopping early could reduce cost by 50% with negligible loss in solution quality.The contribution also presents evidence that the paradigm is accessible to individual researchers at minimal cost.
2 Background
The paper studies variable-radius circle packing in the unit square, where the objective is to maximize the sum of radii under containment and non-overlap constraints. It situates Discovery Loop among LLM-based program-search systems while emphasizing a simpler, smaller-scale design.
- 2.1 Circle Packing: The csqv benchmark packs N variable-radius circles in the unit square to maximize the sum of radii.Packomania tracks best-known solutions for N = 1, . . . , 200+.
- 2.1 Circle Packing: Feasible packings keep every circle within the unit square and prevent pairwise overlap.
- 2.1 Circle Packing: For large N, the optimization landscape is highly non-convex with many local optima.Existing strong solvers combine penalty methods, basin-hopping metaheuristics, and linear programming for radii optimization.
- 2.2 LLM-Guided Program Evolution: AlphaEvolve uses an LLM to modify programs that are evaluated in a distributed sandbox, with program scores guiding future proposals.Its LLM-generated changes can include move operators, initialization strategies, and local-search enhancements.
- 2.2 LLM-Guided Program Evolution: FunSearch applies LLM program search to mathematical constructions by evolving small scoring functions.
- 2.2 LLM-Guided Program Evolution: Discovery Loop differs in scale and simplicity by using one LLM call per iteration without an evolutionary database, island model, or distributed evaluation.The system is implemented as a single Python script.
3 Method
Discovery Loop orchestrates iterative solver replacement, evaluation, verification, and best-result tracking for circle-packing targets. It combines a seed solver and problem plugin with history-guided LLM proposals, independent checking, and plateau-based early stopping.
- 3.1 System Components: Discovery Loop comprises a loop controller, a circle-packing problem plugin, and a seed solver.The controller manages iterations and plateau detection; the plugin defines targets, records, verification, and submission; the seed is a multi-start penalty L-BFGS-B solver with LP-optimal radii.
- 3.2 Iteration Loop: Each iteration constructs a prompt from the champion solver, scoreboard, history, and problem description, then extracts an idea and complete Python code.The algorithm updates per-target bests after evaluation.
- 3.2 Iteration Loop: The prompt includes the current champion source code, a scoreboard of best-known, current-best, and last-run values, and the last 12 tried ideas with outcomes.The LLM is instructed to output one idea description and one complete solver.
- 3.2 Iteration Loop: Complete replacement solvers are independently testable because they avoid accumulated merge conflicts.Each candidate is self-contained rather than a patch.
- 3.3 Per-Target Best Tracking: Per-target tracking preserves the best result for each target across all solvers, even when another solver remains the overall champion.This retains specialized improvements for individual targets.
- 3.4 Verification: Independent verification checks containment, non-overlap, and the recomputed sum of radii, then applies a strict feasibility shrink.The verifier shares no code with the solver to prevent evaluation gaming.
- 3.5 Plateau Detection: The plateau detector stops when recent iterations lack accepted progress, lack a champion iteration, or improve the total below θ.The default window is W = 4 and the default threshold is θ = 0.01.
- 3.5 Plateau Detection: Backtesting shows that stopping after iteration 9 would save $13.77, or 50% of cost, while sacrificing 0.006 on total sum.The reported sacrifice is 0.01% of the final value.
4 Results
Discovery Loop improved Packomania circle-packing results across 10 targets while revealing sharply declining cost-efficiency and generating several algorithmic innovations.
- Experimental setup: $30 budget and 40-iteration maximum defined the experimental constraints on consumer hardware with six parallel workers.The listed setup used a consumer PC and 120-second solver timeouts per target.
- Record-breaking results: 10 targets improved over prior Packomania records, while two additional targets matched or came within 10^-6 of those records without surpassing them.Table 1 covers the improved targets; the two near-matches were N = 26 and N = 32.
- Cost-efficiency dynamics: $4.96 produced a +0.57 improvement during iterations 0–5, compared with only +0.02 for $22.76 during iterations 6–14.Cost per unit improvement rose from $8.70 to $1,138 between the two phases.
- Cost-efficiency dynamics: 130× higher cost per unit improvement between phases motivates plateau detection, which would have stopped at iteration 9 and reduced spending to $13.95.The stated early-stopping configuration used W = 4 and θ = 0.01.
- Algorithmic innovations: The LLM generated basin hopping, hexagonal initialization, island-model parallelism, KKT-Newton polishing, and defect migration across iterations.These changes span initialization, search diversification, local optimization, and combinatorial restructuring.
- Algorithmic innovations: The reported innovations were characterized as algorithmic changes rather than parameter tuning, including domain-informed hexagonal initialization and defect migration.The paper attributes these ideas to synthesis from the problem description and scoreboard feedback.
5 Comparison to Prior Work
Discovery Loop is presented as a simpler, smaller-scale alternative to larger LLM-guided program-evolution systems, emphasizing minimal infrastructure and single-candidate iteration.
- System comparison: The system’s trade-off is scale: it processes one candidate per iteration and targets a single problem domain per run.This contrasts with systems requiring broader infrastructure and population-level exploration.
6 Discussion
The discussion attributes Discovery Loop’s performance to structured feedback, complete solver replacement, independent verification, and an accessible problem setting, while identifying clear scope limitations.
- Success factors: Structured scoreboards and idea histories give the LLM feedback about tried approaches and their outcomes.This is presented as one factor supporting the system’s success.
- Success factors: Complete solver replacement permits architectural changes, such as switching from random to lattice initialization, that incremental patching cannot express.The paper contrasts replacement with patch-based modification.
- Success factors: Independent zero-tolerance verification is intended to prevent evaluation gaming and ensure that improvements are genuine.Verification is part of the system’s evaluation loop.
- Success factors: Circle packing fits within the LLM’s context window and uses relatively cheap evaluation with 120-second target timeouts.The paper identifies this domain accessibility as a supporting condition.
- Limitations: The demonstrated scope is limited to a single problem setting and does not establish performance on expensive evaluations or fundamentally different search spaces.The paper explicitly states that such problems have not yet been demonstrated.
- Limitations: A single-champion design may provide less diverse exploration than a solver population.The paper presents population-based exploration as a possible way to increase diversity.
- Limitations: A 75% MIPLIB code-generation failure rate indicates dependence on the LLM’s ability to produce correct, compilable Python solvers.The paper uses this experiment to suggest sensitivity to problem complexity.
- Limitations: On fixed target sets, cost-efficiency shows rapid diminishing returns, so practical use requires expanding targets or accepting a plateau.This limitation follows from the cost-efficiency analysis.
7 Conclusion
Discovery Loop demonstrates that a minimal, open-source LLM-guided evolution system can break 10 Packomania circle-packing records at a cost of $27.72. Its cost-efficiency and plateau analysis offers practical guidance for deciding when to stop iterating, while the authors argue that such systems are becoming accessible beyond research labs.
- $27.72 enabled Discovery Loop to break 10 circle-packing records on the Packomania benchmark.The system is described as minimal and open source.
- Plateau detection provides practical guidance for deciding when to stop iterating in automated search systems.
- The authors argue that LLM-guided evolution is moving from research-lab infrastructure toward laptop-accessible experimentation.