PaperlayerView exact revision on arXiv

Source-linked AI summary

Flow-based Extremal Mathematical Structure Discovery

Gergely Bérczi, Baran Hashemi, Jonas Klüver

arXiv:2601.18005v1math.COcs.LG

TL;DR

Extremal mathematical structure discovery requires navigating difficult continuous, constrained, and nonconvex optimization landscapes. FlowBoost addresses this with a closed-loop generative framework that combines geometry-aware flow matching, reward-guided policy optimization, exploration, and stochastic local search. Across geometric optimization problems, it matches or exceeds known results, including circle-packing improvements over AlphaEvolve, while using fewer computational resources.

  • Problem

    Extremal geometric discovery involves continuous configuration spaces, hard constraints, and objectives lacking accessible likelihoods or closed-form gradients.

  • Method

    FlowBoost combines conditional flow matching, geometry-aware sampling, reward-guided optimization, exploration, and stochastic local search in a closed-loop pipeline.

  • Results

    FlowBoost matches or exceeds best known results across several geometric problems, with circle-packing constructions surpassing AlphaEvolve and closed-loop optimization requiring orders of magnitude fewer iterations.

  • Takeaways & Limitations

    Generative models can act as active optimization participants that receive objective feedback and adapt their sampling distributions toward rare, high-scoring geometric configurations.

  • Takeaways & Limitations

    Reward-guided training cannot reliably discover better configurations absent from the current endpoint distribution, motivating explicit exploration beyond the current support.

Abstract

from arXiv · show

The discovery of extremal structures in mathematics requires navigating vast and nonconvex landscapes where analytical methods offer little guidance and brute-force search becomes intractable. We introduce FlowBoost, a closed-loop generative framework that learns to discover rare and extremal geometric structures by combining three components: (i) a geometry-aware conditional flow-matching model that learns to sample high-quality configurations, (ii) reward-guided policy optimization with action exploration that directly optimizes the generation process toward the objective while maintaining diversity, and (iii) stochastic local search for both training-data generation and final refinement. Unlike prior open-loop approaches, such as PatternBoost that retrains on filtered discrete samples, or AlphaEvolve which relies on frozen Large Language Models (LLMs) as evolutionary mutation operators, FlowBoost enforces geometric feasibility during sampling, and propagates reward signal directly into the generative model, closing the optimization loop and requiring much smaller training sets and shorter training times, and reducing the required outer-loop iterations by orders of magnitude, while eliminating dependence on LLMs. We demonstrate the framework on four geometric optimization problems: sphere packing in hypercubes, circle packing maximizing sum of radii, the Heilbronn triangle problem, and star discrepancy minimization. In several cases, FlowBoost discovers configurations that match or exceed the best known results. For circle packings, we improve the best known lower bounds, surpassing the LLM-based system AlphaEvolve while using substantially fewer computational resources.

1. Introduction

Extremal geometric discovery poses continuous, constrained optimization challenges that existing discrete or LLM-based search systems handle indirectly. FlowBoost closes this loop with geometry-aware generation and reward-guided optimization, achieving strong results across geometric problems with lower computational requirements.

  • Continuous geometric structure discovery involves high-dimensional configuration spaces, hard feasibility constraints, and objectives without accessible likelihoods or closed-form gradients.
  • Deep generative models motivate learning to generate high-quality out-of-distribution configurations directly for mathematical structure design.
  • PatternBoost and AlphaEvolve use open-loop generate-evaluate-select pipelines that provide no direct objective feedback to the generative model.
  • FlowBoost combines continuous conditional flow matching, geometry-aware sampling, reward-guided optimization, and stochastic search in a closed-loop framework.Geometry-aware sampling interleaves flow integration with projection onto the constraint manifold, generating feasible samples during generation.
  • FlowBoost is reported to achieve competitive or superior continuous-geometric results with 2-5 orders of magnitude less compute and no reliance on LLMs.
  • Circle packing produces new best constructions for n=26 and n=32 circles, surpassing AlphaEvolve.
  • Sphere packing matches or improves known constructions, while the Heilbronn problem improves from A_min = 0.025236 to 0.025727 in four iterations.

• Sphere packing

The paper frames extremal geometric discovery and out-of-distribution generative design as instances of a shared constrained, likelihood-free optimization blueprint. It applies this framework across several geometric problems.

  • Rare structure discovery in mathematics and out-of-distribution generative design share a constrained, likelihood-free generative sampling blueprint.
  • The proposed bridge is intended to support transfer of methods, theoretical tools, and practical heuristics between the two areas.
  • The paper applies its framework to sphere packing, circle packing, the Heilbronn problem, and star discrepancy minimization.

2. Methods

FlowBoost frames extremal geometric discovery as simulation-based optimization over feasible configurations, using flow matching, geometry-aware sampling, and closed-loop reward-guided updates. The framework replaces discrete open-loop generation with continuous, reward-directed sampling that concentrates probability near high-quality solutions.

  • Simulation-Based Optimization: FlowBoost treats candidate configurations as samples from an objective-induced distribution that concentrates mass near high-quality regions.The objective defines a Boltzmann/Gibbs target, with larger β applying stronger optimization pressure.
  • Closed-Loop Optimization: Closed-loop reward-guided policy updates directly bias the generator toward the objective, while action exploration preserves diversity and consistency regularization limits distribution shift.The reward-weighted objective upweights high-reward samples, and the teacher–student term mitigates generative collapse.
  • Closed-Loop Optimization: A small number of effective update rounds replaces slow improvement across many outer cycles because generator updates use the objective rather than elite-set imitation.This distinguishes FlowBoost from the open-loop PatternBoost procedure.
  • Conditional Flow Matching: Conditional flow matching transports a simple prior to a distribution of high-quality configurations through a learned time-dependent vector field.Unlike diffusion, flow matching enables deterministic sampling through ODE integration.
  • Geometric Penalties: The training loss augments conditional flow matching with a geometric overlap penalty whose weight follows a cosine warm-up schedule.The schedule prioritizes distribution matching early and constraint satisfaction later in training.
  • Geometry-Aware Sampling: Geometry-aware sampling interleaves flow integration with constraint projection, generating feasible samples throughout the sampling process instead of filtering invalid outputs afterward.The approach incorporates geometric feasibility directly into inference and improves sample efficiency.

2.3. Geometry-Aware Sampling (GAS).

Geometry-Aware Sampling maintains approximate feasibility during flow generation by alternating integration with boundary handling, constraint projection, proximal relaxation, and terminal refinement. For sphere packing, adaptive margins depend on the configuration’s minimum separation rather than a fixed radius.

  • Adaptive Constraints: Adaptive wall constraints scale with the configuration’s minimum pairwise separation, enabling tighter packings without imposing a fixed radius during sampling.The effective radius is determined by the current configuration’s minimum separation.
  • Flow Integration: GAS discretizes flow time into K steps and advances the state through ODE integration before applying constraint corrections.The procedure integrates from the prior toward the data distribution using the model’s reversed internal time coordinate.
  • Boundary Handling: Iterative reflection enforces adaptive box boundaries while preserving minimum pairwise separation and preventing boundary violations.Boundary handling is applied after each integration step.
  • Constraint Projection: Gauss–Newton projection linearizes active overlap constraints and applies Jacobian-based corrections to project samples toward the feasible manifold.The implementation approximates these corrections with degree-normalized pairwise updates.
  • Proximal Relaxation: Proximal relaxation balances constraint satisfaction against fidelity to the learned flow trajectory using an anchor point and a lookahead penalty.The lookahead term penalizes constraint violations at the predicted endpoint.
  • Terminal Refinement: Terminal refinement applies additional projection passes with tightened tolerances until the maximum overlap residual falls below a prescribed threshold.This final stage is intended to ensure strict feasibility.

2.4. Reward-Guided Fine-Tuning.

Reward-guided fine-tuning shifts the flow model toward higher-reward configurations while consistency regularization and structured exploration preserve diversity and expand the searchable support.

  • Reward-guided updates: Reward-weighted flow matching progressively concentrates the learned terminal distribution around reward-maximizing modes.Repeated retraining amplifies probability mass according to successive powers of the weighting function.
  • Reward-guided updates: Above-average rewards receive amplified training influence, while below-average samples are downweighted rather than discarded.Weights use z-score normalization, exponentiation, temperature control, and clipping for numerical stability.
  • Consistency regularization: Pure reward maximization can collapse generation onto one or a few high-reward configurations, eliminating exploration of alternative basins.Consistency regularization anchors the student velocity field to a frozen teacher to maintain output diversity.
  • Geometry-aware exploration: Support suboptimality prevents reweighting alone from reliably discovering better configurations absent from the training-endpoint distribution.FlowBoost therefore adds smooth, constraint-informed exploration moves followed by projection onto the feasible manifold.
  • Geometry-aware exploration: The exploration operator combines contact- and wall-driven directions, scales them by overlap severity, and expands policy support for novel higher-reward structures.A repair step ensures rewards are evaluated on geometrically valid samples.

2.5. Closing the Loop: Convergence Properties.

FlowBoost closes the optimization loop by updating the generator with rewards from its own samples, while geometry-aware sampling and local refinement support systematic improvement in few boosting rounds.

  • Closing the loop: Reward-guided flow matching updates the current policy toward high-reward trajectories, while consistency regularization and action exploration prevent mode collapse.Geometry-aware sampling maintains feasibility throughout generation.
  • Pipeline: The pipeline initializes SRP-refined data, trains and samples a flow model, fine-tunes it with rewards, refines candidates locally, and repeats these steps.Local refinement uses SRP followed by L-BFGS optimization.
  • Convergence: 1–3 boosting rounds typically achieve convergence while the sample-objective distribution shifts upward and the best configurations improve over time.The authors characterize this as systematic improvement rather than stochastic drift.
  • Comparison with open-loop methods: PatternBoost requires hundreds to thousands of outer iterations because its generator imitates elite solutions without receiving direct objective feedback.FlowBoost instead propagates reward into the generator, making improvement more direct.
  • Local search: SRP alternates random perturbations with gradient relaxation and is used both to generate training data and to refine flow-produced samples.For packing problems, L-BFGS-B further polishes configurations under box constraints.
  • Local search: The physics-push heuristic produces substantially worse circle-packing configurations than SRP across common circle counts.Figure 2 reports consistently higher minimum-excess values for physics-push than SRP/SRS.

3. Results

FlowBoost is evaluated across geometric packing, extremal point-configuration, and discrepancy problems using conditional flow matching, geometry-aware sampling, and local refinement. In sphere packing, it exceeds the training maximum and PatternBoost’s best result while reducing iteration and sampling requirements.

  • Problem settings: FlowBoost studies sphere packing, the Heilbronn triangle problem, circle packing, and low star-discrepancy point-set construction.These tasks involve hard geometric constraints and continuous configuration objectives.
  • Heilbronn triangle problem: The Heilbronn installation trains on SRP-refined point sets and proposes new configurations with conditional flow matching before repeating local refinement.SRP optimizes a smooth soft-min surrogate, followed by L-BFGS-B and an active max-min push.
  • Star discrepancy: Star-discrepancy optimization uses a soft-max surrogate, L-BFGS-B, exact critical-grid evaluation, and projection or proximal steps during sampling.The model is conditioned on N and D* while targeting smaller discrepancy.
  • Model and conditioning: Permutation-equivariant Transformers parameterize the velocity field and use time and problem-specific conditioning to model point-cloud configurations.The conditioning variables include normalized radius or scale, object count, face-contact ratio, and normalized minimum separation.
  • Geometry-aware sampling: 40–60 midpoint-ODE steps with Gauss–Newton projection enforce non-overlap and wall constraints during geometry-aware sampling.Terminal refinement continues until the maximum overlap residual falls below 10^-8.
  • Sampling and refinement: The generative model produces diverse samples across multiple basins, whereas one local-search trajectory explores only locally.A single GAS sample plus projection overhead is comparable in wall-clock time to approximately 100 L-BFGS-B iterations.
  • Sphere-packing results: For N=89 spheres, RG-CFM with push exceeds both the training maximum and PatternBoost’s best result while requiring less outer iteration and sampling time.The comparison is part of the broader N=50–200 sphere-packing evaluation against the Packomania database.

12D Sphere Packing.

FlowBoost is tested on 31 spheres in a 12-dimensional hypercube, where feasible generation and improvement over strong local-search baselines are difficult. Reward-guided flow matching exceeds the training maximum after one round.

  • Challenge: 12-dimensional sphere packing has exponentially growing configuration spaces, shallow local-search basins, and no lattice structure for small N.The training set contains O(10^3) SRP-generated configurations with random restarts.
  • Generalization: In d=3, RG-CFM with final push exceeds training maxima for N=55, N=83, and N=89 sphere-packing configurations.The supplied Figure 3 passage reports the first two quantitative comparisons; the N=89 comparison is described as exceeding the training maximum.
  • Reward guidance: Reward-guided CFM with action exploration shifts the N=83 distribution toward higher-quality samples before local refinement, with dmax_min = 0.2236 versus 0.2157 for vanilla CFM.This comparison isolates the effect of direct objective feedback before the final push.
  • Result: A single round of RG-CFM fine-tuning improves the best minimum separation from 0.673721 to 0.673819 for N=31.The improvement exceeds the maximum in the SRP-generated training dataset.

Large-training-set runs (long per-iteration training).

Large-training-set experiments show FlowBoost shifting probability mass toward better sphere-packing basins. Means improve consistently, while absolute best values usually improve only slightly or remain nearly fixed.

  • Distribution shifts: Across N∈{71, 73, 79, 97, 191}, boosting strongly right-shifts and concentrates the dmin distribution.Higher-dmin configurations become more frequent and the mean improves monotonically.
  • Quantitative results: For N=71, the mean dmin improves from 0.239324 to 0.245881 by iteration 4, while the best changes from 0.246022 to 0.246030.This illustrates substantial mean improvement alongside marginal best-value improvement.
  • Interpretation: FlowBoost increases the probability of sampling near-record configurations even when the best value is difficult to improve.The results are contextualized against best-known sphere-in-a-cube values commonly benchmarked by Packomania.
  • Many-iteration regime: With more than 100 short-budget iterations at N=191, the maximum effective radius remains essentially flat while the average increases steadily.The experiment uses 500 samples and 300 training epochs per iteration.

3.3. The Heilbronn Problem.

For the Heilbronn triangle problem, FlowBoost combines differentiable soft-min optimization, geometry-aware sampling, and stochastic local-search refinement. Final pushing repairs generated configurations, and boosting can exceed training maxima for n=13 and n=15.

  • 3.3.1. Local search: SRP generates training data and performs final refinement, followed by L-BFGS-B polishing and an active-set max-min optimization over the smallest-area triangles.The lifted optimization maximizes t subject to active triangle-area constraints and box constraints.
  • 3.3. The Heilbronn Problem.: The Heilbronn objective is optimized using a differentiable soft-min surrogate whose geometrical annealing concentrates pressure on the worst triangle areas.The soft-min begins smoothly and progressively focuses on the smallest areas as β increases.
  • Sampling: Geometry-aware sampling interleaves ODE integration with projected gradient ascent on soft-min area, then applies SRP before exact A_min evaluation.The temperature is annealed from 5·10^-3 to 5·10^-4 during sampling.
  • Results: For n=13, the best A_min improves from a training maximum of 0.0257271 to 0.0259285 at iteration 2.Raw generation reaches 0.0210753, while pushing recovers 0.0254702.
  • Results: For n=15, the iterative run improves A_min to 0.0187494 by iteration 2, compared with a training maximum of 0.0184912.Pushing recovers 0.0183984 from raw generation at 0.0142529.

3.4. Circles in Unit Square with Maximal Sum of Radii.

For circle packing in the unit square, FlowBoost generates centers and recovers feasible radii through local refinement and an exact linear program. Boosting shifts samples toward higher total radii and increases near-record frequency.

  • 3.4. Circles in Unit Square with Maximal Sum of Radii.: Circle packing maximizes the sum of radii subject to containment and pairwise non-overlap constraints.The objective requires coordinated rearrangements of the contact graph under many hard inequalities.
  • Method: The post-processing LP performs an exact hard projection of radii for fixed centers rather than replacing SRP.It optimizes the radius subproblem, which becomes linear once the centers are fixed.
  • Method: The flow model generates centers only, while SRP and an exact max-sum-radii linear program recover feasible radii after sampling.This division exploits the fact that radii are cheaply optimized once centers are fixed.
  • Results: Across n=26, 30, and 32, boosting shifts probability mass toward the right tail, slightly increases means, and reduces low-score mass.The best observed configurations remain at the extreme right edge.
  • Results: For n=32, the mean total radius increases from approximately 2.9161 to 2.9180 by iteration 2 while the maximum remains 2.939349.The upper tail becomes denser during this sustained right-shift.
  • Comparison: FlowBoost best values beat AlphaEvolve’s reported thresholds of 2.635 for n=26 and 2.937 for n=32.The main reported benefit is that near-extremal solutions occur more frequently in the generated-and-pushed distribution.

3.5. Star Discrepancy Minimization.

Star discrepancy experiments combine a smooth grid-based surrogate with stochastic local search and final L-BFGS-B refinement. The final push is essential: it repairs raw flow samples and can slightly improve the best observed values.

  • Local search: Stochastic local search serves both as a training-set generator and as the final push for repairing flow samples.It optimizes a smooth approximation of the minimax discrepancy objective.
  • Local search: The smooth surrogate replaces the supremum over anchored boxes with a differentiable log-sum-exp soft-max over finite evaluation grids.The grids may be uniform or refreshed from current point coordinates.
  • Generation: The sampling loop interleaves flow-ODE integration with gradient projection and proximal relaxation, starting from Latin hypercube samples.All samples are evaluated only after the stochastic-repair final push.
  • Results: Across both sizes, generation alone is insufficient, whereas final pushing closes the gap to the training distribution and can slightly improve the best tail.This pattern is reported for N=20 and N=60.
  • Results: For N=20, final pushing lowers the mean discrepancy from raw generation’s 0.089340 to 0.069120 and improves the best value from 0.063117 to 0.062909.The training-set mean is 0.070105.
  • Results: For N=60, pushing returns the distribution from raw generation’s mean 0.080165 to approximately 0.035, while iteration 2 improves the best value from 0.029515 to 0.029440.The worst tail also decreases from 0.041322 to 0.040977.

4. Conclusion

FlowBoost is presented as a closed-loop generative framework for continuous extremal-geometry optimization. Its flow matching, geometry-aware sampling, and reward-guided updates achieve strong results across several tasks, including circle packings that beat AlphaEvolve, while the authors identify black-box rewards as an important limitation.

  • Conclusion: FlowBoost combines conditional flow matching, geometry-aware constraint enforcement, and online reward-guided policy optimization.The consistency regularizer is intended to preserve diversity and prevent generative collapse.
  • Conclusion: The key advance is closing the loop by fine-tuning the generator with direct reward gradients while maintaining exploration through a consistency term.This replaces open-loop distribution matching with policy optimization.
  • Results: For n=26 and n=32 circle packings, FlowBoost finds constructions that beat the best known result found by AlphaEvolve.The paper also shows two distinct best arrangements for each of these n values.
  • Results: Across four geometric optimization problems, FlowBoost matches or exceeds best known results and can beat available data within a single boosting round.The reported tasks include sphere packing, circle packing, the Heilbronn problem, and star discrepancy minimization.
  • Future directions: The framework is positioned for broader extremal problems with smooth objectives, many local optima, complex feasible regions, and limited exploitable algebraic structure.The authors mention applications across algebra, geometry, combinatorics, and number theory.
  • Limitations: A current limitation is that FlowBoost treats the reward as a black box and queries it only at generated samples.The authors suggest differentiable objectives or learned surrogate rewards as possible accelerators.
Loading 2601.18005v1…