PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimizing the Optimizer: Language Models Discover Faster Molecular Relaxation

Artem Tsypin, Vladimir Deshchenya, Kuzma Khrabrov, Denis Potapov, Maxim Radchenko, Artur Kadurin, Michael G. Medvedev

arXiv:2610.06577v1physics.chem-ph

TL;DR

Because DFT gradient evaluations dominate geometry-relaxation wall time, this work asks whether language-model-driven autoresearch can improve an already strong optimizer. It evolves two AutoSella variants from Sella using GFN2-xTB and evaluates their performance on held-out molecules and unseen potentials. At r2SCAN-3c, AutoSella-F uses approximately 23–60% fewer force calls per molecule than Sella across four out-of-distribution datasets while achieving greater mean energy recovery.

  • Problem

    At the DFT level, gradient calculations dominate geometry-relaxation wall time, motivating methods that reduce the optimizer’s force-call count.

    Source: §1 · p. 1

  • Method

    Two independent language-model-driven autoresearch runs edited Sella to produce AutoSella-F and AutoSella-A, with candidates evaluated on training and validation splits.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    At r2SCAN-3c, AutoSella-F used approximately 23–60% fewer force calls per molecule than Sella across four out-of-distribution datasets while achieving greater mean energy recovery.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The results suggest inexpensive surrogate potentials can support automated algorithm development when direct search at the target level is too costly.

    Source: §5 · p. 9

    Held-out optimizations were capped at 200 force calls, increased to 500 for Solvated PubChem.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Geometry optimization is a major cost in many quantum-chemical workflows: each optimization step requires one force evaluation, and at the density-functional level that evaluation dominates the wall time. Research in this area has produced a broad range of optimization methods, and we ask whether a language model can improve on the best of them through autoresearch. An agent rewrites the optimizer itself to minimize force-call counts, restrained by two admission gates that reject premature stopping and improvements that do not generalize to unseen molecules. Starting from Sella, the fastest open-source optimizer available, the search produces AutoSella, a family of two optimizers. Both of them deliver consistent force-call reductions relative to Sella across held-out molecular benchmarks and potentials not used during the search. Most notably, at the \texttt{r2SCAN-3c} DFT level, the best variant requires only $40.2$--$77.2\%$ of Sella's force calls while achieving the same energy reduction, even though agent used no DFT gradients.

1 INTRODUCTION

Geometry optimization is especially costly at the DFT level because gradient evaluations dominate relaxation time. This work uses language-model-driven autoresearch to edit a strong optimizer, with gates intended to preserve relaxation depth and generalization.

1 INTRODUCTION

  • At the DFT level, gradient calculations dominate relaxation wall time, making the optimizer’s force-call count a key cost factor.Geometry optimization is widely used, and reducing force evaluations has motivated decades of algorithm development.Source: §1 · p. 1
  • Sella 2.5.0 used the fewest force calls among 27 tested modern optimizer configurations and served as the autoresearch starting point.The agent repeatedly proposed edits intended to reduce force calls while reaching the same minima.Source: §1 · p. 2
  • The search used GFN2-xTB because DFT relaxations were too costly for evaluating hundreds of candidates, then tested whether improvements transferred to DFT.A validation gate required lower force-call cost without loss of relaxation depth on unseen molecules; a validity gate rejected evaluation errors or lower mean energy recovery than Sella.Source: §1 · p. 2
  • Two independent autoresearch runs produced AutoSella-F and AutoSella-A and evaluated generalization to unseen molecules, new chemical domains, and potentials absent from the search.The work also reports a gated autoresearch loop for improving molecular geometry optimizers.Source: §1 · p. 2

2 RELATED WORK

Related work spans automated code and heuristic search, autonomous research loops, and benchmarks for molecular relaxation. This paper combines code editing against an evaluator with gates intended to constrain reward hacking and overfitting.

Language models that write algorithms

  • Program-search methods evolve code or optimization heuristics against evaluators, with different approaches varying in how their search harness is structured.Examples include FunSearch, AlphaEvolve, EoH, ReEvo, LLaMEA, and Lion.Source: §2 · p. 3

Autonomous research loops

  • Autonomous research systems range from fixed-benchmark, single-file loops to systems that automate hypothesis generation or use multiple agents.The paper’s loop is single-trajectory and uses gates alongside its main objective to address reward hacking and overfitting.Source: §2 · p. 3

Relaxation with MLFFs

  • Existing molecular-relaxation benchmarks assess potentials or compare optimizers, while this work evaluates whether force-call reductions persist across different potentials.The passage positions this cross-potential evaluation as complementary to the described benchmarks.Source: §2 · p. 3

3 METHOD

The method defines force-call cost and relaxation depth, assembles training, validation, and held-out molecular sets, and evolves Sella through a gated search on GFN2-xTB. Candidate edits must reduce cost without errors or reduced energy recovery on either training or validation data.

3.1 TASK AND METRICS

  • Force-call cost is measured using pooled and per-molecule aggregates of candidate-to-reference calls, while relative energy recovery scores relaxation depth against the reference.Pooled cost gives greater weight to molecules with longer reference trajectories; per-molecule cost weights molecules evenly. Relative recovery below 1 indicates a shallower result, while above 1 indicates a deeper minimum.Source: §3.1 · p. 3

3.2 MOLECULES

  • Training and validation use drug-like molecules and non-covalent dimers, while four held-out datasets test drug-like molecules, dipeptides, amino-acid–ligand contacts, and solvated multi-fragment systems.The training and validation sets contain 469 and 465 structures, respectively; held-out sets probe unseen molecules and chemical domains.Source: §3.2 · p. 4

3.3 POTENTIAL AND CONVERGENCE CRITERIA

  • The search uses GFN2-xTB and a five-criterion Gaussian-style convergence test, with training and validation capped at 200 force calls.Held-out runs use the same cap, increased to 500 for Solvated PubChem; Sella converges on all 1,648 structures in the other three held-out datasets, but on 98 of 100 Solvated PubChem systems.Source: §3.3 · p. 4

3.4 THE AUTORESEARCH LOOP

  • Starting from Sella 2.5.0, the autoresearch loop lets an agent edit one file and scores each candidate’s force-call cost on training data before validation.Sella used the fewest force calls among the tested open-source optimizers; two independent 66-hour runs produced AutoSella-F and AutoSella-A.Source: §3.4 · pp. 4–5

3.5 CANDIDATE ACCEPTANCE CRITERIA

  • The validity gate rejects errors, reduced mean energy recovery, or cost improvements below δ = 10^-4; the generalization gate applies the same requirements on validation data.Non-converged optimizations stopped by a force-call limit remain in training and validation averages. The method also compares optimized conformations with Sella using heavy-atom RMSD and vibrational analysis; the table caption notes that reported means exclude errors and nonconverged molecules.Source: §3.5 · pp. 5–6, Table 1 · p. 6

4 RESULTS

The evaluation tests AutoSella on held-out molecular datasets and potentials not used during its GFN2-xTB search. AutoSella-F delivers the most consistent force-call savings, while the progress analysis and ablations identify mechanisms and differences between the two search runs.

4.1 COST ON GFN2-XTB

  • AutoSella-F used 39.86–73.25% of Sella’s force calls per molecule across the four held-out GFN2-xTB datasets, with mean energy recovery above baseline on all four.Its final energies were lower than the reference on most molecules.Source: §4.1 · p. 6

4.2 TRANSFER TO UNSEEN POTENTIALS

  • At r2SCAN-3c on Solvated PubChem, AutoSella-F used 40.24% of Sella’s force calls with 1.27% greater mean energy recovery, and saved more calls than AutoSella-A in every available comparison.Its mean energy recovery exceeded Sella’s on all four datasets, despite the search never evaluating a DFT gradient.Source: §4.2 · pp. 6–7, Table 2 · p. 7

4.3 AUTORESEARCH PROGRESS

  • After 66 active research hours, AutoSella-F/A used 64.99%/76.66% of Sella’s validation force calls and 65.13%/74.51% on training.Fable accepted 37 improvements from 108 candidates, while Astra accepted 29 from 672.Source: §4.3 · p. 7
  • Fable’s mean gap between candidate commits was 36.32 ± 32.08 minutes, versus Astra’s 5.88 ± 3.66 minutes; Fable also used extended local analysis to target an initialization change.The interval between cycles 87 and 88 included 160 minutes of analysis.Source: §4.3 · p. 7
  • Fable’s research history links larger gains to corrected non-covalent-contact curvature, model-aware reuse of gradient differences, and cheaper preparation of conformations and fragment positions.Astra’s later candidates often increased force-call cost or failed validation cost or energy tests.Source: §4.3 · p. 7
  • Removing the model Hessian and coordinates block or fragment handling raised validation force-call cost most for both optimizers; the third-largest effects came from F’s Hessian updates and A’s Gaussian process.The increases were approximately 18.7 and 11.3 percentage points for AutoSella-F, and 8.8 and 5.7 points for AutoSella-A; the third effects were 10.2 and 4.7 points, respectively.Source: §4.3 · p. 8

5 CONCLUSION

Two independent autoresearch runs starting from Sella produced AutoSella-F and AutoSella-A. AutoSella-F achieved larger, more consistent force-call savings that transferred to unseen potentials, and ablations identify several contributing optimizer changes.

5 CONCLUSION

  • Across four out-of-distribution datasets at r2SCAN-3c, AutoSella-F required approximately 23–60% fewer force calls per molecule than Sella while achieving greater mean energy recovery.Its savings also transferred to GFN-FF and r2SCAN-3c, neither of which was used during the GFN2-xTB search.Source: §5 · pp. 8–9
  • Ablations identify improved curvature models and coordinates, explicit handling of disconnected fragments, and Hessian updates that reuse prior gradient information while accounting for geometry changes as contributors to AutoSella-F’s savings.Source: §5 · p. 9
  • The results suggest that inexpensive surrogate potentials can support automated algorithm development when direct search at the target level is too costly.Source: §5 · p. 9

AI USE STATEMENT

AI tools supported both the research process and manuscript preparation, while the authors retain responsibility for the paper’s final content.

AI USE STATEMENT

  • AI agents generated research ideas and developed the improved molecular optimization algorithm within the autoresearch harness.Source: AI USE STATEMENT · p. 9
  • LLMs also helped build the harness, implement and debug code and prompts, run experiments and evaluations, find related work, and revise the manuscript.Source: AI USE STATEMENT · p. 9
  • The authors take full responsibility for the paper’s final content, claims, references, and AI-assisted research artifacts.Source: AI USE STATEMENT · p. 9

A DATA PREPARATION

The appendices describe how molecular datasets were prepared, how the Sella reference was made portable, and how the isolated autoresearch harness and gated candidate cycle operated.

A DATA PREPARATION

  • Training, validation, and held-out inputs came from SPICE 2.0.1 and a fixed 25-structure RowanSci benchmark, whose coordinates were perturbed with mean per-atom displacement of 0.02 Å.Source: A DATA PREPARATION · p. 15
  • Dataset preparation applied relaxation, energy-decrease, connectivity, and geometry checks; the resulting training and validation sets contained 469 and 465 structures, with separate held-out PubChem, dipeptide, amino-acid–ligand, and solvated sets.The held-out sets contained 500, 673, 475, and 100 systems, respectively.Source: A DATA PREPARATION · p. 15

B REFERENCE OPTIMIZER

  • The Sella 2.5.0 reference was compacted into a single file by tracing the executed path, joining its eight modules, and pruning untraced definitions while restoring definitions required at import.The seed contained 5,975 source lines and was identical for the Fable and Astra searches.Source: B REFERENCE OPTIMIZER · pp. 15–16
  • The search began with allow_fragments=False, which uses artificial coordinate connections and achieved greater energy reduction but required more calls; evaluation comparisons used Sella with allow_fragments=True.Both evolved optimizers later enabled disconnected fragments.Source: B REFERENCE OPTIMIZER · p. 16

C.1 AUTORESEARCH HARNESS

  • Isolated agent environments protected other runs’ implementations and histories, while remote workers evaluated candidates and kept scoring and reference data outside the agent’s environment.Frozen optimizers were separately evaluated with ORCA 6.1.1 at r2SCAN-3c.Source: §C.1 · pp. 16–17

C.2 THE AUTORESEARCH LOOP

  • Agents could edit only the optimizer file and could not modify the evaluator, convergence criteria, molecule sets, or baseline metadata; molecule-specific special cases were also prohibited.Source: §C.2 · p. 17
  • Per-molecule records gave agents force-call costs, energy recovery, final geometries, convergence status, and stopping reasons to identify which molecules benefited or regressed.Source: §C.2 · p. 17
  • Each cycle implemented one coherent hypothesis, evaluated it on training, then sent candidates passing validity checks to validation; accepted candidates replaced the champion and rejected work was preserved in research artifacts.The agent consulted decision records, logs, and a backlog across cycles.Source: §C.2 · pp. 17–18

D PER-MOLECULE ENERGY BEHAVIOUR

Per-molecule energy differences show that higher-energy endpoints are rare on the single-molecule datasets, while fragment-containing systems have larger and less uniform differences. A lower final energy alone does not establish that an optimizer reached a deeper local minimum.

D PER-MOLECULE ENERGY BEHAVIOUR

  • Only 0.60–2.08% of Test and Dipeptides molecules finish more than 0.1 kcal mol−1 above the reference, and 0.20–1.19% finish more than 1.0 kcal mol−1 above it.On these single-molecule datasets, the median difference from the reference ranges from −8×10−6 to −5×10−6 kcal mol−1.Source: Table 3 · p. 18, D PER-MOLECULE ENERGY BEHAVIOUR · p. 18
  • On AA–ligand pairs and Solvated PubChem, AutoSella-F finishes more than 1.0 kcal mol−1 below Sella on 24.84% and 51.00% of systems, but more than 1.0 kcal mol−1 above it on 18.74% and 38.00%.Its median difference is −5 × 10−6 kcal mol−1 on AA–ligand pairs and −1.30 kcal mol−1 on Solvated PubChem.Source: D PER-MOLECULE ENERGY BEHAVIOUR · p. 18
  • A lower final energy alone does not establish that an optimizer reached a deeper local minimum, motivating the reported structural and frequency checks.Source: D PER-MOLECULE ENERGY BEHAVIOUR · p. 18

E STRUCTURAL AGREEMENT WITH THE REFERENCE

The structural comparison checks whether AutoSella endpoints match Sella across four held-out datasets using geometry, energy, and vibrational criteria. The section also places classical optimizers on a common evaluation scale.

E STRUCTURAL AGREEMENT WITH THE REFERENCE

  • The same-minimum criterion jointly requires heavy-atom RMSD < 0.1 Å, |∆E| < 0.1 kcal mol−1, and no candidate vibrational frequency below −50 cm−1.The frequency cutoff is a practical comparison tolerance, not proof of a local minimum.Source: Table 4 · p. 19, E STRUCTURAL AGREEMENT WITH THE… · p. 19
  • AutoSella-F and AutoSella-A satisfy the same-minimum criterion on 97.2% and 97.6% of Test molecules, and 93.2% and 95.7% of Dipeptides, respectively.Agreement is 39.8% and 69.5% on AA–ligand pairs, and 1.0% and 18.0% on Solvated PubChem; the authors hypothesize that varied fragment poses and intermolecular contacts contribute.Source: E STRUCTURAL AGREEMENT WITH THE… · p. 19
  • Across all four datasets, every pair meeting both geometry and energy thresholds also passes the candidate frequency cutoff; imaginary modes are rare, including for Sella.Lower agreement on fragment systems reflects differences in final geometry and energy rather than rejection by the frequency screen.Source: E STRUCTURAL AGREEMENT WITH THE… · p. 19

F OPTIMIZERS

  • The classical-optimizer comparison uses the same evaluation protocol so results can be placed on an absolute scale against established open-source optimizers.Table 5 reports cost, mean energy recovery, and unconverged-molecule counts relative to Sella 2.5.0 on GFN2-xTB.Source: F OPTIMIZERS · p. 19, Table 5 · p. 20

F.1 THE CLASSICAL PANEL

Under a shared evaluation protocol, Sella has the lowest force-call cost among the tested classical optimizers. Controlled comparisons show that both coordinate choice and the initial Hessian strongly affect optimization cost.

F.1 THE CLASSICAL PANEL

  • The panel uses a common interface, GFN2-xTB backend, force-call budget, and five convergence criteria, with each optimizer’s own stopping test disabled.The budget is 200 force calls, or 500 on Solvated PubChem; PreconLBFGS uses specified non-default neighbor radii for isolated molecules.Source: §F.1 · p. 20
  • Sella has the lowest per-molecule force-call cost among the tested classical optimizers on all four datasets.Source: §F.1 · p. 20
  • On Dipeptides, same-minimum agreement is 95.8% for pysisyphus in TRIC coordinates and 95.4% for geomeTRIC in TRIC, versus 23.3% for Cartesian BFGS and none of 673 molecules for FIRE.The Cartesian methods frequently stop at a different structure, not merely a marginally different conformer.Source: §F.1 · pp. 20–21

F.2 WHAT ACTUALLY MAKES AN OPTIMIZER EFFICIENT

  • In geomeTRIC on Test, changing coordinates alone raises cost from 129.6% of Sella with TRIC to 572.2% with Cartesian coordinates, a factor of 4.4.The intervening costs are 130.4% for delocalized internals, 145.9% for primitive internals, and 327.2% for HDLC.Source: §F.2 · p. 21
  • In pysisyphus, changing only the initial Hessian changes cost from 110.5% of Sella with the Fischer–Almlöf model to 444.9% with the unit matrix, a factor of 4.0.The other reported settings are 110.8% with Lindh’s model, 130.5% with Swart’s, and 193.2% with a simple diagonal guess.Source: §F.2 · pp. 21–22
  • Both coordinate choice and initial-Hessian choice matter; the cost comparisons are lower bounds because the worst settings also fail to converge more often.Cartesian geomeTRIC leaves 141 of 500 Test molecules unconverged, while pysisyphus with a unit Hessian leaves 56; Sella’s advantage over Cartesian methods largely comes from internal coordinates and a model initial Hessian.Source: §F.2 · p. 22

F.3 THE OPTIMIZERS

The optimizer panel compares classical methods across molecular settings, while the ablations examine how groups of evolved mechanisms affect force-call cost under energy and coverage criteria.

F.3 THE OPTIMIZERS

  • PyBerny costs 2.0× on Test and 2.5× on Dipeptides, reaches the reference minimum on only 62.7% of Dipeptides, and cannot build coordinates for 63 solvated molecules.Source: §F.3 · p. 22
  • PreconLBFGS leads Cartesian methods at 4.4–4.6×; pysisyphus is strongest on single-fragment molecules, while geomeTRIC converges on 99 of 100 solvated systems.Source: §F.3 · pp. 22–23

G.1 EXPERIMENTAL DESIGN

  • Each ablation disables one or two change families and is valid only with complete coverage, no optimizer errors, and mean normalized energy recovery ¯r ≥ 1 − 10^-9.Source: §G.1 · p. 23

G.2 AUTOSELLA-F BLOCKS

  • AutoSella-F combines a richer physical model and coordinates, geometry-following multi-pair Hessian updates, and explicit fragment handling.Its model adds non-covalent contacts and out-of-plane coordinates; its Hessian update corrects stored pairs as geometry changes.Source: §G.2 · pp. 24–25

G.3 AUTOSELLA-A BLOCKS

  • AutoSella-A adds calibrated chemical Hessians and coordinates, nonlinear coordinate transformations, collision-aware step control, and a checked Gaussian-process alternative step.The alternative step uses recent energies and gradients and requires no additional potential calls.Source: §G.3 · pp. 25–26

G.4 DEPENDENCIES, RESULTS, AND UNSUCCESSFUL EVALUATIONS

The ablations identify the largest measured cost increases from removing physical-model and coordinate changes, while joint-removal and threshold-failure results qualify how the blocks should be interpreted.

G.4 DEPENDENCIES, RESULTS, AND UNSUCCESSFUL EVALUATIONS

  • For all four tested validation pairs, joint removal increases cost less than the sum of individual penalties, so single-block effects are not independent or additive.Source: §G.4 · p. 26
  • AutoSella-F costs 65.13% and 64.99% of Sella on training and validation; AutoSella-A costs 74.51% and 76.66%, with physical-model and coordinate removals producing the largest measured single-block increases.Fable C adds 15.57 and 18.68 percentage points; Astra B adds 8.82 points on validation, while its training result fails the energy criterion.Source: §G.4 · p. 26
  • Only five complete evaluations fall below the energy threshold, and only three evaluations contain optimizer errors.The authors interpret this as indicating that the blocks are meaningful families with little or no effect on adjacent blocks.Source: §G.4 · p. 26

H AUTORESEARCH WITH GROK 4.6

A 66-hour Grok 4.6 autoresearch run accepted some candidate improvements, but its final optimizer transferred poorly to held-out intermolecular benchmarks and was not evaluated with DFT.

H AUTORESEARCH WITH GROK 4.6

  • The 66-active-hour run recorded 900 candidates and accepted 44 improvements; its cycle-897 champion reached 97.79% of Sella’s mean relative force-call cost on training and 98.14% on validation.Source: H AUTORESEARCH WITH GROK 4.6 · p. 26, Figure 4 · p. 27
  • AutoSella-G costs 148.77% of Sella on amino acid–ligand pairs and 239.85% on Solvated PubChem; 64 Solvated PubChem optimizations fail to converge within budget, and four encounter execution errors.Its costs are 99.27% on Test and 100.49% on Dipeptides; it was not evaluated with DFT.Source: Table 7 · p. 27, H AUTORESEARCH WITH GROK 4.6 · p. 27

Related papers

Loading 2610.06577v1…