Source-linked AI summary
AutoNovo: Agentic Program Search Discovers Transferable Nucleic Acid Design Algorithms
Joel Shor, Subhashini Venugopalan
TL;DR
Predictive-oracle optimization can produce nucleic-acid sequences that score well in silico but may not perform well in biological systems. AutoNovo uses agentic program search to discover sequence designers, achieving the best observed mean on the evaluated DNA-design and cross-oracle transfer tasks.
Problem
Predictive sequence-to-function models can guide nucleic-acid design, but sequences with strong predicted properties may not perform well in biological systems.
Source: §1 · p. 1Method
Results
AutoNovo achieved the best observed mean on 3/3 DNA design tasks and 6/6 cross-oracle transfer tasks, with different candidates performing best across tasks.
Source: §1 · p. 2Takeaways & Limitations
The results show that agentic program search can produce optimization algorithms whose improvements extend beyond the oracle pair and configuration used during discovery.
Source: §7 · p. 9Evidence is limited to in-silico evaluation of one biological phenotype using three related predictive models, and cross-oracle transfer does not establish wet-lab validity.
Source: §7 · p. 9
Abstract
from arXiv · showhide
In-silico optimization of nucleic acids relies on predictive oracles that imperfectly capture biological function, so sequences optimized against one model may exploit model-specific artifacts and fail to generalize to independent models. Recent studies have shown that not all designers generalize equally well to independent models. In this work, we use agentic program search to iteratively propose, execute, evaluate, and revise code for nucleic-acid designers. We describe the successful strategies used to prompt the search. AutoNovo designers achieved the best observed mean among the evaluated methods on 3/3 DNA-design tasks in terms of in-silico performance, and 6/6 transfer tasks in terms of independent models. The designers come from a family of discovered nucleic acid designers rather than a single universal best designer. We plan to release the resulting designers. These results are the first steps in creating nucleic acid designers that maximize cross-model generalization, and thus improving the odds that in-silico designed sequences reflect real biology.
1 Introduction
In-silico nucleic-acid design uses predictive models to optimize sequences, but strong model predictions do not always translate to biological performance. AutoNovo applies agentic program search to discover designers, with evaluations aimed at both optimization and generalization across tasks and oracles.
Motivation
- In-silico design optimizes nucleic-acid sequences against predictive sequence-to-function models, but sequences predicted to perform well may not succeed in biological systems.Orthogonal validation uses models trained on the same property but with different training data or architectures; agreement does not guarantee biological performance, while disagreement can indicate oracle-specific artifacts.Source: §1 · p. 1
Research question
- The paper asks whether agentic program search can discover nucleic-acid designers that outperform human-designed baselines and generalize to models not used during search.The question builds on benchmark evidence that designers differ in agreement with orthogonal models.Source: §1 · p. 1
Approach
- AutoNovo uses ERA to iteratively search designer programs with an objective that combines one or two optimization oracles and, in most runs, a hidden orthogonal evaluation oracle.Search prompts ask ERA to reimplement hand-crafted designers and include guidance intended to support task generalization.Source: §1 · p. 2
Results
- AutoNovo designers achieved the best observed mean on 3/3 DNA design tasks and 6/6 cross-oracle transfer tasks, with different candidates performing best across tasks.The results motivate a family of designers rather than one universal best.Source: §1 · p. 2
2 AutoNovo: Agentic program search for sequence designers
AutoNovo uses ERA to search over programs that implement sequence designers, then evaluates the frozen programs for direct performance and generalization. The search starts from hand-crafted algorithm implementations and uses optimization and, often, hidden orthogonal-oracle scores to guide revisions.
Search approach
- AutoNovo has nested optimization loops: ERA searches over programs implementing designers, while each generated designer searches over biological sequences.ERA rewrites candidate programs, executes them end-to-end, reads their metrics, and continues searching; its output is inspectable code.Source: §2 · p. 2
- ERA reimplements four hand-crafted algorithms—GrAdaBeam, FastSeqProp, Ledidi, and Proximal Exploration—from two LLM-generated seeds apiece, then improves the implementations.Each Designer has an oracle and length-preserving ACGT start sequence, a time-bounded run function, and a function that returns its best sequence.Source: §2 · p. 2
Initial experiments
- In initial single-task experiments, FastSeqProp and Ledidi reimplementations exceeded NucleoBench baselines, while GrAdaBeam was not performant; four seeds were selected for later searches.The selected seeds were both FastSeqProp seeds, one Ledidi seed, and one PEX seed.Source: §2 · p. 2
Program searches
- Five ERA searches used one or two optimization oracles; four also used an orthogonal oracle for program evaluation, while another panel oracle was held out.The two-oracle search optimized against BPNet ATAC and ChromBPNet, leaving Enformer held out; the final evaluation contained sixteen AutoNovo designers.Source: §2 · pp. 2–3
Search objective
- ERA evaluates candidate sequences using the optimization score plus a weighted orthogonal score when present; the designer can query the optimization oracle but not the hidden orthogonal oracle.The optimization score combines one or both of BPNet-lite ATAC and ChromBPNet; λ is 1 for BPNet ATAC or ChromBPNet and 0.01 for Enformer, and crashes or substantial overruns incur penalties.Source: §2 · p. 3
Evaluation
- After search, each Designer program is frozen; using it with a new design oracle tests algorithm-level generalization, while scoring its output with another oracle tests sequence transfer.The search gives each candidate 30 minutes to design from each of three start sequences, and ERA minimizes the mean candidate metric.Source: §2 · p. 3
3 Tasks
The study evaluates frozen designers across three DNA oracles and distinguishes algorithm-level generalization from cross-oracle sequence transfer. The oracles assess related accessibility readouts but are imperfectly correlated, making transfer a separate evaluation target.
3 Tasks
- AutoNovo designers are tested with each of three design oracles and scored by all three evaluation oracles; matching design and evaluation oracles gives a direct score, and differing oracles give a transfer score.A completely held-out task measures algorithm-level generalization for each designer.Source: §3 · pp. 3–4
3.1 DNA tasks and oracles
- The DNA panel comprises BPNet ATAC, ChromBPNet, and Enformer, which differ in assay, input length, or architecture.BPNet ATAC uses a 3,000 bp window; ChromBPNet inputs are 2,114 bp; Enformer predicts tracks from 196,608 bp of DNA, with K562 DNase tracks scored over the central BPNet window.Source: §3.1 · p. 4
- Across the same 100 NucleoBench start sequences, DNA-oracle scores are substantially but imperfectly correlated (Pearson r ≈ 0.73–0.81).The pairwise comparisons cover BPNet ATAC, Enformer K562 DNase, and ChromBPNet K562.Source: §3.1 · p. 4, Figure 1 · p. 4
4 Results
With 30-minute CPU design budgets, AutoNovo achieved the best observed mean on all three direct DNA-design tasks and all six cross-oracle transfer directions. Performance varied across discovered designers and task directions.
Evaluation setup
- Results used a 30-minute CPU design budget and mean ± standard deviation scores over 20 start sequences, with lower scores better.One AutoNovo candidate’s Enformer direct result and two Enformer-origin transfer results were unavailable because it ran out of memory.Source: §4 · p. 4, Table 2 · p. 5
Direct performance
- 13 of 16 AutoNovo designers beat the strongest baseline mean on BPNet ATAC, 7 of 16 on ChromBPNet, and 11 of 15 completed candidates on Enformer.The Enformer count excludes one candidate that ran out of memory with Enformer as the design oracle.Source: §4 · pp. 4–5
- The top direct designer differed by task: FSP-gradient memory STE on BPNet ATAC, FSP-Best-Anchored STE on ChromBPNet, and FSP-sparse restart STE on Enformer.The final evaluation included 16 AutoNovo-discovered designers derived from FastSeqProp, Ledidi, and PEX, alongside four baselines.Source: §4 · pp. 4–5
- −61.17 vs −40.88 on BPNet ATAC, −74.88 vs −60.08 on ChromBPNet, and −3360.36 vs −124.53 on Enformer: AutoNovo achieved the best observed means.Lower scores are better; each pair compares the best AutoNovo score with the strongest baseline mean for that task.Source: §4 · pp. 4–5, Table 2 · p. 5
Cross-oracle transfer
- An AutoNovo designer achieved the best observed mean in all six cross-oracle transfer directions, but no single candidate won every direction.Performance was less uniform for ChromBPNet-origin transfers: 4 of 16 candidates beat the strongest baseline mean in each direction.Source: Table 3 · p. 6, §4 · p. 6
Baseline comparison
- Baseline rankings differed from NucleoBench, where runs lasted eight hours; the shorter budget and hyperparameters selected from those longer runs also yielded worse absolute scores.Ledidi designs more quickly than FastSeqProp, AdaBeam, and GrAdaBeam under the study’s shorter budget.Source: §4 · p. 7
5 What Algorithms Did AutoNovo Discover
AutoNovo discovered a family of hybrid continuous–discrete designers that use gradients, preserve strong sequences, and recover from stalled search. Their strongest programs vary across tasks, and transfer and scaling depend on the search configuration and compute budget.
5 What Algorithms Did AutoNovo Discover
- The discovered family commonly combines hard-sequence gradient optimization, best-sequence preservation, and relatively simple restarts after progress slows.The seeds acted as loose priors, though benchmark preferences and language-model priors may also explain convergence toward similar mechanisms.Source: §5 · pp. 7–8
- Every best observed mean in Tables 2 and 3 comes from a FastSeqProp-seeded program, and eight of nine winning entries use a straight-through estimator.The programs optimize continuous logits, convert them into discrete sequences, and retain the best discrete sequence found.Source: §5 · p. 7
- FSP-Best-Anchored STE pulls logits toward the best sequence, FSP-gradient memory STE uses accumulated gradient magnitudes for jumps, and FSP-sparse restart STE restarts near sparse mutations.Each variant performs best on different direct-design or transfer tasks.Source: §5 · p. 7
- Two-oracle searches produced both BPNet-origin transfer winners, including a BPNet ATAC →Enformer winner whose Enformer oracle was absent from the search loop.The BPNet ATAC →ChromBPNet winner had the weakest direct BPNet ATAC score among AutoNovo designers, illustrating that design-oracle score and independent-oracle transfer differ.Source: §5 · p. 7
- FSP-sparse restart STE, searched with ChromBPNet and BPNet ATAC, was the best Enformer designer and won both Enformer-origin transfer tasks.This strategy transferred to an oracle and sequence length not used for its inner-loop optimization during search.Source: §5 · p. 7
- Large candidate batches can run out of memory on Enformer’s longer sequences, while single-trajectory sparse restarts performed strongest for Enformer design.The comparison suggests candidate batching and evaluation frequency affect scaling under fixed wall-clock and memory budgets.Source: §5 · pp. 7–8
- Generated programs converged on similar continuous–discrete mechanisms across seed algorithms, but some contained stale gradients, unused controls, or mismatched comments and conditions.None set random seeds; the authors call for code inspection, reproducibility checks, and targeted ablations.Source: §5 · p. 8
6 Agent and Harness Design Observations
The authors identify prompting and harness choices that helped ERA generate useful designers: seed it with strong algorithmic priors, expose runtime, and make gradient use adaptive. They also penalize failures and provide working gradient examples to keep search focused.
6 Agent and Harness Design Observations
- The prompt bans try/except handlers, and the harness AST-checks the Designer class so handlers receive the worst-case score and crashes surface as failures.Timeouts and crashes also receive large penalties.Source: §6 · p. 8
- Seeding ERA with descriptions of GrAdaBeam, FastSeqProp, Ledidi, and Proximal Exploration substantially improved performance over prompts that only asked it to add gradient information.The prompt also conveyed that gradients help on some tasks and hurt on others, and asked for generalization.Source: §6 · p. 8
- Passing the wall-clock budget and requiring designers to use it improved results over algorithms that did not know the remaining runtime.The interface accepts run(allowed_runtime_secs).Source: §6 · p. 8
- Prompts treat gradients as optional and costly, asking designers to decide whether they are useful for a task or a phase of a run.A backward pass that does not improve performance can consume time in a time-bounded evaluation.Source: §6 · p. 8
- Working PyTorch gradient snippets for each oracle keep ERA from spending generations on framework code.The snippets are provided in notebook cells, not as rewrite targets.Source: §6 · p. 9
7 Conclusion
Across three chromatin-accessibility oracles, AutoNovo designers achieved the best observed mean on all direct-design tasks and all cross-oracle transfer directions. The study’s evidence remains in-silico and limited in biological scope.
7 Conclusion
- An AutoNovo designer achieved the best observed mean on all three direct optimization tasks and all six cross-oracle transfer directions across three chromatin-accessibility oracles.Different generated programs performed best on different oracles, and multiple programs surpassed the strongest baseline on several tasks.Source: §7 · p. 9
- Providing strong algorithmic priors, exposing wall-clock budget, and allowing adaptive gradient use improved ERA’s ability to generate useful designers.The prompt strategies address gradient computational cost and task-dependent usefulness.Source: §7 · p. 9
- The evidence is limited to in-silico evaluation of one biological phenotype using three related predictive models, and cross-oracle transfer does not establish wet-lab validity.The authors call for testing additional design problems and evaluating whether cross-oracle improvements predict biological experiments.Source: §7 · p. 9
A Appendix
The appendix frames the task as iteratively modifying a nucleic-acid sequence to minimize an oracle score while seeking designers that generalize across tasks. It also cautions that gradient value varies by task and can be offset by computation time.
A Appendix
- The designer modifies a nucleic-acid string to minimize the pretrained oracle’s score, with the broader goal of generalizing to different tasks.The task interface uses a Designer class with init, run, and get_samples methods; candidate sequences retain the starting sequence length and use A, C, G, and T.Source: A Appendix · p. 11
- The appendix recommends logging model and gradient computation and avoiding try-except clauses so failures are visible.The supplied instructions state that exception handlers can conceal unexpected behavior.Source: A Appendix · p. 11
- Gradient methods help on some tasks, including GATA2, CTCF, MAC, and ATAC, but perform poorly on Rinalmo, JUNB, and OTX1.The text suggests gradients may be uninformative on some problems or not worth their computation time, and may be more useful at some design stages than others.Source: A Appendix · p. 11