MoLRSI: A Recursive Self-Improving AI agent for Molecular Discovery

1 Introduction

• Molecular workflows must coordinate models, training objectives, and evaluation procedures to balance properties such as activity, toxicity, specificity, and novelty.Workflow and model choices determine which candidates advance to experimental testing.Source: §1 · pp. 1–2
• Effective workflow development requires searching a large configuration space that varies across targets, data, and desired properties, while wet-lab evaluation is expensive.Experience across projects can help researchers choose strong starting points and prioritize useful changes.Source: §1 · p. 2
• MolRSI combines validated executable skills, experience-guided workflow search, and persistent memory of outcomes to improve later workflow selection, refinement, and execution.Memory includes workflow performance, modification outcomes, and execution failures and repairs.Source: §1 · p. 2
• MolRSI achieved a 15.9% antibiotic hit rate versus 8.7% for the previous expert-driven workflow using the same training data and screening library.The pMHC workflows produced eight hits with preferential binding to intended targets over tested off-targets, while Claude Science yielded none.Source: §1 · pp. 2–3

2.1 Overview of MolRSI

• MolRSI combines an Explore Agent’s validated executable skills with experience-guided Monte Carlo tree search and persistent memory of workflows, refinements, and execution outcomes.Memory informs workflow selection, refinement, resource allocation, and error repair on current and later tasks.Source: §2.1 · p. 3

2.2 MolRSI for antibiotic discovery

• Across six antibacterial tasks, RSI improved mean test AUPRC on all six, and zero-search transfer outperformed search without RSI on three targets.With experience from all 25 source tasks, transferred mean test AUPRC was approximately 0.37, exceeding 0.361 for search without RSI after 20 iterations.Source: Figure 2 · p. 5, §2.2 · p. 6
• The antibiotic workflow combined optimized activity and cytotoxicity predictors with structural filters and diversity-based selection to choose 107 compounds for testing.It used the same training datasets and approximately 12.1 million-compound screening library as the previous expert-driven study.Source: §2.2 · p. 7
• MolRSI’s antibiotic workflow yielded 17 hits among 107 tested compounds, a 15.9% hit rate versus 8.7% for the prior expert-driven workflow.Both workflows used the same training datasets and screening library; on a shared held-out set, MolRSI also achieved AUROC 0.828 versus 0.590 and AUPRC 0.300 versus 0.090.Source: §2.2 · p. 7

2.3 MolRSI for de novo peptide–MHC binder design

• On 14 miniprotein-binder tasks, MolRSI achieved macro EF@1% 8.01 versus 7.62 for the next-best agent and EF@2% 7.29 versus 6.71.For held-out MART-1, accumulated experience yielded EF@1% 3.65 and EF@2% 5.55; without prior-task experience, both were 0.00.Source: Figure 4 · p. 9, §2.3 · p. 10
• In 30-round pMHC design searches, the best objective reached 6.76 for MART-1 and 7.56 for SARS-CoV-1, improvements of 66% and 28% over initial configurations.MolRSI designs had higher median on-target scores than Claude Science designs under both scoring measures.Source: Figure 4 · p. 9, §2.3 · p. 10
• MolRSI yielded five MART-1 and three SARS-CoV-1 pMHC hits, while Claude Science yielded none among evaluable designs.Hits required strong on-target enrichment exceeding the highest measurable off-target enrichment.Source: §2.3 · p. 11, Figure 5 · p. 11

3 Discussion

• MolRSI’s recursive process links task-level workflow refinement with cross-task changes to the policy guiding later searches.Performance on related tasks informs exploration priorities; current-task evaluations, modification outcomes, and execution feedback further update them.Source: §3 · p. 12
• Under matched conditions, prior-task experience improved selected workflows for antibacterial prediction and pMHC ranking, and the pMHC workflows produced binders preferentially enriched for intended targets over tested off-targets.Claude Science yielded no hits under the same pMHC screening conditions.Source: §3 · p. 12
• Comparing prior-experience search, search from scratch, and direct transfer can distinguish inherited workflow value from the benefit of further development.The authors propose that future benchmarks assess how effectively agents adapt their search policies over successive tasks.Source: §3 · pp. 12–13
• Whether policy updates remain useful as biological targets and evaluation criteria change is an open boundary, and biological validation is needed to establish whether better computational scores yield more effective candidates.The authors suggest incorporating experimental outcomes into future policy updates.Source: §3 · p. 13

4.1.1 Resource library construction

• MolRSI’s Explore Agent builds a resource library of reusable skills from relevant papers, code, tools, benchmarks, and algorithms, validating skills before retaining them for use.Skills that still fail after repair are removed from the library.Source: §4.1.1 · p. 13

4.1.2 Persistent memory

• Persistent memory retains executable workflows and scores, workflow revisions and their outcomes, and execution requirements, errors, and repairs for use in later tasks.Refinement records changes whether performance improves, declines, or stays unchanged; execution records include the conditions and outcomes of repairs.Source: §4.1.2 · pp. 13–14

4.1.3 Recursive self-improvement of the search policy

• Updated memories guide subsequent searches within and across tasks, while MolRSI can also transfer a source-task workflow directly or use it to start a new search.Search ends at the task-specific evaluation or computation limit, and the returned workflow is the successfully evaluated one with the best development score.Source: §4.1.3 · pp. 14–15
• MolRSI uses experience-guided MCTS to select workflow families and parents, propose modifications, execute and evaluate candidates, and update its memories after each attempt.Family selection combines current-task performance, an exploration bonus, and historical transfer scores; modification and repair outcomes also inform later iterations.Source: §4.1.3 · pp. 14–15

4.2.1 Datasets and benchmark evaluation

• Six antibacterial prediction tasks used five similarity-based data splits each, with mean validation AUPRC guiding workflow search and test AUPRC reported across splits.The six targets were S. aureus, P. aeruginosa, A. baumannii, E. coli, K. pneumoniae, and N. gonorrhoeae; test outcomes did not guide search.Source: §4.2.1 · p. 16

4.2.2 Evaluation of recursive self-improvement

• The RSI comparison initialized one MolRSI condition with experience from 25 source tasks and the other without prior-task experience, while updating memory during each search.Antibacterial-task experience was not passed between those tasks; zero-search transfer applied a source-task workflow without further workflow search.Source: §4.2.2 · pp. 16–17

4.2.3 Virtual screening and compound selection

• The MRSA screening workflow filtered candidates for structural similarity, alerts, and predicted cytotoxicity, then ranked survivors with a weighted ensemble and selected candidates across structural clusters.The ranking weights were 0.55, 0.30, and 0.15 for two MiniMol-based variants and a LANTERN–RADR ensemble, respectively.Source: §4.2.3 · p. 17

4.2.4 Experimental validation

• Antibacterial validation tested compounds against MSSA and MRSA, defining hits as compounds with MSSA MIC ≤32 µg mL−1; cytotoxicity was measured in three human cell types.MIC was the lowest concentration producing at least 90% growth inhibition; cytotoxicity assays estimated CC50 from viability measurements.Source: §4.2.4 · pp. 17–18

4.3.1 Datasets and benchmark evaluation

• The protein study accumulated source experience from 14 miniprotein-binder tasks, developed a pMHC ranking workflow using gp100 data, and reserved MART-1 for benchmark testing.The miniprotein tasks used leave-one-target-out evaluation across 14 targets.Source: §4.3.1 · p. 18

4.3.2 Evaluation of recursive self-improvement

• Ranking workflows were selected using AUPRC for miniprotein tasks and AUROC for the pMHC task, with comparisons against other agents and a memory ablation isolating prior cross-task experience.The benchmark budget was 300 A100 GPU-hours and 20 attempted workflow evaluations per task.Source: §4.3.2 · p. 19

4.3.3 Binder generation and candidate selection

• For each target, MolRSI optimized generation methods and settings while holding the learned ranking procedure fixed, then selected designs using specified binding, structure, contact, and on/off-target score thresholds.Each search allowed up to 30 rounds and 1,000 A100 GPU-hours; final candidates were ranked by the on-target frozen-verifier score.Source: §4.3.3 · pp. 19–20

4.3.4 Experimental validation

• Experimental validation used phage libraries with positive selection for on-target pMHC binding and negative selection against off-target pMHC binding, measuring enrichment by sequencing.The tested complexes were HLA-A*02:01 presenting MART-1 or SARS-CoV-1 peptide.Source: §4.3.4 · p. 20

• Molecular-property prediction datasets came from the Therapeutics Data Commons.Source: Data Availability · p. 20
• Antibacterial-activity and human-cell cytotoxicity datasets, including the 271-compound held-out S. aureus test set, came from the authors’ previous study.Source: Data Availability · p. 20
• Miniprotein-binder datasets came from Cao et al. and Bennett et al., while pMHC binder datasets from Liu et al. are available upon reasonable request.Source: Data Availability · p. 20

• Figure S1 links solution, refinement, and execution memories to workflow selection and transfer, modification proposals, resource allocation, and successful repairs.Its examples illustrate stored information types rather than new performance measurements.Source: Figure S1 · p. 30
• Figure S2 contrasts expert-driven and MolRSI screening sequences using the same prospective training datasets and screening library; the campaigns tested 241 and 107 compounds, respectively.Final compound selection included documented exceptions to the nominal cytotoxicity thresholds.Source: Figure S2 · p. 31
• Figure S3 compares macro EF at four top-ranked fractions and macro AUPRC across agent workflows and fixed expert-defined objectives, averaging EF cutoffs across 14 held-out targets.Each task used 13 targets for workflow search and the remaining target for testing, without cross-task memory.Source: Figure S3 · p. 32
• Figure S4 compares MART-1 test AUPRC for MolRSI with and without RSI, agent baselines, and fixed expert-defined objectives.Workflows were selected using two gp100 collections and frozen before MART-1 evaluation; the dashed line marks the positive fraction, 27/3,845.Source: Figure S4 · p. 33

S1.1 Resource library construction

• The Explore Agent retrieves and ranks task-relevant resources, examines code and documentation to construct skills, and organizes them in a shared index with implementations, dependencies, examples, and source references.Construction checks address package identity, syntax, imports, dependencies, and selected interfaces; first-use execution validation repairs examples or removes unsuccessful skills.Source: S1.1 Resource library construction · p. 34, Table S1 · p. 35

S1.2 Persistent memory

• Solution memory records workflow configurations, task conditions, execution status, and development scores, while refinement memory links proposed changes and rationales to outcomes, including declines and failures.When several components change together, the outcome is associated with the combined modification.Source: S1.2 Persistent memory · pp. 34–35
• Execution memory retains runtime and resource measurements, failure details, attempted repairs, and rerun outcomes to guide later resource allocation and repairs.Memory updates after each execution attempt, but held-out test scores are excluded from updates and search decisions.Source: S1.2 Persistent memory · p. 35

S1.3 Recursive self-improvement of the search policy

• The search forest represents concrete workflows as nodes and proposed modifications as edges; family and parent selection combine current-task evidence with historical transfer and exploration.Historical scores are oriented, normalized within source tasks, mapped to bounded utilities, and aggregated with task-similarity weights.Source: S1.3 Recursive self-improvement of the… · p. 36
• Task-similarity weights combine metric compatibility, task type, dataset-size similarity, and description similarity, and transfer scores aggregate eligible source-task workflow evidence.Tasks without eligible records are excluded; defined transfer scores lie in [−1, 1].Source: S1.3 Recursive self-improvement of the… · pp. 36–37
• The agent retrieves prior modifications to propose feasible workflow changes, executes and repairs candidates within task limits, then updates memory and search decisions from outcomes.Direct transfer can return a source workflow without comparing alternatives; otherwise, the retrieved workflow can initialize a new search.Source: S1.3 Recursive self-improvement of the… · p. 38

S2.1 Shared evaluation protocol

• Agents received shared task inputs, development data, objectives, and evaluation allowances, but used native tools and procedures without a common workflow search space.Searches proceeded sequentially with development feedback; candidate code, outputs, status, logs, runtime, scores, and test predictions were retained, and failed or timed-out evaluations counted toward the allowance.Source: S2.1 Shared evaluation protocol · p. 39

S2.2 Antibacterial task instructions and permitted tools

• Across six antibacterial targets, agents had 20 attempted workflow evaluations per task, selecting by mean validation AUPRC across five fixed splits before test scoring.The five splits were covered by one candidate evaluation, and test predictions were scored only after search.Source: S2.2 Antibacterial task instructions… · pp. 39–40

S2.3 Protein task instructions and permitted tools

• Protein ranking agents selected workflows using AUPRC on 13 search targets for miniproteins and AUROC on two gp100 collections for pMHC, before held-out evaluation.Claude Science, Biomni, STELLA, and Codex were asked to produce a scalar score for each target–binder complex.Source: S2.3 Protein task instructions and… · p. 43
• For prospective binder design, Claude Science worked independently per target, without target coordinates, and was asked to return 500 ranked designs under specified scoring and diversity constraints.Its budget was 1,000 A100 GPU-hours per target, matching MolRSI; prompts supplied target sequences and MSAs, peptide sequences, binder-length limits, epitope hints, and novelty references.Source: S2.3 Protein task instructions and… · pp. 43–44
• The protocol allows at most 20 candidate workflows, evaluated one at a time, with each next candidate guided by prior search scores and selection based on mean search AUROC.Each workflow is evaluated on both search collections; test labels and performance must not inform tuning or selection.Source: S2.3 Protein task instructions and… · pp. 45, 47
• AUROC ranks designed pMHC miniprotein binders on two gp100 search collections, while unrelated MART-1 is reserved as the held-out test collection.The search collections contain 2,611 and 3,271 candidates; the test collection contains 3,845 candidates.Source: S2.3 Protein task instructions and… · pp. 45–46
• Only 18 positives occur across the search collections, making small AUROC differences unreliable; the test peptide is unrelated, and all negatives are highly ranked designs.The two search collections share an HLA-A2 scaffold but differ in peptide, limiting transfer from gp100-specific features; the benchmark does not represent designed-versus-random separation.Source: S2.3 Protein task instructions and… · p. 46
• Claude Science instructions were adapted from Anthropic’s published multi-target binder-design prompt, with pMHC-specific modifications and run settings described in Section S4.5.Source: S2.3 Protein task instructions and… · p. 47

S3.1 Datasets and benchmark evaluation

• The experience pool comprised 25 source tasks—22 TDC ADMET and three cytotoxicity tasks—and initialized searches on six antibacterial targets.The cytotoxicity tasks covered HepG2, HSkMC and IMR-90 cells.Source: S3.1 Datasets and benchmark evaluation · p. 48, Table S3 · p. 48
• AUPRC assessed rare-positive retrieval across six datasets of 38,650–39,390 entries, with positives ranging from 109 for P. aeruginosa to 1,321 for N. gonorrhoeae.Repeated SMILES entries were retained in benchmark files.Source: S3.1 Datasets and benchmark evaluation · p. 49, Table S4 · p. 49
• Each antibacterial and cytotoxicity task used five fixed 8:1:1 splits formed by Morgan-fingerprint clustering at a Tanimoto threshold of 0.40.Within each seed, training, validation and test sets had no overlapping dataset identifiers or exact SMILES strings; the five seeds repartitioned the same data.Source: S3.1 Datasets and benchmark evaluation · p. 49
• 38,583 (98.2%) of 39,274 distinct SMILES strings appeared in all nine tasks, despite source and target tasks differing in biological endpoints.The overlap analysis used exact SMILES matching without additional structure standardization.Source: S3.1 Datasets and benchmark evaluation · p. 49
• Mean validation AUPRC across five splits selected final workflows, while held-out test outcomes were excluded from search feedback and workflow selection.Test AUPRC was summarized as the mean and sample standard deviation across the five splits.Source: S3.1 Datasets and benchmark evaluation · p. 51

S3.2 Evaluation of recursive self-improvement

• RSI searches began with pooled experience from source tasks, whereas no-RSI searches lacked prior-task experience; both conditions updated memory during each target task.The conditions shared the resource library, backbone model, partitions, evaluation procedure and 20-attempt search allowance.Source: S3.2 Evaluation of recursive… · p. 51
• The evaluation compared zero-search transfer with searched workflows and retrospectively analyzed transfer across 150 source–target evaluations and source-task pool sizes.Pool-composition percentile bands describe variation across sampled pools, not independent search replicates or confidence intervals for the routing method.Source: S3.2 Evaluation of recursive… · pp. 51–52

S3.3 Case studies of experience-guided search

• The AMES ensemble’s four components were transferred unchanged, then target-task fitting and validation selected among five inherited configurations.The selected configuration beat the unchanged ensemble, whose A. baumannii test AUPRC was 0.4742.Source: S3.3 Case studies of experience-guided… · p. 52
• With target validation selecting among five inherited configurations before test evaluation, LANTERN–RADR achieved mean test AUPRC 0.5356 on A. baumannii, exceeding the 20-step search’s best observed 0.4988 by 0.0368 (7.4% relative).Source: S3.3 Case studies of experience-guided… · p. 52
• The 20-step search’s best observed test AUPRC rose from 0.3716 at step 1 to 0.4988 at step 7, with later evaluations not exceeding it.Refinements changed features, prediction heads, imbalance handling, and training; later evaluations also tested ensembles and other modifications.Source: S3.3 Case studies of experience-guided… · p. 52, Table S7 · p. 53

S3.4 Virtual screening and compound selection

• Screening combined separate activity and cytotoxicity predictors with structural-novelty filtering, activity ranking, Butina clustering, and diverse picking.The activity ranking combined two MiniMol variants and a LANTERN–RADR ensemble weighted 0.55, 0.30, and 0.15.Source: S3.4 Virtual screening and compound… · p. 54

S3.5 Comparison with the previous expert-driven workflow

• MolRSI filtered structural novelty before cytotoxicity prediction and activity ranking, while the expert workflow applied activity prediction first.The expert workflow tested 241 filtered candidates; MolRSI ranked candidates with a weighted activity ensemble and selected diverse compounds across Butina clusters.Source: S3.5 Comparison with the previous… · p. 54

S3.6.1 Antibacterial activity assays

• MSSA screening began at 128 µg mL−1, and growth-inhibiting compounds advanced to an eight-point, two-fold dose–response series from 128 to 1 µg mL−1.MIC was the lowest concentration producing ≥90% growth inhibition relative to the untreated control.Source: S3.6.1 Antibacterial activity assays · p. 55

S3.6.2 Cytotoxicity assays

• Cytotoxicity was assessed in HepG2, HSkMC, and IMR-90 cells using PrestoBlue viability readouts and CC50 estimates from the 50% crossing.Untreated wells defined 100% viability, and Triton X-100 lysis wells defined the assay floor.Source: S3.6.2 Cytotoxicity assays · pp. 55–56, Table S8 · p. 55

S3.6.3 Predictive performance and experimental hit rates

• 17/107 (15.9%) MolRSI-selected compounds and 21/241 (8.7%) expert-selected candidates met the MSSA MIC hit threshold of ≤32 µg mL−1.The hit proportions used campaign-specific denominators; no between-group hypothesis tests were reported.Source: S3.6.3 Predictive performance and… · p. 56

S4.1 Datasets and benchmark evaluation

• Miniprotein workflows were searched on 13 targets and evaluated on the held-out target; pMHC workflows were searched using gp100 collections and tested on MART-1.Held-out labels and performance were excluded from workflow search and selection.Source: S4.1 Datasets and benchmark evaluation · p. 57, Table S9 · p. 57, Table S10 · p. 58

S4.2 Fixed expert-defined ranking objectives

• Fixed expert-objective benchmarks compared internal ranking criteria from BoltzGen, BindCraft, and PXDesign on shared target–binder sequence pairs.The comparison did not reproduce generator-specific filters, iterative sequence optimization, or final diversity selection, so it was not an end-to-end pipeline comparison.Source: S4.2 Fixed expert-defined ranking… · p. 58

S4.3 Ranking benchmark results

• Across 14 miniprotein targets, MolRSI achieved macro EF values of 8.01 at 1% and 7.29 at 2%, and the highest macro AUPRC at 0.4305.Claude Science led at 5% EF (5.56 versus 5.55), while MolRSI led at 10% EF with 4.11.Source: S4.3 Ranking benchmark results · pp. 58–59, Table S11 · p. 59
• 0.0172 AUPRC on held-out MART-1 was the highest among agent workflows, versus a 0.0070 positive-fraction reference and 0.0122 for BindCraft.MolRSI also had the highest EF among agent baselines at all four cutoffs and exceeded fixed objectives in AUPRC and three of four EF cutoffs.Source: S4.3 Ranking benchmark results · p. 62, Table S12 · p. 62
• Prior miniprotein-binder memory increased MART-1 AUPRC from 0.0125 to 0.0172, approximately 37%, under matched search conditions.EF increased from 0.00 to 3.65 at 1% and from 0.00 to 5.55 at 2%.Source: S4.3 Ranking benchmark results · p. 62, Table S13 · p. 62

S4.4 Binder generation and candidate selection

• For prospective design, MolRSI searched generation methods and settings, then retained designs passing on-target quality, HLA-contact, and on/off-target score-difference thresholds.Final candidates were ranked by s8_on after requiring on_ipsae_min ≥0.50, monomer_plddt ≥0.80, at least 15 HLA contacts, and s8_on − s8_off > 0.Source: S4.4 Binder generation and candidate… · p. 63

S4.5 Comparison with reference design workflows

• Claude Science and MolRSI received 1,000 A100 GPU-hours per target, with target specifications and objectives held fixed while each system retained its native design procedure.MolRSI’s 30-round allowance applied to workflow search; Claude Science instead controlled job scheduling within the shared budget.Source: S4.5 Comparison with reference design… · p. 64