SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarm

1 Introduction

• Poor results can leave unclear whether to revise a hypothesis or its implementation, while a program without overall gains may contain components useful to another agent.The value of a hypothesis depends on its implementation and testing, and intermediate results may become stepping stones when combined with later findings or implementations.Source: Figure 1 · p. 1, §1 · p. 2
• SWARMRSI updates shared, inheritable research states through feedback-guided hypothesis or program revision, cross-agent reuse of retained hypotheses and components, and evaluation of resulting candidates.Each experiment adds useful results, feedback, and unresolved needs to guide later agents.Source: §1 · p. 2
• About 77% of final best programs across 90 NatureBench tasks descend from rebase, crossover, or a feedback-triggered update in one DeepSeek-V4-Pro seed.Biologists reproduce two programs for follow-up analyses of intestinal and splenic immunity.Source: §1 · p. 2
• SWARMRSI achieves SOTA results on OpenProblems and NatureBench and outperforms hypothesis and program evolution agents with the same backbone.The authors also report that biologists reproduce its programs for follow-up analyses.Source: §1 · p. 2

2.1 LLM Agents for Scientific Research

• Scientific research agents span program search, specialized roles, memory, and hypothesis trees; SWARMRSI also grounds hypotheses in experiments and uses other agents’ feedback and partial results.The passage identifies Arbor and SAGE as especially close settings for linking hypotheses, programs, evidence, and interventions.Source: §2.1 · p. 2

2.2 Recursive Self-Improvement of Agents

• Prior RSI methods improve code, policies, workflows, or proposals, whereas SWARMRSI revises research states agents inherit using cross-agent feedback and reused hypotheses and program components.Examples include AIDE2’s accepted code modifications, Dream-RSI’s replay-based exploration policies, MASS’s workflow evolution and fine-tuning, and AQuA’s recorded-experiment-guided proposals.Source: §2.2 · pp. 2–3

3 SwarmRSI

• SwarmRSI updates inheritable research states through feedback-guided mutation, complementary recombination, and a recursive loop that records each candidate’s outcome.Each experiment informs subsequent research through the shared state.Source: §3 · p. 3

3.1 Preliminaries

• A scientific agent is a research policy for proposing hypotheses, developing programs, or analyzing evidence, acting from records in a shared graph.The graph retains tested hypotheses, programs, outcomes, and evidence, including failed or non-improving attempts.Source: §3.1 · pp. 3–4

3.2 Feedback-Guided Swarm Mutation

• Feedback-Guided Swarm Mutation uses other agents’ outcomes and implementation evidence to revise hypotheses when instance differences matter or repair programs after failures.H-mutation changes the hypothesis; P-mutation keeps it and repairs only the implementation.Source: §3.2 · p. 4

3.3 Complementary Swarm Recombination

• Complementary Swarm Recombination rebases retained hypotheses onto valid programs from other agents and crosses over components that perform better on particular instances.Rebase addresses unresolved implementation needs; crossover retains the parent’s components on instances where donors do not outperform it.Source: §3.3 · p. 4

3.4 Recursive Swarm Evolution

• Recursive Swarm Evolution selects between hypothesis and program frontiers using recent improvement per token cost, unresolved-request pressure, and exploration priorities.Every attempt, including failures and outcomes without gain, updates the graph; implementation failures prompt repair, mixed instance outcomes prompt hypothesis revision, and the loop returns the best valid program when the budget ends.Source: §3.4 · pp. 4–5

4.1 Research Questions

• The evaluation asks about performance and token cost, mechanism contributions, how feedback and reused results guide research, and the frequency of failure modes.Source: §4.1 · p. 5

4.2 Benchmarks

• The benchmarks cover six OpenProblems single-cell analysis tasks, 90 NatureBench scientific-coding tasks across six domains, and 22 low-complexity MLE-Bench Lite competitions.Source: §4.2 · pp. 5–6
• Figure 3 encodes Match-SOTA and Any-Medal as full bars and Surpass-SOTA and Gold as solid portions; Table 4 groups NatureBench ablations by removed mechanism.Source: Figure 3 · p. 6, Table 4 · p. 6

4.3 Baselines

• Baselines include program- and hypothesis-evolution agents, domain-specific scientific agents, and general coding agents.Source: §4.3 · p. 6

4.4 Evaluation Metrics

• Metrics include normalized OpenProblems scores, NatureBench Match-SOTA and Surpass-SOTA rates, MLE-Bench Lite median and medal rates, and uncached input and output token costs.Rates keep invalid or missing results in the task denominator; some NatureBench comparisons use paired bootstrap intervals and McNemar tests, while others are descriptive.Source: §4.4 · p. 6

4.5 Implementation Details

• SWARMRSI and rerun baselines use DeepSeek-V4-Pro under matched benchmark settings, with results averaged over three seeds; a separate evaluation uses Gemini 3.1 Pro Preview.Agents in each run share one backbone and exchange results through the shared graph.Source: §4.5 · p. 6

5.1 Main Results (RQ1)

• SWARMRSI achieves SOTA on OpenProblems and NatureBench, outperforms same-backbone hypothesis and program evolution agents, and remains strong across backbones.With Gemini 3.1 Pro Preview, it reaches an OpenProblems average of 76.2 and retains an 81.8% Any-Medal rate on MLE-Bench Lite.Source: §5.1 · p. 6

5.2 Ablation Studies (RQ2)

• Removing H- or P-mutation lowers Match-SOTA by 10.0 and 13.3 pp; removing recombination lowers it 6.7 pp and Surpass-SOTA from 15.6% to 7.8%.Without the shared graph Match-SOTA drops 7.8 pp; without adaptive H/P selection it drops 8.9 pp, also the largest mean-g decline.Source: §5.2 · p. 7

5.3 Trajectory Analysis (RQ3)

• 35% of failed implementations recover to a new best, versus 12% for AIDE and 5% for Arbor; 62% of proposed hypotheses lead to improvement, versus 17% for Arbor.Figure 4 reports these search diagnostics across 90 NatureBench tasks.Source: §5.3 · p. 7, Figure 4 · p. 7
• About 77% of final best programs descend from a rebase, crossover, or feedback-triggered update; inherited state raises held-out Match-SOTA from 41.7% to 86.1%.The reuse share is from one DeepSeek-V4-Pro seed over all 90 NatureBench tasks; held-out results compare inherited state after three source-task generations with cold start.Source: §5.3 · p. 7

5.4 Failure Analysis (RQ4)

• 93.3% of annotated failures are faithful interventions without gains, while implementation or execution problems account for 6.7%.On task 00838, downstream code overwrites a proposed amino-acid substitution, so evaluation never tests it.Source: §5.4 · p. 8

5.5 Biological Case Studies

• Helminth day-10 enterocytes have higher MHC-II scores than Salmonella (1.17 vs 0.34), and their representation has greater locally adjusted MHC-II coherence than RNA PCA.Antimicrobial scores are less locally coherent than on RNA PCA in each of the four sources.Source: §5.5 · p. 8
• g rises from 0.045 to 0.102 with the spatial program, but matched marginal-zone protein markers show no consistent neighbourhood shift across three images.The B220 effect changes sign across images.Source: §5.5 · p. 8

• Scientific swarms face two obstacles: a failed experiment does not reveal whether its hypothesis or implementation was at fault, and a weak overall program may still contain useful components.Source: §6 · p. 8
• Feedback-Guided Swarm Mutation uses other agents’ feedback to choose between hypothesis revision and program repair, while Complementary Swarm Recombination carries retained hypotheses and useful components into other agents’ work.Recursive Swarm Evolution returns outcomes to the shared graph, changing the states later agents start from.Source: §6 · p. 8
• SWARMRSI achieves SOTA on OpenProblems and NatureBench, outperforms same-backbone baselines, and its programs are reproduced for intestinal and splenic immunity analyses.Source: §6 · pp. 8–9

A Method Details

• The search loop selects a frontier and an operation, runs planning, coding, evaluation and audits, updates the shared graph, and returns the best valid program.Source: A Method Details · p. 14

A.1 Recursive Swarm Evolution

• The hypothesis frontier balances active hypotheses across mechanism families, while the program frontier retains the incumbent and the best program from each realized family.H-mutation changes the hypothesis; P-mutation keeps it and refines an existing program lineage, while discounted improvement-per-cost and unresolved requests guide frontier allocation.Source: §A.1 · p. 14

A.2 Execution and Feedback

• Code and scientific audits check execution, task-contract validity, and implementation evidence; flagged or unscored attempts retain program-level evidence and unresolved requests.Implementation failures request deeper program refinement; mixed per-instance gains and losses request broader hypothesis exploration.Source: §A.2 · p. 15

A.3 Complementary Swarm Recombination

• Rebase keeps the hypothesis but selects a different program base by need-label overlap; crossover selects up to two donors by per-instance advantage and assigns their components to instances they win.Source: §A.3 · p. 15

B.1 Benchmarks

• The evaluation covers six OpenProblems tasks, 90 NatureBench tasks across six research areas, and 22 low-complexity MLE-Bench Lite competitions.Source: §B.1 · p. 15

B.2 Baselines

• Comparisons include published and leaderboard results; matched-backbone reruns cover selected baselines, with AIDE serving as the NatureBench program-evolution representative.MLEvolve is not rerun on NatureBench because its harness uses MLE-Bench’s competition submission and grading format.Source: §B.2 · pp. 15–16

B.3 Evaluation Details

• NatureBench Match-SOTA and Surpass-SOTA count tasks with g ≥ 0 and g > 0.1, respectively; both retain invalid and missing tasks in the denominator.Task-level g averages direction-adjusted instance improvements; MLE-Bench Lite Any-Medal counts competitions earning any medal, with invalid or missing submissions receiving none.Source: §B.3 · p. 16
• Benchmark metrics are averaged equally across three seeds, while published references retain their original reporting protocols; task time limits and hardware are matched within each benchmark.OpenProblems scores are normalized and scaled by 100; NatureBench uses judge-valid Match-SOTA and Surpass-SOTA rates, while MLE-Bench Lite uses official private-test grading.Source: §B.3 · p. 16, §B.4 · p. 17

B.7 Statistical Reliability

• 7.8 pp over Arbor (95% CI [2.2, 13.3]) and 16.7 pp over AIDE (95% CI [8.9, 24.4]) are the DeepSeek-V4-Pro NatureBench Match-SOTA improvements.Paired bootstrap intervals exclude zero; exact McNemar tests give p-values of 0.015625 and 0.0000610, respectively.Source: §B.3 · p. 16

B.5 Ablation Settings and Aggregation

• Ablations share the planner, coder, evaluator, and search budget; they disable graph reuse, one mutation mode, recombination, or adaptive hypothesis/program selection.Table 6 reports task-paired bootstrap 95% confidence intervals for mean-g drops, and all intervals exclude zero.Source: §B.5 · p. 17, §B.7 · p. 17, Table 6 · p. 17

B.6 Performance and Token Cost

• Figure 6 compares benchmark performance with mean tokens per task for SWARMRSI and rerun baselines using DeepSeek-V4-Pro.OpenProblems uses normalized average scores scaled by 100; NatureBench and MLE-Bench Lite use Match-SOTA and Any-Medal rates, respectively.Source: §B.6 · p. 17, Figure 6 · p. 17

B.7 Statistical Reliability

• † methods use three seeds, with means and Student’s t-distribution confidence intervals; thresholded rates remain identical across seeds because no task crosses a success boundary.Underlying g scores vary across seeds, but no task crosses Match-SOTA or Surpass-SOTA thresholds and no MLE-Bench Lite competition crosses an Above-Median or medal boundary.Source: §B.7 · p. 17

C.1 Comparative Trajectories

• In the 00917 trajectory, hypothesis mutations that changed the executed readout reached g = 0.0928 and 0.1807, while earlier program mutations did not improve the incumbent.In 00639, rebase activated message-passing and ranking-loss paths already present in a candidate but not executed, raising mean g from 0.0575466 to 0.0644094.Source: §C.1 · pp. 18–19
• On 00795, a retained hypothesis was tested on a later program base, improving target Fmax from 0.5032 to 0.5087; later crossover and refinement reused and improved transferred predictors.The later refinement reached Fmax 0.5133 while preserving the other four instance scores.Source: §C.1 · p. 19, Table 10 · p. 19
• About 77% of runs’ final best programs descend from a rebase, crossover, or feedback-triggered update; those update types have higher new-best rates than other updates at every search stage.In one complete seed spanning 90 tasks, feedback-triggered updates and rebase comprise 296 attempts and produce 84 new-best candidates; including crossover, the union comprises 731 attempts and 173 new bests.Source: §C.1 · p. 19, Table 11 · p. 20

C.2 Cross-Task Transfer

• After three generations of source-task research, held-out NatureBench mean g rises from 12.01 at cold start to 21.89, Match-SOTA from 41.7% to 86.1%, and Surpass-SOTA from 33.3% to 58.3%.Tokens per task fall from 12.17M to 10.94M; gains are largest in the first two generations, and Gen 3 matches Gen 2 on Match-SOTA while improving mainly through Surpass-SOTA.Source: §C.2 · p. 20, Table 12 · p. 20

C.3 Aggregate Search Behavior

• SWARMRSI’s stagnation rate is 43% on NatureBench, compared with 63% for AIDE and 70% for Arbor.Stagnation is the share of scored candidates that do not improve the preceding best valid result; failure recovery and hypothesis-improvement proportions are also compared across the three methods.Source: §C.3 · p. 20

D Human Failure Analysis

• We reviewed 644 attempts across 35 failed SWARMRSI trajectories in NatureBench and MLE-Bench Lite, assessing each against its best preceding valid result using the benchmark’s metric direction.OpenProblems trajectories were excluded because the benchmark reports continuous normalized scores without a binary success criterion.Source: D Human Failure Analysis · p. 20
• Two bioinformatics PhD annotators reviewed attempts using intervention, implementation, execution, evaluator, and prior-score evidence, achieving Cohen’s κ = 0.92.A faithful, valid intervention without improvement counted as no gain; unreliable testing due to code, interface, interruption, or proposal–implementation mismatch counted as implementation or execution failure.Source: D Human Failure Analysis · pp. 20–21
• The task 00838 substitution was overwritten before evaluation, whereas task 02316’s successfully executed spatial mixture collapsed locations into one cluster and yielded ARI zero.The first is an implementation failure despite a valid score; the second is a no-gain outcome.Source: D Human Failure Analysis · p. 21

E.1 Enterocyte: Infection-Associated Expression Variation

• In pooled enterocytes, helminth day-10 MHC-II mean expression is 1.168, compared with 0.335 for Salmonella and 0.202 for controls.The interferon means are 0.356, 0.438, and 0.150, respectively; the condition means weight each sequencing source by its enterocyte count.Source: §E.1 · p. 21, Table 13 · p. 21
• Within the two helminth day-10 sources, adjusted MHC-II local coherence is higher on the reproduced representation than RNA PCA, while antimicrobial coherence is lower in all four infection sources.The analysis correlates each cell’s RNA program score with the mean score of its ten nearest same-source enterocyte neighbours; Figure 7’s correlations are unadjusted, while Table 14 reports covariate-adjusted values.Source: §E.1 · p. 22, Figure 7 · p. 23, Table 14 · p. 23

E.2 CytoCommunity: Spatial Context and Protein Follow-up

• The selected program raises task-level g from 0.04507 to 0.10215, but the expert follow-up separately analyzes measured protein intensities in the three healthy spleen images.The program uses phenotype-derived anatomical priors, neighbourhood composition and interface features, Gaussian EM, and an anatomically constrained spatial MRF.Source: §E.2 · p. 24
• E1–E3 are post-hoc groups defined by local cell composition, and the primary E2-versus-E1 comparison is restricted to the manual marginal zone.The groups are distinct from the program’s anatomical partitions; Figure 9 shows group proportions and six matched protein contrasts.Source: §E.2 · p. 24, Figure 9 · p. 24
• B220 effects for matched E2–E1 neighbourhood contrasts are +0.082, −0.130, and −0.190 across the three images, and neighbourhood labels add little predictive information for six protein markers.The matched protein contrasts use 815, 995, and 1,025 pairs; spatially blocked validation gives ∆R2 from −0.00098 to +0.00967 across 18 image–protein combinations.Source: §E.2 · p. 24, Figure 9 · p. 24

F Limitations

• SWARMRSI requires executable candidate-program evaluation, while execution feedback may not verify faithful implementation and performance gains alone do not establish scientific mechanisms.Rebase matches implementation needs to program mechanism labels; crossover requires per-instance scores, and H/P selection adapts discounted UCB without its guarantees, with benefits established only empirically.Source: F Limitations · p. 25
• Evaluation covers matched-budget comparisons on three benchmark families; cross-task evidence uses six source tasks, 12 held-out tasks, and one backbone, with generations confounded by source-task count.The human failure analysis covers 644 annotated failed attempts from 35 task trajectories.Source: F Limitations · p. 25
• The evaluation does not separately assess literature synthesis, idea quality, or peer-review quality, and complete research workflows and information-access agents require additional evaluation.Other untested settings include legal reasoning, zero-shot entity extraction, conversational recommendation, and shopping research, which require task-specific objectives and protocols.Source: F Limitations · p. 25

G Ethics Statement

• Benchmark gains do not establish factual reliability or alignment with human preferences, and the study does not assess robustness to shortcuts, stereotypes, cognitive biases, or harmful-content generation.Applications affecting people should preserve user agency and provide domain-appropriate human oversight.Source: G Ethics Statement · p. 25
• Generated programs and scientific interpretations require independent validation beyond benchmarks; the audits do not certify security or scientific correctness, and biological associations establish neither causal mechanisms nor clinical validity.Deployment should restrict code execution, external-tool access, and credentials to approved scope and review proposed uses for potential harm.Source: G Ethics Statement · p. 25
• Data reuse should respect source licenses, access restrictions, and privacy requirements, with provenance and intended uses documented.Token consumption supports resource-aware comparisons but does not directly measure energy use or carbon emissions.Source: G Ethics Statement · p. 25