PaperlayerView exact revision on arXiv

Source-linked AI summary

STELLA: Self-Evolving LLM Agent for Biomedical Research

Ruofan Jin, Zaixi Zhang, Mengdi Wang, Le Cong

arXiv:2507.02004v1cs.AIcs.CLq-bio.BM

TL;DR

Biomedical research resources are fragmented and constantly evolving, while AI agents often rely on static, manually curated toolsets. STELLA pairs a multi-agent architecture with an evolving Template Library and Tool Ocean; it achieves state-of-the-art results across biomedical benchmarks and improves with experience.

  • Problem

    Biomedical research resources are numerous, dispersed, and constantly evolving, while AI agents typically rely on manually curated, static toolsets.

    Source: Introduction · p. 2

  • Method

    STELLA uses Manager, Developer, Critic, and Tool Creation Agents with an evolving Template Library and a Tool Ocean that can expand through new tools.

    Source: Introduction · p. 2

  • Results

    STELLA achieved state-of-the-art results across biomedical benchmarks, including 26% on Humanity’s Last Exam: Biomedicine, and its HLE: Biomedicine accuracy rose from 14% to 26% with increased experience.

    Source: Introduction · p. 4, Conclusion · p. 7

  • Takeaways & Limitations

    The authors present STELLA’s capacity to learn and grow as a shift from AI as a static tool toward a dynamic scientific partner.

    Source: Conclusion · p. 7

    The gap between benchmark performance and real-world laboratory application remains to be bridged.

    Source: Conclusion · p. 7

Abstract

from arXiv · show

The rapid growth of biomedical data, tools, and literature has created a fragmented research landscape that outpaces human expertise. While AI agents offer a solution, they typically rely on static, manually curated toolsets, limiting their ability to adapt and scale. Here, we introduce STELLA, a self-evolving AI agent designed to overcome these limitations. STELLA employs a multi-agent architecture that autonomously improves its own capabilities through two core mechanisms: an evolving Template Library for reasoning strategies and a dynamic Tool Ocean that expands as a Tool Creation Agent automatically discovers and integrates new bioinformatics tools. This allows STELLA to learn from experience. We demonstrate that STELLA achieves state-of-the-art accuracy on a suite of biomedical benchmarks, scoring approximately 26\% on Humanity's Last Exam: Biomedicine, 54\% on LAB-Bench: DBQA, and 63\% on LAB-Bench: LitQA, outperforming leading models by up to 6 percentage points. More importantly, we show that its performance systematically improves with experience; for instance, its accuracy on the Humanity's Last Exam benchmark almost doubles with increased trials. STELLA represents a significant advance towards AI Agent systems that can learn and grow, dynamically scaling their expertise to accelerate the pace of biomedical discovery.

Introduction

Biomedical research resources are fragmented and constantly evolving, while AI agents typically rely on static, manually curated toolsets. STELLA combines a multi-agent workflow with mechanisms for evolving its reasoning strategies and tools, and the paper reports strong benchmark performance that improves with experience.

Introduction

  • STELLA uses a Manager, Dev, Critic, and Tool Creation Agent to plan and execute research tasks, assess intermediate results, and create an iterative problem-solving workflow.The Dev Agent generates and runs Python code for bioinformatics analyses, while the Critic provides feedback on intermediate results.Source: Introduction · pp. 2–3
  • Its Template Library accumulates successful reasoning workflows, while the Tool Creation Agent identifies, tests, and integrates tools beyond a predefined set.The Tool Ocean contains accessible bioinformatics tools, databases, and APIs.Source: Introduction · pp. 2–3
  • The paper reports that STELLA’s accuracy systematically improves with use, presenting this as evidence of its self-evolving design.The reported results average accuracy across three independent evaluation runs.Source: Introduction · p. 4
  • STELLA achieved state-of-the-art biomedical benchmark results, including 26% on Humanity’s Last Exam: Biomedicine, 54% on LAB-Bench: DBQA, and 63% on LAB-Bench: LitQA, outperforming next-best models by up to 8 percentage points.On HLE: Biomedicine, accuracy rose from 14% to 26% with increased computational experience.Source: Introduction · p. 4, Figure 2 · p. 4

Results

STELLA combines a multi-agent research workflow with evolving reasoning templates and an expanding set of executable tools. The reported benchmark results show strong performance against tested models and agents, with accuracy improving as computational trials increase.

STELLA’s Overall Framework

  • The Manager assigns analysis tasks to the Dev Agent, and the Critic’s feedback can prompt the Tool Creation Agent to build and validate a tool for a capability gap.In the chemoresistance example, the new virtual perturbation screening tool helped identify MTF1 as a keystone regulator.Source: STELLA’s Overall Framework · p. 5

STELLA’s Self-evolving Mechanisms for Biomedical Research

  • STELLA preserves successful workflows as reasoning templates and expands the Tool Ocean with database-querying functions, foundation-model interfaces, and customized analysis tools.The paper says the combined mechanisms support increasingly complex biomedical challenges with growing autonomy and scientific sophistication.Source: STELLA’s Self-evolving Mechanisms for… · pp. 5–6

STELLA Outperforms State-of-the-art LLMs and Agents

  • 26% on Humanity’s Last Exam (Biomedicine), approximately 54% on LAB-Bench DBQA, and 63% on LAB-Bench LitQA were the highest scores reported across the tested models and agents.STELLA was benchmarked against state-of-the-art large language models and specialized agents.Source: STELLA Outperforms State-of-the-art… · p. 6
  • LitQA accuracy rose from approximately 52% at a 1x trial budget to 63% at a 9x budget, with improvement reported across all three benchmarks.The authors describe a positive correlation between computational budget and performance.Source: STELLA Outperforms State-of-the-art… · pp. 6–7

Conclusion

STELLA combines a multi-agent architecture with an adaptive Template Library and dynamic Tool Ocean to expand its reasoning and capabilities through experience. It achieves state-of-the-art performance across challenging biomedical benchmarks and improves systematically with experience, though real-world laboratory application remains a challenge.

Conclusion

  • STELLA integrates a multi-agent architecture with an adaptive Template Library for reasoning and a dynamic Tool Ocean for capabilities.The authors state that these mechanisms allow it to learn from experience and expand its knowledge and skills.Source: Conclusion · p. 7
  • STELLA achieved state-of-the-art performance across multiple challenging biomedical benchmarks and improved systematically as it gained experience.The authors identify this pattern as evidence that the self-evolving design is effective.Source: Conclusion · p. 7
  • Bridging benchmark performance to real-world laboratory application remains a challenge, and future work will focus on research-workflow deployment and collaboration with human scientists.Source: Conclusion · p. 7

Methods

The evaluation compared STELLA with leading language models and a biomedical research agent on LAB-Bench and Humanity’s Last Exam. It used sampled benchmark questions and followed the official LAB-Bench protocol.

Baselines

  • The baselines included Gemini 2.5 Pro (29), Claude 4 Opus (30), DeepSeek-R1 (31), OpenAI o3 (32), and the biomedical agent Biomni (14).Source: Baselines · p. 8
  • Claude 4 Sonnet powered STELLA’s Dev and Tool Creation Agents, while Gemini 2.5 Pro powered its Manager and Critic Agents.Source: Baselines · p. 8

Q&A Benchmarks

  • LAB-Bench DBQA and LitQA used the same 12.5% sampled subset, no development sets, and the official multiple-choice protocol with abstention allowed.The sampling was described as a cost-effective yet representative assessment.Source: Q&A Benchmarks · pp. 8–9
  • Humanity’s Last Exam evaluation used 50 representative questions spanning fourteen Biology and Medicine subdisciplines.The questions were selected following the sampling protocol from the Biomni study and evaluated on the test set.Source: Q&A Benchmarks · p. 9

Related papers

Loading 2507.02004v1…