PaperlayerView exact revision on arXiv

Source-linked AI summary

ToolUniverse: An open platform for democratizing AI scientists

Shanghua Gao, Richard Zhu, Pengwei Sui, Zhenglun Kong, Sufian Aldogom, Yepeng Huang, Ayush Noori, Reza Shamji, Krishna Parvataneni, Theodoros Tsiligkaridis, Marinka Zitnik

arXiv:2509.23426v3cs.AIcs.LG

TL;DR

ToolUniverse provides an extensible platform for integrating diverse scientific tools and research capabilities into AI-scientist workflows. Case studies demonstrate end-to-end analyses across target assessment, chemical strategy, and clinical-trial safety, while conclusions remain bounded by underlying tools and data.

  • Problem

    AI-scientist workflows require integration across diverse scientific models, datasets, APIs, retrieval utilities, and analysis systems.

  • Method

    ToolUniverse standardizes and composes callable scientific tools, supporting tool addition, optimization, creation, language-model integration, retrieval, and agentic workflows.

  • Results

    ToolUniverse supported end-to-end analyses spanning target assessment, chemical strategy, and clinical-trial safety, with tools autonomously selected and chained from natural-language questions.

  • Takeaways & Limitations

    The case studies show that one workflow can assemble evidence across genetics, functional genomics, structures, chemistry, machine learning, and statistics.

  • Takeaways & Limitations

    Conclusions remain bounded by underlying databases and models, and machine-learning outputs and association evidence require provenance assessment, validation, and experimental confirmation.

Abstract

from arXiv · show

AI scientists are emerging computational systems that serve as collaborative partners in discovery. These systems remain difficult to build because they are bespoke, tied to rigid workflows, and lack shared environments that unify tools, data, and analyses into a common ecosystem. We present ToolUniverse, an open platform for building AI scientists from any language or reasoning model across open- and closed-weight models. ToolUniverse standardizes how AI scientists identify and call tools through an AI-tool interaction standard, in which every tool declares its purpose in natural language, a typed schema for its inputs and outputs, and a backend-agnostic invocation format, and applies that standard to more than 2,700 scientific tools and over 130 research skills spanning machine learning models, datasets, APIs, and scientific packages for data analysis, knowledge retrieval, and experimental design. It automatically refines tool interfaces for correct use by AI scientists, generates new tools from natural language descriptions, iteratively optimizes tool specifications, and composes tools into agentic workflows. In case studies, ToolUniverse was used to create AI scientists that carried out end-to-end analyses in target assessment, chemical strategy, and clinical-trial safety. The open-source ToolUniverse is available at https://aiscientist.tools.

Supplementary Information for

TOOLUNIVERSE is an open ecosystem that exposes standardized scientific tools to language models, reasoning models, agents, and human users. Its shared interaction standard and modular architecture support adding, finding, calling, composing, generating, and refining tools.

  • Overview: TOOLUNIVERSE exposes over 2,700 scientific tools to open and closed AI models through a backend-agnostic interaction standard.The standard uses natural-language descriptions, typed input and output schemas, and a uniform invocation format.
  • AI-tool Interaction Standard: Each tool declares its purpose, typed parameters and outputs, constraints, and a uniform invocation format so clients can use different backends consistently.Tool descriptions are supplied to clients such as LLMs, reasoning models, agents, and human users.
  • Core Components: TOOLUNIVERSE organizes tool management into six modules: Manager, Finder, Caller, Composer, Discoverer, and Optimizer.These modules register tools, retrieve candidates, execute validated calls, assemble workflows, generate tools, and refine descriptions.
  • Tool Access: Interactions use a standardized tool-call schema that lets clients invoke local or remote tools without backend-specific configuration.Tool Caller executes the requested tool and returns results through local or remote interfaces.

S3 Supplementary Note: Core Modules of TOOLUNIVERSE

TOOLUNIVERSE provides modular infrastructure for registering, discovering, executing, composing, and remotely exposing scientific tools. Tool Finder combines keyword, LLM-based, and embedding search to match natural-language requests with available tools.

  • Registration: Tool Manager registers local functions and remote services so connected models can invoke them through a shared interface.Remote tools can be loaded from MCP servers and accessed like standard tools.
  • Tool Finder: Tool Finder searches more than 2,700 candidates using TF-IDF keyword search, LLM in-context reasoning, or embedding-based semantic matching.The strategies trade off speed, semantic understanding, and computational cost.
  • Tool Finder: Embedding search retrieves tools by comparing query and tool-description vectors with cosine similarity.Tool descriptions are indexed offline, while user queries are embedded online for nearest-match retrieval.
  • Execution and Composition: Tool Caller executes validated requests, while Tool Composer chains heterogeneous tools into multi-step workflows.The caller dispatches validated arguments to tool implementations and returns structured results or errors.

S3.5 Tool Discoverer

Tool Discoverer converts natural-language functional requests into runnable tools through staged discovery, specification, implementation, evaluation, and iterative refinement. The workflow reduces manual construction while retaining human review requirements.

  • Generation Process: Tool Discoverer transforms a plain-text functional request into a runnable tool by integrating discovery, definition generation, code implementation, and quality refinement.The workflow is designed to require limited human intervention but does not bypass human review.
  • Core Principles and Architecture: Its core principles are pattern-guided generation, structured definition synthesis, automated code generation and validation, and iterative refinement.These principles promote consistency with existing ecosystem tools and executable implementations.
  • Quality Evaluation: QualityEvaluator scores functionality, reliability, maintainability, performance, and test coverage on 0–10 scales and computes a weighted overall score.The refinement loop ends when target quality scores are reached or iteration limits are met.
  • Generation Process: The system searches GitHub, PyPI, and existing TOOLUNIVERSE tools for reference implementations before generating a complete tool configuration.Search combines semantic similarity and keyword-based strategies.
  • AI Scientist Integration: The generated workflow can be used by AI scientists that find tools, execute validated calls, gather information, test hypotheses, and request human feedback.TOOLUNIVERSE connects the generated tool to the broader tool-use and reasoning loop.

S4.2 Building an AI Scientist from an AI Agent

TOOLUNIVERSE supports AI agents by combining a broad scientific toolkit with agentic tools for analysis, retrieval, prediction, and human feedback. Agents can use these capabilities across domains and access remote or specialized computational resources.

  • Scientific Toolkit: TOOLUNIVERSE contains over 2,700 scientific tools spanning machine learning, databases, APIs, embedding stores, robotics, and human-feedback systems.The categories address diverse computational and analytical requirements.
  • Agentic Tools: Agentic tools autonomously perform tasks such as literature summarization, code analysis, hypothesis generation, experiment planning, and results analysis.Each agentic tool can be configured with prompts, tool descriptions, and different backend models.
  • Data and API Tools: Database and API tools provide structured search, filtering, validation, and access to resources including clinical trials, molecular databases, FDA data, and disease–target associations.These integrations support tabular, hierarchical, XML-based, graph-structured, REST, and GraphQL resources.
  • Human Feedback: Expert-feedback tools let AI scientists request suggestions or approval from human experts through a connected server and interface.The Tool Caller forwards requests to the expert-facing system.
  • Machine Learning Tools: Machine-learning tools support predictions including disease–target scoring, disease-state prediction, gene interactions, ADMET, and binding affinity.Remote registration helps integrate models requiring specialized environments or hardware.

S6 Supplementary Note: Pre-built Research Skills

TOOLUNIVERSE provides more than 130 pre-built research skills as natural-language workflows that orchestrate multiple tools for end-to-end scientific tasks and return reproducible results with provenance.

  • More than 130 research skills orchestrate multiple tools into end-to-end scientific workflows with reproducible, provenance-traceable results.

S6.1 Worked examples

Worked examples show TOOLUNIVERSE routing natural-language requests to validated tools for both simple retrieval and multi-step RNA-seq analysis. The RNA-seq workflow produced differential-expression and pathway-enrichment results, including glucocorticoid-response terms at BH-adjusted p = 4.7 × 10−3.

  • Sequence retrieval: The sequence-retrieval example resolved TP53 to Ensembl protein ID ENSP00000269305 and returned its 393-amino-acid canonical sequence.
  • Sequence retrieval: The simplest skill shape maps one natural-language request to one validated tool call.
  • RNA-seq differential expression: BH-adjusted p = 4.7 × 10−3 for cellular response to glucocorticoid stimulus among 58 enriched biological-process terms.
  • RNA-seq differential expression: A single natural-language request triggered an end-to-end RNA-seq workflow whose results remained traceable to the producing tools.
  • RNA-seq differential expression: 4,028 genes were differentially expressed, including 2,211 upregulated and 1,817 downregulated genes, in the airway RNA-seq analysis.

S7 Supplementary Note: Evaluation of Tools

TOOLUNIVERSE evaluates tools through automated validation, provenance-dependent human review, admission decisions, and ongoing maintenance. These measures target correctness, reproducibility, and scientific reliability while leveraging validation from established resources.

  • Each tool undergoes automated checks, human expert review, and an approval-or-revision decision before admission to TOOLUNIVERSE.
  • Review verifies source-record fidelity for authoritative data integrations and reproduction of published behavior for literature-backed tools.
  • Regular maintenance, monitoring, and structured bug reporting support continued tool reliability after admission.
  • The evaluation process aims to ensure correctness, reproducibility, and scientific reliability while leveraging prior validation from established resources.

Supplementary Note: Quantitative Evaluation of TOOLUNIVERSE on End-to-End Tasks

The end-to-end evaluation compares matched agents with and without TOOLUNIVERSE on identical LAB-Bench DbQA and SeqQA items, alongside Biomni. TOOLUNIVERSE improves accuracy across both agent settings and subtasks, and the augmented agents outperform Biomni on every subtask.

  • The comparison used identical matched agents differing only in TOOLUNIVERSE availability across two base-model settings and the same released benchmark items.
  • Accuracy rose from 56.7 to 78.3 for Claude Code and from 81.1 to 92.8 for Codex on DbQA after adding TOOLUNIVERSE.These correspond to improvements of 21.6 and 11.7 percentage points, respectively.
  • Accuracy rose from 94.3 to 99.5 for Claude Code and from 91.4 to 96.2 for Codex on SeqQA after adding TOOLUNIVERSE.These correspond to improvements of 5.2 and 4.8 percentage points, respectively.
  • Codex improvements were stable across runs, with standard deviations at most 3.5 points and 1.0 point after adding TOOLUNIVERSE on DbQA.
  • TOOLUNIVERSE-augmented Claude Code and Codex agents outperformed Biomni on both DbQA and SeqQA using Biomni’s released items and scoring.Biomni scored 74.4 on DbQA and 81.9 on SeqQA, while augmented Claude Code scored 78.3 and 99.5 and Codex scored 92.8 ± 1.0 and 96.2 ± 1.6.

S9 Supplementary Note: Step-by-step real-world case studies

The case studies show AI scientists answering real research questions by autonomously selecting and chaining tools across databases, predictive models, and analysis services.

  • Three end-to-end case studies demonstrate AI scientists autonomously composing tools to answer real research questions and report concrete results at each step.The workflows integrate databases, predictive models, and analysis services rather than stopping at database lookup.

S9.1 Case 1: Is BLM a viable anticancer target, and is its chemical probe developable?

The BLM case separates genetic safety concerns from context-specific dependency signals and chemical liabilities. It treats BLM as a testable context-selective hypothesis, while ML216 remains an early probe rather than a developable lead.

  • BLM loss is linked to Bloom syndrome and broad cancer predisposition, making inhibition a genotoxicity concern rather than genetic validation of the target.A pathogenic BLM variant is independently classified as Pathogenic, reinforcing the risk interpretation.
  • BLM is not pan-essential: only 7.5% of cancer cell lines are dependent, with vulnerability concentrated in mature T/NK neoplasms, cutaneous squamous-cell carcinoma, and neuroblastoma-related settings.The mean gene effect is −0.18 across 1,258 lines, while dependent lines have scores below −0.5.
  • Experimental helicase structures should anchor structure-based design because the AlphaFold model has mean pLDDT 60.5 and unreliable interdomain linkers.The case identifies high-resolution experimental structures of the helicase core for design use.
  • The literature supports testing BLM in defined contexts by analogy to validated WRN synthetic lethality in MSI/MMR-deficient cancers, not treating BLM as an established target.Candidate contexts include mature T/NK neoplasms, cutaneous SCC, and ALT-positive tumors.
  • ML216 is drug-like but low quality, with QED 0.66, no Lipinski violations, and predicted DILI of 0.99 that persists across five close analogs.The persistent DILI signal is consistent with a scaffold-associated model hypothesis requiring experimental hepatotoxicity testing.

S9.2 Case 2: Assessing OXTR druggability and the chemical strategy for a CNS indication

The OXTR assessment finds a structurally and chemically tractable GPCR, but its autism association remains an emerging hypothesis with inconsistent clinical support. The resulting strategy prioritizes a brain-penetrant OXTR agonist or positive allosteric modulator selective over vasopressin receptors, with confirmed central exposure.

  • Outcome: The autism association is an emerging hypothesis rather than a clinically validated indication, despite 393 target-disease associations overall.The evidence base includes mixed clinical support and no approved oxytocin product for autism’s core social-communication symptoms.
  • Confirm tractability and enumerate drugs: Approved agents establish ligand precedent for reproductive and uterine indications, but that precedent does not de-risk autism drug development.Nine associated agents include approved peptides and non-peptide compounds reaching advanced clinical stages.
  • Resolve mechanism: OXTR pharmacology supports opposing mechanisms: oxytocin is an agonist, whereas atosiban is an antagonist used as a tocolytic.This makes the desired mechanism indication-dependent rather than reducible to a generic agonist-versus-antagonist rule.
  • Outcome: OXTR is tractable and well-liganded, with experimental structures in active and inactive states and measured affinity as strong as Ki 3.2 nM.It is classified as a druggable-family GPCR with high-quality ligands and a ligand-bound structure.
  • Outcome: The proposed CNS strategy is a brain-penetrant agonist, partial agonist, or positive allosteric modulator selective over vasopressin receptors, with confirmed central exposure.Nolasiban has predicted BBB penetrance of 0.93 but is an antagonist, so the prediction establishes plausibility rather than a validated therapeutic match.

S9.3 Case 3: Clinical-trial safety biostatistics for BCG vaccination and adverse-event severity

The clinical-trial safety workflow analyzes raw BCG-CORONA trial tables by constructing an adverse-event cohort, testing an unadjusted association, and fitting an adjusted ordinal model. Both analyses associate BCG assignment with higher maximum adverse-event severity, subject to the cohort and proportional-odds assumptions.

  • Outcome: TOOLUNIVERSE performs cohort construction, endpoint derivation, and modeling directly on raw trial tables while making definitions explicit.The workflow merges demographic and adverse-event tables and applies the tool’s analysis modes.
  • Construct the analysis cohort: The analysis cohort contains 791 subjects with evaluable adverse-event records, each represented by one maximum-severity value.The severity distribution is grade 1: 328, grade 2: 402, grade 3: 43, and grade 4: 18.
  • Unadjusted association: The unadjusted treatment-by-severity association is significant at χ2 = 10.12, dof = 3, p = 0.018.The unadjusted and adjusted analyses share the outcome and overlapping data, so they are not independent.
  • Adjusted model: Adjusted modeling estimates higher adverse-event severity with BCG vaccination at OR = 1.53 (95% CI 1.16–2.01, p = 0.0024).The multivariable ordinal logistic model includes patient-interaction covariates, which are not significant.
  • Outcome: The endpoint is restricted to participants with adverse-event records and is a derived per-subject maximum, limiting how the association should be interpreted.The proportional-odds estimate also assumes a common effect across severity thresholds, which should be checked before over-interpretation.

S9.4 Strengths and limitations illustrated by the case studies

The case studies illustrate ToolUniverse’s ability to connect heterogeneous evidence, reconcile conflicts, calibrate conclusions, and preserve provenance across end-to-end analyses. Its main limitation is that it orchestrates existing tools rather than certifying their outputs or the scientific correctness of conclusions.

  • End-to-end analyses autonomously select and chain tools from natural-language questions, producing concrete results without user-written code.
  • Strengths: Workflows combine genetics, functional genomics, protein structures, cheminformatics, bioactivity, machine-learning predictions, and statistics in a single run.
  • Strengths: Conflicting safety, opportunity, and chemical-liability signals are kept separate and reconciled into a testable, biomarker-defined hypothesis.
  • Strengths: Calibrated outputs distinguish hypotheses from targets, plausibility from exposure, and predicted patterns from experimentally confirmed findings.
  • Strengths: Typed calls, versioned releases, provenance, and optional audit logging make workflows inspectable and re-runnable, while cross-checking can expose mechanism-level mismatches before recommendations.
  • Limitations: Conclusions remain bounded by underlying databases and models, so quantitative outputs and associations require provenance-aware domain judgment and experimental confirmation.
Loading 2509.23426v3…