Source-linked AI summary
ToolRosella: Translating Code Repositories into Standardized Tools for Scientific Agents
Shimin Di, Xujie Yuan, Hanghui Guo, Chaoqian Ouyang, Yongxu Liu, Ling Yue, Zhangze Chen, Libin Zheng, Jia Zhu, Shaowu Pan, Jian Yin, Yong Rui, Min-Ling Zhang
TL;DR
Scientific agents remain limited by narrow curated tool inventories, while useful open-source computational repositories are difficult to standardize and invoke reliably. ToolRosella automatically converts these repositories into standardized agent-invocable tools through analysis, interface construction, testing, and iterative repair. Across 122 repositories, it reached 61.5% conversion success after repair, produced 1,580 callable tools, and supported 84.0% downstream task success.
Problem
Scientific agents have limited capability because curated tool inventories are narrow, while open-source repositories remain difficult to operationalize reliably.
Method
ToolRosella analyzes repositories, constructs standardized interfaces, tests execution, and iteratively repairs failures in an MCP-compatible pipeline.
Results
61.5% repository conversion success after iterative repair and 84.0% average downstream task success were achieved across the evaluated tool pool.
Takeaways & Limitations
The standardized tools form a reusable capability layer that broadens downstream agents’ operational tool space and integrates with other agent frameworks.
Takeaways & Limitations
Conversion remains constrained by environment reconstruction and repositories built around interactive or notebook-based workflows lacking programmatic interfaces.
Abstract
from arXiv · showhide
Large Language Model (LLM)-based agent systems are increasingly used for scientific tasks, yet their practical capability remains constrained by the narrow scope of manually curated tools they can invoke. Much scientific computational functionality already exists in open-source code repositories, but these resources remain difficult to standardize, operationalize, and invoke reliably for agent use. Here we present ToolRosella, a framework that automatically transforms heterogeneous scientific code repositories into standardized, agent-invocable tools. ToolRosella combines repository analysis, tool interface construction, execution testing, and iterative repair to address the problem of repository-to-tool standardization. Across 122 GitHub repositories spanning 35 subdisciplines in six domains, ToolRosella reaches a 61.5\% repository conversion success rate after iterative repair, with a 4.4 speedup over human engineers. The resulting 1,580 callable tools support a downstream task success rate of 84.0\% and improve performance when integrated into other agent frameworks, particularly on tasks whose required tools are absent from fixed, curated inventories.
1 Introduction
Scientific agents need computational capabilities beyond fixed curated inventories, but open-source repositories are difficult to invoke reliably because their interfaces and environments are heterogeneous. ToolRosella addresses this gap by automatically standardizing repositories into reusable agent-invocable tools.
- Motivation: Scientific agent systems increasingly target complex tasks that reduce to concrete computational operations.These operations include data processing, structure analysis, simulation, and optimization.
- 1 Introduction: Curated tool inventories improve reliability but constrain agents to a fixed, precompiled capability space.Broader repository ecosystems offer more coverage but introduce heterogeneous interfaces, environments, and calling conventions that reduce invocation reliability.
- 1 Introduction: Open-source repositories already contain useful scientific functionality, yet inconsistent interfaces, fragile dependencies, incomplete specifications, and variable implementation styles make them difficult for agents to use.The challenge is transforming existing code into directly usable, standardized tools rather than merely locating relevant repositories.
- Approach: ToolRosella automatically transforms heterogeneous scientific code repositories into standardized, agent-invocable tools through repository analysis, interface construction, execution testing, and iterative repair.The framework is implemented as an MCP-compatible, hierarchical multi-agent pipeline.
- Results: 61.5% repository conversion success was achieved after iterative repair across 122 repositories spanning 35 subdisciplines and six domains.The framework standardized 1,580 callable tools while substantially reducing average standardization time relative to human engineers.
- Results: The resulting tools achieved 84.0% average downstream task success and improved other agent frameworks, especially when required tools were absent from curated inventories.This establishes a reusable capability layer rather than only a one-time conversion outcome.
2 Results
ToolRosella combines repository search, MCP service construction, validation, and iterative repair to convert heterogeneous scientific repositories into callable tools. Across broad conversion and downstream evaluations, it improves reliability, supports strong task performance, and extends existing agent frameworks.
- Automated tool standardization: ToolRosella uses a hierarchical multi-agent pipeline combining repository analysis, environment setup, service generation, and iterative validation.A Tool-search Agent retrieves and evaluates repositories, while an MCP-construction Agent builds standardized services.
- Automated tool standardization: 61.5% of repositories were successfully converted after three repair rounds, versus 33.6% on the first pass and 90.2% for human engineers.The benchmark covered 122 GitHub repositories across 35 subdisciplines and six domains.
- Automated tool standardization: 7.2 min was ToolRosella’s average conversion time per repository, compared with 31.6 min for Codex-assisted human engineers, a 4.4× speedup.The reported reduction in standardization time was 77.2%.
- Automated tool standardization: 27.9 percentage points of conversion success were recovered through three Review-Revise-Fix rounds, with 34 of 81 first-pass failures repaired.Most gains occurred in the first round, while dependency and environment issues accounted for 75.3% of first-pass failures.
- Downstream task performance: 84.0% was ToolRosella’s average verified task success rate, the highest among evaluated systems across all six scientific domains.On subdisciplines outside curated baseline inventories, ToolRosella achieved 82.5% verified success compared with 0.0% for SciToolAgent and ChemCrow.
- Downstream task performance: ToolRosella-converted tools increased average success rates for RepoMaster, OpenAgents, and OpenClaw to 81.5%, 82.0%, and 77.9%, respectively.Improvements occurred across all six domains, especially where original inventories were weak or lacked the required tools.
3 Discussion
ToolRosella makes repository-to-tool standardization a practical capability layer for scientific agents, while exposing limitations in environment reconstruction, heterogeneous workflows, safety governance, and language coverage.
- Contributions and limitations: 61.5% repository conversion success after iterative repair supports repository-to-tool standardization across 122 repositories spanning 35 subdisciplines and six domains.The downstream tool pool also achieved strong verified task performance across six domains and could be reused by different agent frameworks.
- Contributions and limitations: Environment reconstruction remains the dominant conversion barrier because broken dependencies, platform-specific configurations, and implicit system-state assumptions often resist code-level repair.Interactive repositories such as GUI applications and notebook workflows also lack programmatic interfaces needed for standardized wrapping.
- Safety Governance Considerations: Repository-derived tools shift the trust boundary from pre-vetted inventories to externally sourced code that may trigger unintended file access, unsafe commands, or unbounded resource consumption.Functional executability alone does not guarantee operation within intended boundaries.
- Safety Governance Considerations: Wrapper-level safeguards include path whitelisting, restricted working directories, and container isolation for confidentiality.These controls govern how each repository-derived tool is implemented internally.
- Safety Governance Considerations: Action-level controls complement wrapper safeguards by constraining whether and how tools are invoked within particular agent actions.They function as a safety brake when tools with different consequences share similar invocation interfaces.
- Extension to other programming languages: ToolRosella currently targets Python repositories, while extending the standardization layer to R, C/C++, Java, and other languages requires language-specific adapters.Repository retrieval, capability analysis, interface generation, validation, and iterative repair remain language-agnostic once callable units are identified.
4 Methods
ToolRosella retrieves task-relevant repositories, standardizes them into callable tools, and orchestrates those tools for user-defined scientific tasks through three coordinated agents.
- 4 Methods: The Tool-search Agent retrieves task-relevant code repositories.It supplies capabilities beyond a predefined tool pool.
- 4 Methods: The MCP-construction Agent standardizes retrieved repositories into callable tools.This component handles repository code understanding, environment construction, and interface validation.
- 4 Methods: The Planning Agent orchestrates standardized tools for user-defined scientific tasks.The three components coordinate retrieval, construction, and task execution.
4.1 Tool-search Agent
The Tool-search Agent expands beyond predefined inventories by retrieving repositories from open-source ecosystems, evaluating their suitability, and reformulating searches when initial candidates are inadequate.
- 4.1 Tool-search Agent: The Tool-search Agent retrieves task-relevant repositories from open-source ecosystems beyond a predefined tool pool.It uses repository retrieval to access capabilities not already fixed in the agent's inventory.
- 4.1 Tool-search Agent: User queries are converted into up to m topic descriptors that summarize functional requirements before GitHub API search.The LLM first infers task intent and extracts descriptors W = {W1, . . . , Wm}.
- 4.1 Tool-search Agent: Candidate repositories are ranked primarily by popularity signals, using star count as a practical proxy for community adoption.The LLM evaluates whether the initial results are sufficiently relevant.
- 4.1 Tool-search Agent: Search reformulation removes overly specific scenario descriptions or constraints to improve retrieval recall without altering task intent.The reformulated query is used when the initial results lack sufficiently relevant candidates.
- 4.1 Tool-search Agent: Candidates are assessed for structural completeness and functional relevance before the highest-ranked repository is selected.Structural completeness covers executable assets such as source code, dependencies, and setup instructions.
- 4.1 Tool-search Agent: Unless otherwise specified, retrieval uses K = 50 repositories and m = 5 descriptors, while direct repository links bypass retrieval.Directly specified repositories proceed directly to MCP construction.
4.2 MCP-construction Agent
The MCP-construction Agent converts heterogeneous GitHub repositories into MCP-compatible services through a staged pipeline designed for reliable language-model invocation.
- 4.2 MCP-construction Agent: The MCP-construction Agent transforms retrieved GitHub repositories into standardized MCP-compatible services.The design addresses differences in interface design, dependency structure, and execution logic.
- 4.2 MCP-construction Agent: The construction pipeline contains eight functional nodes: Download, Analysis, Env, Generate, Code check, Run, Review, and Finish.These stages organize repository parsing, environment setup, service generation, testing, and review.
- 4.2 MCP-construction Agent: The selected repository is cloned into a local workspace for subsequent parsing, testing, and service construction.Local cloning precedes the remaining pipeline operations.
Analysis Node.
ToolRosella analyzes repository source code and documentation to separate reusable functionality from project-specific execution logic and infer callable interfaces. It then prepares a reproducible runtime environment by resolving and installing the repository’s dependencies.
- Analysis Node.: Repository analysis decouples reusable computational components from command-line wrappers, visualization layers, and project-specific execution logic.
- Analysis Node.: The Analysis Node uses repository source code and accompanying documentation to infer standardized callable interfaces.
- Environment preparation: The Env Node initializes a Python environment, defaulting to Python 3.10 when no version is specified.
- Environment preparation: Dependency specification files such as requirements.txt or Dockerfile are parsed to identify required packages and environment settings.
- Environment preparation: Resolved dependencies are installed automatically to yield a reproducible environment for downstream validation and deployment.
Generate Node.
The Generate Node synthesizes an MCP-compatible service layer that exposes repository functionality through standardized tool interfaces. It also supports reliability by encapsulating repository functions in adapters and checking generated interfaces against publicly accessible repository symbols.
- Generate Node.: The Generate Node uses an LLM to synthesize an MCP-compatible service layer exposing repository functionality through standardized tool interfaces.
- Generate Node.: Service synthesis produces core service code, interface wrappers, and validation scripts required for deployment.
- Generate Node.: Repository functions are encapsulated in an adapter layer and converted to asynchronous execution for concurrent, non-blocking tool invocation.
- Code check Node.: The Code check Node scans source modules to build an available-symbol dictionary for verifying consistency between exposed interfaces and repository code.
Run Node.
The Run Node executes the generated service in the prepared environment and validates initialization and core tool behavior. When tests fail, the Review Node uses execution evidence and repository structure to diagnose failures and plan repairs.
- Run Node.: The Run Node executes the generated MCP service within the prepared runtime environment using an automated validation suite.
- Run Node.: Validation checks whether the service initializes correctly and whether core tool interfaces behave as expected during execution.
- Run Node.: If all tests pass, the build is marked successful and proceeds to the Finish Node.
- Review Node.: When validation fails, execution traces and error logs are forwarded to the Review Node for diagnosis and repair planning.
- Review Node.: The Review Node considers execution errors, failing tests, wrapper code, and repository structure when identifying failure sources and required modifications.
Finish Node.
After all tests pass, ToolRosella organizes the service artifacts and supporting materials into a standardized directory structure. It consolidates these files into a deployable MCP package conforming to community standards.
- Finish Node.: The Finish Node organizes final service files, the dependency list, usage instructions, and example calls into a standardized directory structure.
- Finish Node.: The resulting service is consolidated into a deployable MCP package.
- Finish Node.: The packaged service conforms to community standards.
4.3 Planning Agent
The Planning Agent translates a user request into an ordered tool-invocation strategy and iteratively revises its actions using intermediate results or refined requests.
- The agent iteratively interprets tasks, selects tools, and updates subsequent actions from intermediate results.This follows a reasoning-and-action paradigm related to ReAct.
- It maps available tools to subtasks and execution order using task objectives, constraints, and tool descriptions.
- For a refined request Q′, the planner re-evaluates objectives and dynamically updates the workflow.