PaperlayerView exact revision on arXiv

Source-linked AI summary

Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agents

Jiacheng Miao, Joe R. Davis, Yaohui Zhang, Jonathan K. Pritchard, James Zou

arXiv:2509.06917v2cs.AIcs.CLcs.LG

TL;DR

Researchers often face substantial technical barriers when adapting papers’ computational methods to new data. Paper2Agent automatically converts papers and codebases into tested MCP-based AI agents that users can query in natural language. In a novel-query evaluation, its AlphaGenome agent achieved 100.0% (15/15) accuracy against the original code.

  • Problem

    Applying computational methods from research papers often requires locating code, configuring environments, and interpreting inputs and outputs.

  • Method

    Paper2Agent analyzes a paper and its codebase, encapsulates contributions in an MCP server, and iteratively tests the resulting natural-language agent.

  • Results

    The AlphaGenome agent achieved 100.0% (15/15) accuracy on novel queries, verified by manually executing the original AlphaGenome code.

  • Takeaways & Limitations

    Paper2Agent turns static research outputs into interactive agents that can demonstrate, apply, and adapt methods to new projects.

  • Takeaways & Limitations

    Paper2Agent cannot reliably expose incomplete, poorly documented, or error-filled codebases as functioning tools.

Abstract

from arXiv · show

We introduce Paper2Agent, an automated framework that converts research papers into AI agents. Paper2Agent transforms research output from passive artifacts into active systems that can accelerate downstream use, adoption, and discovery. Conventional research papers require readers to invest substantial effort to understand and adapt a paper's code, data, and methods to their own work, creating barriers to dissemination and reuse. Paper2Agent addresses this challenge by automatically converting a paper into an AI agent that acts as a knowledgeable research assistant. It systematically analyzes the paper and the associated codebase using multiple agents to construct a Model Context Protocol (MCP) server, then iteratively generates and runs tests to refine and robustify the resulting MCP. These paper MCPs can then be flexibly connected to a chat agent (e.g. Claude Code) to carry out complex scientific queries through natural language while invoking tools and workflows from the original paper. We demonstrate Paper2Agent's effectiveness in creating reliable and capable paper agents through in-depth case studies. Paper2Agent created an agent that leverages AlphaGenome to interpret genomic variants and agents based on ScanPy and TISSUE to carry out single-cell and spatial transcriptomics analyses. We validate that these paper agents can reproduce the original paper's results and can correctly carry out novel user queries. Paper2Agent automatically created AI co-scientist that identified new splicing variant associated with ADHD risk. By turning static papers into dynamic, interactive AI agents, Paper2Agent introduces a new paradigm for knowledge dissemination and a foundation for the collaborative ecosystem of AI co-scientists.

1 Introduction

Paper2Agent addresses the technical barriers and passivity of conventional research papers by transforming them into interactive AI agents that can demonstrate, apply, and adapt their methods through natural language. It represents papers as MCP servers, extending executable-paper efforts toward interactive scientific collaboration.

  • Motivation: Technical barriers—including finding code, installing dependencies, configuring environments, and interpreting inputs and outputs—often prevent readers from using computational methods on new data [1].Even well-maintained repositories can require substantial technical expertise, as illustrated by AlphaGenome’s setup requirements.
  • Contribution: Paper2Agent turns static papers into interactive AI agents that act as paper-specific experts capable of demonstrating, applying, and adapting methods to new projects.This shifts research outputs from passive documents or codebases into knowledgeable entities capable of execution and dialogue.
  • Related work: Paper2Agent builds on AI-agent systems that use LLMs with external tools or APIs to reason, act across multiple steps, and adapt based on feedback [12] [21].Related systems have organized AI scientists for collaborative research and supported hypothesis generation or research-lifecycle automation [16].
  • Method: Paper2Agent represents each paper as a Model Context Protocol server, exposing its datasets, methods, models, or workflows through structured tools accessible to LLMs and agent frameworks.The workflow identifies key contributions, encapsulates them as MCP tools, and connects the server to an AI agent for diverse tasks.
  • Novelty: Natural-language access lowers barriers to adopting advanced methods and enables interactive collaboration beyond prior executable-paper and notebook-based approaches.Earlier efforts improved reproducibility but still required substantial technical familiarity for full engagement.

2 Results

Paper2Agent converts research papers into interactive, reproducible AI agents and demonstrates this capability across genomics, spatial transcriptomics, and single-cell analysis. Its agents support natural-language scientific workflows, benchmarked reliability and efficiency, and downstream hypothesis generation.

  • AlphaGenome: Paper2Agent generated 22 reusable AlphaGenome MCP tools in about 3 hours without human intervention, covering variant scoring, sequence prediction, tissue exploration, and visualization.The tools comprehensively cover AlphaGenome’s methodological innovations.
  • AlphaGenome: The AlphaGenome agent achieved 100% accuracy on tutorial-based and novel queries, exceeding Claude + Repo and Biomni.The benchmark evaluated numerical and qualitative results and figures against human-configured executions.
  • AlphaGenome: The AlphaGenome agent reduced median tutorial-query runtimes by 1.8x versus Claude + Repo and 3.1x versus Biomni, with novel-query improvements of 3.2x and 4.6x.These results demonstrate improved computational efficiency relative to both comparison systems.
  • AlphaGenome: The AlphaGenome agent independently prioritized SORT1 over the original paper’s CELSR2 and PSRC1, enabling users to reassess published conclusions with model-based evidence.The agent favored SORT1 based on a quantile score of 0.99982 and its role in LDL/VLDL secretion.
  • TISSUE: For TISSUE, Paper2Agent generated six tools for spatial prediction, prediction intervals, and uncertainty-aware analyses, while also providing interactive guidance on inputs and workflows.The agent can automatically filter and download relevant mouse spatial transcriptomics data before running the TISSUE pipeline.
  • Scanpy and AI co-scientist: The framework also supports converting selected method components, such as workflows from Scanpy, into tools and connecting MCPs to an AI co-scientist that autonomously generates hypotheses and analyses.The co-scientist used AlphaGenome to prioritize causal variants across all 39 loci within two hours.

A Paper2Agent enables autonomous AI-driven collaboration and discovery

Paper2Agent converts scientific papers into MCP resources for methods and data, enabling an AI co-scientist to integrate them and autonomously generate hypotheses and actionable research plans. In an ADHD GWAS and AlphaGenome application, the agent generated and tested hypotheses and identified causal variants.

  • Paper2Agent converts scientific papers into MCP resources for both methods and data, enabling AI co-scientists to integrate them.
  • The integrated AI co-scientist autonomously generates novel hypotheses and actionable research plans.
  • Using an ADHD GWAS dataset and AlphaGenome method MCPs, the agent autonomously generates and tests scientific hypotheses and identifies causal variants.

3 Discussion

Paper2Agent turns methodological papers into interactive agents that expose research contributions through natural-language interaction. The discussion frames agentification as both a practical route to reproducibility and a foundation for broader scientific collaboration, while noting code-quality and scope limitations.

  • 3 Discussion: Paper2Agent instantiated AlphaGenome, Scanpy, and TISSUE as agents, making their methodological contributions directly accessible through natural-language interaction.The initial focus is methodological papers because applying their algorithms and workflows often requires substantial technical expertise.
  • 3 Discussion: Incomplete, poorly documented, or error-prone codebases can prevent reliable agentification, while transformation difficulty may indicate a paper’s reproducibility and rigor.The discussion connects agentification readiness with expectations for clear data and code availability.
  • 3 Discussion: 100.0% accuracy: the AlphaGenome agent executed both tutorial-based and novel queries, outperforming Claude Code and Biomni in expert-evaluated benchmarking.The benchmark combined human expert-evaluated examples from the paper with novel examples testing generalizability.
  • 3 Discussion: Agentification may need to encompass related publication sequences rather than single papers, with one MCP aggregating refinements, benchmarks, and applications into a coherent interface.A paper is not always the best unit for representing an evolving research idea.
  • 3 Discussion: An “agent availability” section could make interactive embodiments explicit, incentivize agent-conducive research, and enable agents to connect methods, datasets, and cross-domain insights.The discussion envisions communities of interacting agents as a dynamic layer of scientific intelligence and AI-driven collaboration.

Agent availability

The Paper2Agent-generated AlphaGenome agent is publicly available through a Hugging Face Space.

  • Agent availability: The Paper2Agent-generated AlphaGenome agent is publicly available at https://huggingface.co/spaces/Paper2Agent/alphagenome_agent.

Extended Methods · Details on implementing Paper2Agent

Paper2Agent uses a Claude Code multi-agent workflow to convert a paper and public codebase into a production-ready MCP server, then connect it to an AI agent interface. The workflow coordinates codebase extraction, environment setup, tutorial-based tool synthesis, testing, refinement, deployment, and traceable recording.

  • Extended Methods: Paper2Agent implements four stages in Claude Code: codebase identification, environment configuration, MCP tool synthesis and generation, and testing, refinement, deployment, followed by agent connection.An orchestrator coordinates four specialized sub-agents across the workflow.
  • Details on implementing Paper2Agent: A tutorial-scanner indexes repository materials, distinguishes tutorials from other files, and summarizes resources that are useful for reuse.The resulting reports identify which materials should be retained for tooling.
  • Details on implementing Paper2Agent: The tutorial-tool-extractor-implementor turns generalized tutorial tasks into single-purpose functions with parameterized values, file inputs, saved artifacts, and standardized summaries.The implementation preserves tutorial fidelity while defining clear inputs, outputs, and defaults.
  • Details on implementing Paper2Agent: The test-verifier-improver repeatedly generates, runs, diagnoses, and fixes tests using tutorial examples to verify numerical and visualization fidelity.Repeatedly failing functions lose their MCP decorators and are excluded from the server, while results and logs are recorded.
  • Details on implementing Paper2Agent: The workflow locates the official repository, downloads associated resources, and provisions a clean workspace with pinned dependencies and verified imports.These steps target consistent execution across machines.
  • Details on implementing Paper2Agent: Selected tutorials are executed end-to-end with example data to capture inputs, outputs, figures, runtime constraints, and implicit assumptions.This audit makes tutorial behavior explicit before tool extraction.
  • Details on implementing Paper2Agent: The completed tools, resources, and prompts are assembled into a versioned MCP server with a manifest and basic security defaults, while stage results are recorded for traceability and reproducibility.The orchestrator invokes sub-agents as needed, and detailed setup instructions are available in the Paper2Agent GitHub repository.

Generation and analysis of AlphaGenome agent

Paper2Agent was applied to AlphaGenome to generate an MCP server and connect it with Claude Code as an AlphaGenome agent. Reproducibility was evaluated using 15 original tutorial-based and 15 novel queries compared against ground-truth answers.

  • Agent generation and deployment: Paper2Agent generated an AlphaGenome MCP and connected it with Claude Code to create an AlphaGenome agent.The MCP server is remotely hosted on Hugging Face Spaces.
  • Reproducibility evaluation: The AlphaGenome agent was evaluated on 30 queries comprising 15 original tutorial-based queries and 15 novel queries.The evaluation compared the agent’s responses with ground-truth answers to verify reproducibility.

Benchmarking the AlphaGenome agent against Claude + Repo and Biomni

The benchmark compares AlphaGenome, Claude + Repo, and Biomni agents using curated ground-truth answers, captured responses, manual grading, and aggregate performance summaries. All evaluations use Claude Sonnet 4 locally, while each agent receives distinct context and execution requirements for answering AlphaGenome queries.

  • Evaluation protocol: All benchmark queries were evaluated against manually curated and executed ground-truth answers, with agent responses manually reviewed and performance summarized across each dataset.The evaluation also captured runtime and cost metrics for each response.
  • Evaluation implementation: All agents used claude-sonnet-4-20250514, ran locally on an M2 MacBook Air in non-interactive command-line mode, and produced JSON-captured outputs.The authors state that model APIs were used as needed and provide a representative Claude command-line invocation.
  • AlphaGenome agent: The AlphaGenome agent was instructed to use its MCP tools for genomics questions and return valid JSON containing only a concise final answer plus reasoning.Its context specified that calculations or analyses should use the appropriate AlphaGenome MCP tools and that the API key was available in the project’s .env file.
  • Claude + Repo agent: The Claude + Repo agent was required to write and execute Python code with the AlphaGenome library, extract the exact result, and provide the requested quantitative answer.Its instructions required loading the API key from .env, creating the DNA model, executing complete code, and showing the execution results.
  • Biomni agent: Biomni was evaluated through its repository-provided API-based version, with prompts directing it to use AlphaGenome and return a valid JSON answer with reasoning.The Biomni context supplied the AlphaGenome repository and project API key location.

Generation and analysis of TISSUE agent

The Paper2Agent framework generated a TISSUE MCP and connected it to Claude Code to create a TISSUE agent. Reproducibility was assessed by comparing its outputs with human researchers’ analyses of identical Mouse somatosensory cortex ST data.

  • Generation and analysis of TISSUE agent: Paper2Agent generated a TISSUE MCP connected to Claude Code, and reproducibility was evaluated against human researchers using identical Mouse somatosensory cortex ST data.The human analysis followed the TISSUE GitHub tutorial.

Generation and analysis of Scanpy agent

Paper2Agent generated a focused Scanpy agent for reproducible single-cell RNA-seq preprocessing and clustering, deployed as an MCP server integrated with Claude Code. Across three public PBMC datasets, the agent reproduced the key workflow steps used by human researchers following official Scanpy tutorials.

  • Generation and analysis of Scanpy agent: Paper2Agent restricted the Scanpy agent to preprocessing and clustering workflows, creating a focused and reproducible single-cell RNA-seq pipeline.
  • Generation and analysis of Scanpy agent: The resulting Scanpy agent was deployed as an MCP server and integrated with Claude Code for natural-language interaction.
  • Generation and analysis of Scanpy agent: The generated MCP prompts encoded the standard Scanpy pipeline by requiring data inspection first and deviations from defaults only when necessary for correctness.
  • Generation and analysis of Scanpy agent: The agent reproduced gene filtering, normalization, principal component analysis, neighborhood graph construction, and clustering across three public 10x Genomics PBMC datasets, matching human tutorial-based workflows.

Analysis for AI co-scientists linking with AlphaGenome and ADHD GWAS data MCP

The analysis links AlphaGenome and ADHD GWAS data MCPs to generate and evaluate actionable scientific questions, using standardized genomic resources and strict evidence, feasibility, and validation constraints.

  • Data preparation: Paper2Agent converts ADHD GWAS publication text and supplementary Excel tables into cleaned, standardized MCP resources with consolidated metadata.This preprocessing makes publication-derived data available through structured Model Context Protocol resources.
  • Hypothesis generation: The hypothesis-generation prompt requests 10 actionable scientific questions that use the available tools to explore the data.Each proposed question is expected to include a title, summary, rationale, integration sketch, experiment plan, risks, mitigations, and resource checklist.
  • MCP integration: The workflow connects AlphaGenome and ADHD GWAS data as two MCP servers for integrated genomic analysis and hypothesis generation.The prompt requires questions to use both research artifacts and leverage their complementary capabilities.
  • Evaluation and safeguards: Questions are scored on novelty, feasibility, impact, validation clarity, and resource fit while restricting outputs to available MCP resources and explicit uncertainty handling.The instructions prohibit fabricated citations or tool outputs, require missing evidence to be labeled Unknown, and request fallbacks for failed tool calls.

Supplementary Figure

Supplementary Figure 1 shows the MCP tools and resources exposed for variant scoring and visualization.

  • Supplementary Figure: Supplementary Figure 1 presents the exposed MCP tools and resources that enable variant scoring and visualization.
Loading 2509.06917v2…