PaperlayerView exact revision on arXiv

Source-linked AI summary

MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science

Ran Xu, Yuchen Zhuang, Yishan Zhong, Yue Yu, Zifeng Wang, Xiangru Tang, Hang Wu, May D. Wang, Peifeng Ruan, Donghan Yang, Tao Wang, Guanghua Xiao, Xin Liu, Carl Yang, Yang Xie, Wenqi Shi

arXiv:2506.04405v2cs.CLcs.AIcs.LG

TL;DR

Biomedical coding agents need benchmarks and training environments that support executable, privacy-conscious reasoning across real-world data-science tasks. MedAgentGym provides a large interactive benchmark and scalable trajectory-generation infrastructure, and Med-Copilot improves through systematic training while the platform benchmarks commercial and open-source LLMs. The resource is intended as an extensible environment for developing biomedical coding assistants, with downstream use bounded by privacy, validation, and regulatory requirements.

  • Problem

    Biomedical coding requires integrated medical reasoning and programming, but proprietary LLM deployment faces privacy and cost constraints while open-source models show substantial capability gaps.

  • Method

    MedAgentGym combines 72,413 executable biomedical coding tasks across 129 categories and 12 scenarios with isolated sandboxes, interactive feedback, verifiable outputs, and scalable trajectory generation.

  • Results

    Benchmarking 29 LLMs reveals substantial commercial–open-source performance disparities, while Med-Copilot improves coding proficiency through systematic training and trajectory sampling.

  • Takeaways & Limitations

    MedAgentGym provides a privacy-preserving, extensible training and evaluation environment for advancing LLM-based biomedical coding assistants.

  • Takeaways & Limitations

    The resources use publicly available or credentialed deidentified datasets, exclude protected health information, and require additional subgroup, setting-specific, validation, and regulatory review before downstream use.

Abstract

from arXiv · show

We introduce MedAgentGym, a scalable and interactive training environment designed to enhance coding-based biomedical reasoning capabilities in large language model (LLM) agents. MedAgentGym comprises 72,413 task instances across 129 categories derived from 12 authentic real-world biomedical scenarios. Tasks are encapsulated within executable sandbox environments, each featuring detailed task specifications, interactive feedback mechanisms, verifiable ground truth annotations, and scalable training trajectory generation. Extensive benchmarking of 29 LLMs reveals substantial performance disparities in biomedical data science between commercial and open-source LLMs. Leveraging efficient multi-threaded and multi-turn trajectory sampling in MedAgentGym, Med-Copilot achieves performance gains of +43.02% and +45.28% from offline and online reinforcement learning, respectively, demonstrating MedAgentGym as an effective training ground while establishing itself as a cost-effective, privacy-preserving alternative competitive with proprietary LLMs (gpt-4o). By offering a unified execution environment with a comprehensive benchmark and accessible, extensible training resources, MedAgentGym delivers an integrated platform to develop LLM-based coding assistants for advanced biomedical data science.

1 INTRODUCTION

Biomedical data science requires both medical expertise and programming, while proprietary LLMs face privacy and cost barriers and open-source models remain deficient. MedAgentGym addresses these gaps with a scalable benchmark and interactive training environment for coding-centric biomedical reasoning.

  • The platform targets two infrastructure gaps: comprehensive benchmarks for diagnosis and evaluation, and interactive training environments for robust biomedical coding reasoning.
  • 72,413 instances across 129 categories and 12 real-world scenarios form MedAgentGym’s comprehensive code-centric biomedical reasoning suite.Tasks span information retrieval, clinical reasoning, bioinformatics, and machine learning across EHRs, clinical notes, genomics, drugs, and biological sequences.
  • MedAgentGym provides executable, isolated, reproducible task environments with pre-installed dependencies, parallel execution, and scalable trajectory collection.These capabilities support multi-threaded training and automated evaluation across diverse agent scaffolds.

2 RELATED WORKS

Existing biomedical benchmarks emphasize narrative reasoning or isolated tasks rather than complete coding-intensive data-science workflows. MedAgentGym extends this landscape with executable, interactive environments spanning the biomedical data-science lifecycle.

  • Prior benchmarks cover medical knowledge, isolated algorithms, or partial clinical workflows but do not capture the full lifecycle from data extraction to model development.
  • The platform focuses on computational code generation for biomedical reasoning rather than traditional ICD-9 or ICD-10 medical coding systems.
  • MedAgentGym uniquely integrates executable environments, interactive feedback, and task-isolated runtime facilities for biomedical coding reasoning.
  • MedAgentGym targets computation- and coding-intensive biomedical tasks requiring agents to retrieve, transform, analyze, and compute data while executing code.

3 MEDAGENTGYM: A SCALABLE AND INTERACTIVE LLM AGENT TRAINING ENVIRONMENT FOR CODE-CENTRIC BIOMEDICAL REASONING

MedAgentGym formulates biomedical coding as executable problem solving with verifiable outputs and builds a large, extensible benchmark around isolated interactive environments. Its design supports scalable trajectory generation, reproducible evaluation, and integration of new biomedical tasks.

  • 3.1 PROBLEM FORMULATION: Each task maps a problem description x to code c and output y, with correctness verified by comparing y against a ground-truth output y* through E.
  • 3.1 PROBLEM FORMULATION: MedAgentGym generates multiple single-turn or multi-turn coding trajectories through parallel agent execution, retaining both successful and erroneous executions as learning signals.
  • 3.2 DATA CONSTRUCTION: FROM INDIVIDUAL DATASETS TO UNIFIED BENCHMARK: The unified benchmark contains approximately 72,000 instances across 129 categories and 12 biomedical scenarios, with full-scale and lightweight versions plus train, test, and external validation splits.The dataset also uses n-gram string matching to reduce contamination and supports different research and leaderboard needs.
  • 3.3 CODING ENVIRONMENT: FROM STATIC BENCHMARK TO INTERACTIVE INTERFACE: Task-specific Docker sandboxes provide pre-installed biomedical dependencies, isolation, interactive error feedback, and reproducible execution for coding agents.
  • 3.3 CODING ENVIRONMENT: FROM STATIC BENCHMARK TO INTERACTIVE INTERFACE: Multi-threading, parallel execution, sequential sampling, and a modular plug-and-play architecture make trajectory collection scalable and task integration extensible.

4 EVALUATING LLMS AS MEDICAL CODING AGENTS WITH MEDAGENTGYM

MedAgentGym evaluates coding agents across diverse biomedical datasets using interactive agent scaffolds and execution-based metrics. Zero-shot results show commercial–open-source and task-structure performance gaps, while the setup supports offline and online reinforcement-learning comparisons.

  • 4.1 EXPERIMENTS SETUP: The default scaffold models agents as interacting with a partially observable environment through information retrieval, terminal, code execution, and debugging actions.
  • 4.1 EXPERIMENTS SETUP: Med-Copilot is evaluated on 7,794 coding-based biomedical reasoning tasks spanning eight datasets with success rate, exact-match execution, and accuracy metrics.
  • 4.2 RESULTS: BENCHMARKING LLMS AND REASONING MODELS WITH MEDAGENTGYM: Zero-shot benchmarking reveals a substantial performance gap between commercial API-based and open-source LLMs on biomedical data science tasks.The results motivate development of lightweight open-source models that address privacy and cost constraints.
  • 4.2 RESULTS: BENCHMARKING LLMS AND REASONING MODELS WITH MEDAGENTGYM: LLMs perform better on structured database and calculation tasks than on open-ended data-analysis and machine-learning tasks requiring advanced coding and reasoning.
  • 4.2 RESULTS: BENCHMARKING LLMS AND REASONING MODELS WITH MEDAGENTGYM: Coding-specialized and medical-reasoning LLMs both show suboptimal performance, indicating that biomedical coding reasoning is not captured by either specialization alone.

5 TRAINING LLM AGENTS FOR CODE-CENTRIC BIOMEDICAL REASONING

MedAgentGym trains biomedical coding agents through interactive execution, trajectory sampling, supervised fine-tuning, and reinforcement learning. Scaling, self-improvement, external evaluation, debugging, and error analysis show where performance improves and remains constrained.

  • RL fine-tuning with trajectory sampling: SFT improves structured coding tasks, DPO benefits open-ended tasks, and GRPO achieves stronger online-RL performance than the other evaluated online methods.These methods use successful trajectories, preference optimization, or correctness and format rewards to improve agent behavior.
  • Scaling LLM agent improvements with MedAgentGym: Pass@K rises from 17.0% at K = 1 to 45.0% at K = 16, while Best@K increases from 17.0% to 41.7%.The small gap between metrics indicates that the trained verifier can identify successful trajectories effectively.
  • Model performance scaling with self-improvement: Rejection-sampling SFT produces the largest self-improvement gain, while later DPO iterations yield smaller incremental improvements.The authors attribute diminishing returns to a shrinking exploration space on increasingly difficult tasks.
  • Generalization, ablation, and error analysis: GRPO markedly improves performance on unseen, out-of-distribution biomedical coding tasks.
  • Generalization, ablation, and error analysis: Removing debugging significantly decreases performance across all tasks, showing that interactive error interpretation and correction support biomedical coding reasoning.
  • Generalization, ablation, and error analysis: Loop-related errors comprise 50.39% of gpt-4.1’s errors, with repeated final-turn actions indicating difficulty adapting or exploring alternatives.

6 CONCLUSION

MedAgentGym is presented as an executable, privacy-preserving, extensible environment for scaling code-based biomedical reasoning. Its benchmark and training resources support Med-Copilot improvements and broader biomedical data-science applications.

  • MedAgentGym provides 72K task instances across 129 categories for benchmarking 29 proprietary and open-source LLMs.Its modular architecture supports flexible and extensible biomedical data-science experiments.
  • Med-Copilot improves coding proficiency through systematic training and trajectory sampling in MedAgentGym.The paper positions the environment as a path from structured medical retrieval toward open-ended computational biomedical research.

ETHICS STATEMENT

The study uses credentialed or publicly available deidentified datasets and executes tasks in isolated containers. It restricts redistribution of credentialed data and cautions that outputs require validation before downstream clinical use.

  • The study uses publicly available or credentialed deidentified datasets and does not redistribute data requiring credentialed access.Instead, it provides scripts for obtaining and preparing such datasets under their licenses or data-use agreements.
  • Isolated Docker containers protect environment integrity and prevent unintended modification of underlying data.The benchmark verifies solutions through execution outputs rather than raw code, and released artifacts contain no protected health information.
  • Released outputs must not guide diagnosis or treatment without formal validation and regulatory review.The authors also recommend subgroup and setting-specific audits before downstream use.

REPRODUCIBILITY STATEMENT

The paper targets reproducibility through an anonymized artifact containing benchmark and agent code, environment specifications, evaluation tools, and dataset-preparation scripts. It documents the task, data, split, sandbox, interface, scaffold, and action-space specifications.

  • The released artifact includes source code, Dockerfiles, pinned dependencies, evaluation harnesses, and licensed data-preparation scripts.
  • The paper specifies task taxonomy, data sources, train/test splits, sandbox and interaction interface, agent scaffold, and action space.

A.1 LIMITATIONS

MedAgentGym has substantial resource, coverage, modality, access, and deployment limitations despite reported improvements in coding-aided biomedical reasoning. Its current scope is mainly text and structured data, while broader scaling and clinical use require additional infrastructure, validation, oversight, and multimodal extensions.

  • A.1 LIMITATIONS: MedAgentGym requires substantial computational resources for trajectory sampling, fine-tuning, and iterative self-improvement, limiting accessibility for resource-constrained groups.The dataset and trajectory collection are also constrained by computational budget rather than data availability, limiting exploration of scaling behavior.
  • A.1 LIMITATIONS: Its current support for text and structured data excludes multimodal biomedical inputs such as medical imaging, EEG, audio, and video.The authors identify data collection, curation, and standardized evaluation as challenges for effective multimodal integration.
  • A.2 BROADER IMPACTS: Limited computational resources and inadequate data infrastructure may prevent some institutions from benefiting fully, potentially widening healthcare disparities.Reliance on publicly available datasets may also perpetuate biases from uneven representation of underrepresented patients and rare diseases.
  • A.3 PRIVACY STATEMENTS: Clinical deployment requires rigorous validation, transparent communication of limitations and uncertainties, human oversight, and collaboration with healthcare professionals.These safeguards are presented as necessary to mitigate ethical and practical risks.

C.6 SAMPLED TRAJECTORY DETAILS

Sampled trajectories differ by task type: structured tasks emphasize data retrieval, whereas open-ended tasks require more coding, debugging, and dependency management. Performance improves consistently with more training data, but current trajectory volume is constrained by computational budget.

  • C.6 SAMPLED TRAJECTORY DETAILS: Over 50% of structured-task actions involve data retrieval, while open-ended tasks require substantial coding, debugging, and terminal interactions.Open-ended tasks may require installing specialized biomedical packages.
  • C.6 SAMPLED TRAJECTORY DETAILS: Increasing training data volume produces consistent performance improvements, indicating that additional trajectory sampling could yield further gains.The current trajectory count primarily reflects computational budget constraints.

D BASELINE DETAILS

The baseline suite combines coding-focused, medical-domain, and medical-reasoning models to characterize capabilities relevant to biomedical coding tasks. These models differ in their pretraining or fine-tuning emphasis, including code generation, medical QA, and verifier-guided reasoning.

  • D BASELINE DETAILS: Qwen2.5-Coder-Instruct is explicitly fine-tuned on large-scale coding data and coding-specific instructions to improve code generation, debugging, and programmatic reasoning.It is described as outperforming similarly sized general-purpose models on coding tasks.
  • D BASELINE DETAILS: medgemma-4b-it is a medical-domain model fine-tuned on medical QA and instruction datasets for medical reasoning and question answering.
  • D BASELINE DETAILS: HuatuoGPT-o1-7B uses PPO with verifier-based rewards and a medical-specific verifier to guide complex reasoning trajectories.Its intermediate reasoning steps support iterative refinement and introspective evaluation.
  • D BASELINE DETAILS: m1-7B-23K, MedReason-8B, and related medical models are fine-tuned for medical QA, clinical reasoning, or step-by-step rationales.The supplied baseline descriptions position these models as domain-specialized comparators.

E IMPLEMENTATION DETAILS

The implementation evaluates executable biomedical code with task-appropriate metrics under bounded interactive sessions and reports additional measures of interaction, complexity, readability, and efficiency. Training uses distinct hardware and optimization configurations for SFT, DPO, PPO, and GRPO.

  • E IMPLEMENTATION DETAILS: Success rate is the primary metric, using exact-match execution outputs for explicit-ground-truth tasks and accuracy for open-ended clinical ML tasks.The evaluation recognizes that code-generation tasks have effectively infinite solution spaces unlike bounded classification problems.
  • E IMPLEMENTATION DETAILS: Interactions are capped at 15 turns and 120 seconds per session, with input capped at 32,768 tokens and output capped at 8,192 tokens per round.Python 3.10 is used for agent-code execution.
  • E IMPLEMENTATION DETAILS: SFT trains models with AdamW at a 1e-4 learning rate, while DPO uses 5e-6 with batch size 64 and KL coefficient β=0.1.SFT uses batch size 8 and hardware varying from eight A100s for models up to 8B to eight H200s for 14B models.
  • E IMPLEMENTATION DETAILS: PPO and GRPO use batch size 128 and learning rate 1 × 10^-5 within the VeRL framework and Med-Copilot Docker environment.
  • E IMPLEMENTATION DETAILS: Additional evaluation measures interaction turns, cyclomatic complexity, maintainability, lines of code, and logical lines of code.Across datasets, advanced closed-source models generate more complex and longer code, while training changes Med-Copilot’s code profiles.

F.2 COST ANALYSIS

The cost analysis compares API token efficiency and training interventions, while qualitative cases show how feedback and fine-tuning improve biomedical code generation.

  • Cost analysis: Lower token usage makes gpt-4.1-mini and gpt-4o-mini cost-efficient choices for large-scale biomedical reasoning.These smaller variants offer superior token efficiency without substantial performance compromise.
  • Backbone and training analysis: SFT markedly improves structured coding across four OSS backbones, whereas DPO is particularly effective on open-ended tasks.
  • Warm-up ablation: Combining an initial SFT warm-up with later DPO improves results more than DPO alone.The two stages provide complementary strengths.
  • Interactive coding: Interactive debugging feedback enables agents to identify coding errors and iteratively refine solutions for complex biomedical programming tasks.In the case study, feedback led the agent to replace an incorrect graph method call with the standalone text_type function.
  • Case studies: Fine-tuned models handle domain-specific biological constraints and current clinical standards more accurately than baseline models in qualitative cases.Examples include ploidy-dependent chromosome copy numbers and correct medical-formula implementation.

F.7 DIFFICULTY ANALYSIS ON EXTERNAL EVALUATION SET

External evaluations examine generalization across unseen biomedical coding tasks and human reference performance, including structured and open-ended settings.

  • External evaluation sets: The external evaluation includes four challenging unseen medical coding tasks, including raw MIMIC-Extract data requiring feature engineering and preprocessing.A fully pre-processed MIMIC-Extract version is additionally evaluated for comparison.
  • External generalization: BixBench performance demonstrates Med-Copilot’s robustness beyond medical coding to broader scientific analytical tasks.
  • Trajectory validation: Analysis of more than 250 trajectories finds logically sound paths in the vast majority of successful solutions, with flawed-code correct answers below 1%.
Loading 2506.04405v2…