PaperlayerView exact revision on arXiv

Source-linked AI summary

MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning

Shengyuan Liu, Liuxin Bao, Qi Yang, Wanting Geng, Boyun Zheng, Chenxin Li, Wenting Chen, Houwen Peng, Yixuan Yuan

arXiv:2602.03320v1cs.CVcs.AI

TL;DR

Existing interactive segmentation methods often depend on single-turn or rigid interactions and lack process-level supervision for efficient autonomous tool use. MedSAM-Agent reformulates segmentation as multi-step decision-making with hybrid expert trajectories and two-stage process-aware training. Across 6 medical modalities and 21 datasets, it achieves state-of-the-art performance and robust generalization.

  • Problem

    Existing interactive segmentation methods rely on expert prompting, single-turn or rigid strategies, and limited process-level supervision, restricting autonomous and efficient tool use.

  • Method

    MedSAM-Agent uses hybrid box-and-point expert trajectories and a two-stage pipeline combining multi-turn outcome verification with clinical-fidelity process rewards.

  • Results

    MedSAM-Agent achieves state-of-the-art performance across 6 medical modalities and 21 datasets, with superior performance and robust generalization.

  • Takeaways & Limitations

    The framework unifies autonomous medical reasoning with iterative tool optimization while promoting interaction parsimony and decision efficiency.

Abstract

from arXiv · show

Medical image segmentation is evolving from task-specific models toward generalizable frameworks. Recent research leverages Multi-modal Large Language Models (MLLMs) as autonomous agents, employing reinforcement learning with verifiable reward (RLVR) to orchestrate specialized tools like the Segment Anything Model (SAM). However, these approaches often rely on single-turn, rigid interaction strategies and lack process-level supervision during training, which hinders their ability to fully exploit the dynamic potential of interactive tools and leads to redundant actions. To bridge this gap, we propose MedSAM-Agent, a framework that reformulates interactive segmentation as a multi-step autonomous decision-making process. First, we introduce a hybrid prompting strategy for expert-curated trajectory generation, enabling the model to internalize human-like decision heuristics and adaptive refinement strategies. Furthermore, we develop a two-stage training pipeline that integrates multi-turn, end-to-end outcome verification with a clinical-fidelity process reward design to promote interaction parsimony and decision efficiency. Extensive experiments across 6 medical modalities and 21 datasets demonstrate that MedSAM-Agent achieves state-of-the-art performance, effectively unifying autonomous medical reasoning with robust, iterative optimization. Code is available \href{https://github.com/CUHK-AIM-Group/MedSAM-Agent}{here}.

1. Introduction

MedSAM-Agent addresses the limited autonomy and inefficient interaction strategies of existing medical segmentation systems by treating interactive tool use as multi-step decision-making. It combines hybrid expert trajectory prompting with two-stage, process-aware training and reports state-of-the-art performance across 6 medical modalities and 21 datasets.

  • Existing SAM-derived interactive models remain dependent on expert prompting, preventing autonomous generalized segmentation without manual intervention.
  • Current approaches use single-turn prompts or rigid point-only trajectories, limiting adaptive refinement and underusing bounding boxes for global spatial context.
  • Its hybrid prompting strategy generates expert-curated trajectories using boxes and points to internalize human-like heuristics and adaptive refinement.
  • The two-stage training pipeline combines multi-turn outcome verification with clinical-fidelity process rewards to promote precise, non-redundant interactions.
  • MedSAM-Agent achieves state-of-the-art performance across 6 medical modalities and 21 datasets, demonstrating cross-modal generalization and tool-use versatility.
  • MedSAM-Agent reformulates medical image segmentation as a dynamic, multi-step decision-making process rather than static pixel-wise classification.

2. Related Works

Prior medical segmentation systems are often task-specific, while MLLM-driven approaches and RL-based agents address generalization and reasoning but retain important limitations. MedSAM-Agent is positioned within work combining language-based reasoning with interactive segmentation tools.

  • Traditional architectures such as UNet and nn-UNet require dedicated training for individual tasks, limiting generalization across diverse medical imaging scenarios.
  • Some MLLM-driven methods alter the output space with implicit tokens and pixel decoders, while remaining constrained in fine-grained pixel-level detail capture.
  • SAM-based interactive frameworks provide iterative refinement, but related MLLM-driven methods lack this capability.
  • Recent reinforcement-learning studies use simple verifiable rewards to extend model reasoning from text-only tasks to multimodal scenarios.

3. Methods

MedSAM-Agent models interactive medical segmentation as multi-step tool use, combining expert-curated trajectories with two-stage training and process-aware rewards. Its action space supports adaptive box and point interactions, visual feedback, and efficient termination.

  • Interactive decision process: The agent iteratively selects bounding boxes, positive or negative clicks, or a stop action to refine segmentation.Boxes provide a global spatial prior, while clicks target positive or negative refinement; stopping terminates the sequence when fidelity is reached.
  • Interactive decision process: MedSAM-Agent represents each state as the history of previous actions and observations, with updated masks fed back to guide subsequent actions.The segmentation tool produces the updated mask observation, which is appended to the rollout sequence for the next policy decision.
  • Expert-curated trajectory generation: The hybrid prompting strategy combines Box-to-Point and Sequential-Click trajectories to capture flexible clinical workflows beyond rigid point-only interactions.Box-to-Point trajectories begin with a jittered target bounding box and refine false-negative and false-positive regions using corrective clicks sampled from major error clusters.
  • Expert-curated trajectory generation: Progress-constrained sampling retains simulated actions only when they produce an incremental IoU gain above a predefined threshold, reducing redundant interactions.A retry mechanism excludes candidate actions that fail to satisfy the progress constraint.
  • Two-stage training: Training uses supervised fine-tuning for cold-start initialization followed by RLVR with a multi-dimensional reward that balances segmentation quality and interaction efficiency.The reward design includes protocol adherence, final IoU and Dice quality, progressive improvement, and penalties for overshoot and interaction cost.
  • Two-stage training: During supervised fine-tuning, expert actions are serialized as tool-call tokens and the model receives re-encoded mask overlays as visual prompts for autoregressive trajectory prediction.The visual feedback loop helps the agent observe refined boundaries and localize remaining errors at each step.

4. Experiments

Experiments evaluate MedSAM-Agent across 21 datasets, multiple segmentation backends, interaction strategies, training paradigms, and reward components. The results support strong multi-turn segmentation, tool-agnostic transfer, and efficient hybrid interaction.

  • Experimental Setup: 21 open-source datasets spanning 6 medical modalities support evaluation across CT, MRI, X-Ray, Ultrasound, Fundus, and Endoscopy.
  • Segmentation Performance: MedSAM-Agent consistently achieves high mask fidelity across the evaluated datasets while extending gains over general-purpose MLLM baselines.The reported segmentation results cover 6 modalities, with detailed performance for 21 datasets provided in the Appendix.
  • Multi-turn Interaction: The agent outperforms static Single-Turn Point and Single-Turn Box prompts through iterative refinement and adaptive selection of interaction depth.The final Mean IoU exceeds the theoretical upper bound of static prompts across evaluated modalities, while green outcome segments predominate across turns.
  • Zero-shot Tool Agnosticism: Training on one segmentation backend transfers to alternative engines with negligible performance degradation, demonstrating zero-shot tool agnosticism.Trajectories synthesized with MedSAM2 or IMISNet are evaluated against alternative backends including SAM2.
  • Effectiveness of Action Strategies: The Hybrid strategy reaches an IoU of 0.686, while box-to-point improves IoU from 0.623 to 0.649 and reduces average interaction turns from 2.94 to 2.06.Boxes provide global spatial context and subsequent points enable local refinement.
  • Analysis of Training Strategies: The two-stage SFT+RL pipeline combines anatomical grounding with strategic flexibility, whereas direct RL struggles with localization in challenging CT and MRI settings.SFT establishes domain grounding before RL optimizes tool generalization and procedural efficiency.
  • Impact of Reward Design: Removing Rimp decreases Dice from 0.794 to 0.772 and lowers IoU to 0.688, indicating that process-aware rewards contribute to mask fidelity.The ablation describes the baseline as producing inefficient trajectories and redundant operations.

5. Conclusion

MedSAM-Agent reframes medical image segmentation as autonomous, multi-step decision-making that connects high-level reasoning with precise tool interaction. Across diverse imaging modalities, the framework reports superior performance and robust generalization, with potential to reduce clinical workload.

  • MedSAM-Agent shifts medical image segmentation from static classification to an autonomous, multi-step decision-making process.
  • The framework combines hybrid prompting with a two-stage training pipeline and fine-grained process rewards to support self-refinement across diverse imaging modalities.
  • Experimental results report superior performance and robust generalization across diverse imaging modalities, with potential to reduce clinical professionals’ workload.

A. Datasets

The experiments use a broad dataset collection spanning six medical imaging modalities, with dataset descriptions organized by modality, region of interest, and sample-triplet counts.

  • The study uses 21 open-source datasets spanning CT, MRI, X-Ray, Ultrasound, Fundus, and Endoscopy.
  • Table 4 describes the datasets by imaging modality, region of interest, and number of image-text-label triplets.

B. Implementation Details

MedSAM-Agent constructs expert-like interaction trajectories through hybrid initialization and adaptive error-driven refinement. Each accepted action must improve IoU, while a structured closed-loop protocol lets the agent iteratively select box, point, or stop actions from visual feedback.

  • Trajectory construction: The hybrid strategy initializes with either a jittered bounding box or a centroid click, then applies unified error-driven refinement.False-negative and false-positive regions are localized with distance transforms to target the most significant morphological errors.
  • Trajectory construction: Each simulated action must exceed an incremental IoU threshold, with up to N retries to replace ineffective interactions.The threshold τ is typically 0.04; failed actions are resampled, and trajectory generation terminates early if no valid action is found.
  • Trajectory construction: 334,616 simulated trajectories use Box-to-Point and Sequential-Click initializations before filtering low-performing samples for SFT data.The trajectories are generated with MedSAM2; an IoU threshold of 0.7 retains 188,687 click-based and 260,446 box-based trajectories.
  • Interaction protocol: The interaction protocol enforces one tool action per turn and feeds the updated mask back to the agent before the next refinement decision.Available actions include adding a bounding box, adding a positive or negative point, and stopping when the mask accurately covers the target.
  • Interaction protocol: The resulting interaction sequences are formatted as standard SFT data to support cold-start training.The system prompt frames the model as a professional segmentation annotator and requires structured function-call outputs.

B.3. Training Details

Training uses normalized spatial prompts, frozen visual components during SFT, and asynchronous multi-turn reinforcement learning. The RL design omits explicit thinking tokens to prioritize dense image features and reduce token overhead.

  • Training setup: All images use 1024 × 1024 resolution, with spatial coordinates normalized to absolute integers in [0, 1000].This normalization is used for compatibility with Qwen3-VL and high-precision interactive segmentation.
  • Training setup: During SFT, only the LLM backbone is updated while the vision encoder and projector remain frozen.Training uses Llama-Factory, a learning rate of 1 × 10−5, batch size 64, four epochs, and DeepSpeed ZeRO-3.
  • Reinforcement learning: RL uses asynchronous multi-turn sampling with a maximum interaction depth of 5 turns.The Verl implementation uses learning rate 1×10−5, batch size 8, and sampling size 8 per prompt with SGLang runtime integration.
  • Reinforcement learning: The RL stage omits a dedicated <think> token to accelerate inference and emphasize dense image-level features over textual deliberation.This choice is motivated by the fine-grained visual demands of medical segmentation.

C.1. Comparison Experiment

MedSAM-Agent is evaluated against general-purpose and medical interactive segmentation baselines across 21 datasets spanning 6 medical modalities. The experiments emphasize multi-turn correction and qualitative mask refinement from global localization to local error correction.

  • Quantitative comparison: 21 datasets across 6 medical modalities are evaluated in the comparison experiments.Baselines include SAM2, MedSAM2, and IMISNet with both Point and Box prompts.
  • Quantitative comparison: MedSAM-Agent improves baseline segmentation through strategic multi-turn interaction that identifies residual errors and applies corrective logic.The reported pattern links agent performance to the underlying segmentation model while showing gains beyond single-turn approaches.
  • Qualitative analysis: The agent iteratively evaluates the current mask and changes its strategy using visual feedback, unlike traditional single-turn models.The qualitative examples cover various medical imaging modalities and illustrate interactive reasoning capabilities.
  • Qualitative analysis: Sequential refinement typically begins with a global constraint and then applies local corrective measures to remove hallucinations and fill under-segmented regions.The paper presents this loop as mimicking a clinician’s iterative workflow and improving mask fidelity in challenging clinical environments.

D. Future Works

Future work targets volumetric segmentation, broader multimodal clinical assistance, and computational efficiency. The planned extensions address 3D anatomical continuity, additional imaging tasks, and the latency costs of iterative inference.

  • Expansion to volumetric modalities: The framework is planned for extension from 2D slices to 3D CT and MRI volumetric contexts.The intended action space and reinforcement-learning environment would capture cross-slice anatomical continuity.
  • Unified multimodal agent: Future work proposes a unified medical AI assistant spanning segmentation, Medical VQA, lesion classification, and report generation.The stated goal is a cohesive architecture supporting transitions among perception, reasoning, and high-precision interaction.
  • Computational efficiency: Multi-turn iterative paradigms have increased cumulative inference latency as a potential limitation.Planned remedies include early exits, speculative decoding, and KV-cache optimization for multimodal agents.
Loading 2602.03320v1…