Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery

1 Introduction

• Clinical imaging interpretation is multi-step and tool-centric, combining visual evidence with patient context and procedures such as detection, segmentation, quantification, and longitudinal tracking.Single-task or single-turn evaluations under-represent this planning, interaction, and contextual variability [52].Source: §1 · pp. 1–2
• Existing medical agents keep tool sets and invocation strategies static after deployment, making them brittle as clinical protocols, patient distributions, modalities, and diagnostic requirements evolve.Fixed workflows may degrade under domain shifts and require extensive manual re-engineering.Source: §1 · pp. 2–3
• MACRO’s design draws on clinicians’ accumulation of reusable diagnostic routines, which are refined as new cases and constraints arise.The proposed agent grows and reuses multi-step primitives from interaction experience.Source: §1 · p. 3
• MACRO combines image-feature memory with a module that discovers, validates, registers, and reuses multi-step tool sequences as high-level primitives.A reward-aligned training loop reinforces reliable invocation of these composites with minimal manual supervision.Source: §1 · p. 3
• Autonomous composite tool discovery consistently improves multi-step orchestration accuracy and cross-task generalization over established baselines and recent state-of-the-art methods.The evaluation spans diverse medical imaging tasks and datasets.Source: §1 · pp. 3–4

LLM-based Medical Agent.

• Medical agents commonly have designer-defined pipelines and tools, and rigid workflows can struggle with novel or out-of-distribution cases.The cited work reports 15–30% accuracy on unseen multimodal clinical tasks.Source: LLM-based Medical Agent · p. 4

Tool-integrated Reasoning.

• Tool-integrated LLM agents use planner–executor or iterative thought–action–observation patterns, but many retain fixed tool libraries and do not update tool selection from outcomes.The passage gives Toolformer, ReAct-style agents, and HuggingGPT as examples of static tool designs.Source: Tool-integrated Reasoning · pp. 4–5

Agent Self-evolving.

• The paper claims no prior healthcare AI system continuously mines its own clinical experiences to create new composite tool chains.MACRO is described as identifying repeated multi-step sequences and encapsulating them as tools [44].Source: Agent Self-evolving · p. 5

3 Methods

• MACRO’s pipeline retrieves image-similar experiences, executes tool calls while appending their results to evidence, rewards use of registered composites, and updates memory and composite frequencies after a correct final answer.Successful trajectories and their contiguous subsequences support online composite discovery.Source: Figure 2 · p. 6, §3.1 · p. 7

3.1 Problem Formulation

• The agent–environment interaction is modeled as a POMDP whose state includes the clinical/environment state and accumulated experience memory.The agent receives only a partial observation of the augmented state.Source: §3.1 · p. 6
• The agent’s observation includes the medical image, action history, tool feedback, and memory retrieved using those inputs.Its action space includes atomic tools, dynamically growing composite tools, and an answer action that terminates the episode.Source: §3.1 · pp. 6–7
• Tool execution adds the selected action and returned result to the accumulated evidence and action history.At episode end, successful predictions gate memory updates; reinforcement learning also rewards rollouts containing validated composite procedures.Source: §3.1 · p. 7

Off-policy Trajectory Bootstrap.

• A strong teacher VLM generates high-quality multi-step tool-execution trajectories that both supervise behavior cloning and seed the experience memory.The passage identifies DeepSeek as the teacher and frames demonstrations as a way to avoid cold-start exploration in the large tool space.Source: Off-policy Trajectory Bootstrap · p. 7

• The memory stores each successful-interaction fragment as prompt/history context, a tool sequence, a summarized result, and an image feature vector from a frozen visual encoder.Source: Memory Store and Retrieval · p. 7
• For a query image, MACRO ranks memory entries by cosine similarity and adds the top-k examples to the prompt as visually grounded demonstrations.The similarity is the cosine between the query and memory image-feature vectors.Source: Memory Store and Retrieval · p. 7
• After an episode ends with a correct final prediction, its trajectory is decomposed into step-level entries and added to memory.Source: Memory Store and Retrieval · p. 8

3.3 Composite Tool Discovery

• Composite discovery turns recurring contiguous subsequences from successful tool trajectories into reusable action abstractions that can be invoked as single units.Source: §3.3 · p. 8
• A candidate subsequence becomes a composite tool when its frequency exceeds threshold τ.Source: §3.3 · p. 8
• The composite registry is updated throughout training, allowing new patterns to be registered as the policy improves and discovers successful procedures.Source: §3.3 · p. 8

3.4 Two-Stage Policy Optimization for Self-Evolving

• The supervised cold start initializes the policy on teacher demonstrations by minimizing a supervised action-prediction loss.Source: §3.4 · p. 8
• Training contexts come from the student’s own sampled, executed actions rather than teacher-generated contexts, mitigating exposure bias from standard behavior cloning.Source: §3.4 · p. 8
• After a correct final prediction, the resulting trajectory updates both the memory store and composite registry, enabling discovery before reinforcement learning begins.Source: §3.4 · p. 8
• GRPO follows the supervised warm-start because the policy may use atomic tools reliably yet underuse discovered composites.Source: §3.4 · p. 8

Stage 2: GRPO-based Reinforcement for Composite Utilization.

• For each input, MACRO samples a group of G on-policy trajectories and computes group-normalized advantages from their rewards.Source: Stage 2: GRPO-based Reinforcement for… · p. 8
• The rollout reward includes an indicator that is 1 when the rollout contains any registered composite and 0 otherwise.Source: Stage 2: GRPO-based Reinforcement for… · p. 8
• The GRPO update uses a clipped policy objective with a KL penalty relative to a reference policy frozen after Stage 1.The importance ratio compares the current policy with that reference policy.Source: Stage 2: GRPO-based Reinforcement for… · p. 9
• During rollouts, tool calls are executed in the environment, and trajectories with correct final predictions continue to update memory and the composite registry.Source: Stage 2: GRPO-based Reinforcement for… · p. 9

Reward Function.

• A step receives reward 1 when its invoked tool-name sequence contains a registered composite tool as a contiguous subsequence, and 0 otherwise.The reward encourages the agent to invoke composites as single conceptual units.Source: Reward Function · p. 9
• When the final answer matches the ground-truth label, the full tool sequence updates the memory store and composite registry.This trajectory-level success signal supports discovery of new composite patterns but does not directly affect Stage 2 policy-gradient updates.Source: Reward Function · p. 9
• Step-level composite rewards and trajectory-level success feedback together refine multi-step reasoning and acquire reusable action abstractions.The framework combines experience replay, automated action abstraction, and group-relative policy optimization in a POMDP-grounded training loop.Source: Reward Function · p. 9

4.1 Experimental Setup

• The experimental setup uses REFUGE2 for glaucoma, MITEA for heart disease, and RAM-W600 for bone-erosion diagnosis.For MITEA echocardiography, three randomly selected slices per volume serve as visual input; the base model is Qwen2.5-VL-3B-Instruct.Source: §4.1 · p. 10, Comparison with General VLMs · p. 10

Comparison with General VLMs.

• MACRO exceeds Qwen by 38.4% in glaucoma BACC and 64.0% in F1, and by 27.2% in heart-disease BACC and 74.9% in F1.The comparison covers general VLMs on REFUGE2 and MITEA; Table 1 reports results for these datasets.Source: Comparison with General VLMs · p. 10, Table 1 · p. 10

Comparison with Medical Agentic Systems

• Against MedAgent-Pro, MACRO gains 2.3% BACC and 3.9% F1 on glaucoma, while heart-disease BACC decreases 0.6% and F1 improves 2.6%.The prose describes MACRO as consistently outperforming the compared medical agentic systems across both disease domains, while noting the heart-disease BACC decrease.Source: Table 1 · p. 10, Comparison with General VLMs · p. 11

Comparison with Task-specific Models.

• On RAM-W600, mainstream task-specific models reach at most 52.64% BACC and 12.40% F1; ConvKAN has 94.70% specificity but 3.82% sensitivity.The authors attribute the pattern to majority-class bias and describe MACRO as gaining across all metrics; Table 2 covers the test set.Source: Table 2 · p. 11, Comparison with Task-specific Models · p. 12

Effectiveness of the Proposed Key Components.

• With memory, composite-tool discovery, and GRPO, the full system achieves the best results across both datasets, including 92.7% REFUGE2 BACC and 77.2% MITEA BACC.The ablation reports REFUGE2 BACC 82.8% and F1 69.7% for the baseline, a 6.1% REFUGE2 F1 increase with memory, and 90.6% REFUGE2 BACC and 80.3% F1 with composite discovery before GRPO.Source: Effectiveness of the Proposed Key… · pp. 12–13, Table 3 · p. 12
• Fig. 3 reports an 8.8% performance improvement with abstract tools over the basic tool library as tools and demonstrations improve.Its other panel depicts tool complexity decreasing as higher-level tools replace multi-step patterns.Source: Figure 3 · p. 13
• During training, registered composites grow rapidly and then stabilize, while their growth strongly correlates with BACC improvement.The account describes a shift from early discovery to later saturation as common combinations become covered; usage stabilizes through reward and memory reinforcement.Source: Effectiveness of the Proposed Key… · p. 13, Figure 3 · p. 13

Quantitative Analysis

• The glaucoma case converts the image to grayscale, segments the optic cup and disc, computes their ratio, and registers the workflow as a composite tool.The cup-to-disc ratio is identified as a key indicator for glaucoma diagnosis.Source: Quantitative Analysis · p. 13, Figure 4 · p. 14

From static orchestration to continual capability growth

• Recurring successful trajectories become reusable composite tools, turning interaction history into an expandable skill library for continual capability growth.The approach shifts from rigid orchestration toward a process likened to clinicians refining expertise through practice.Source: §5 · p. 14

A practical path toward maintainable clinical deployment

• Newly validated workflows can be absorbed as auditable, versioned primitives, offering a route to continual improvement without repeated end-to-end re-engineering.The proposed deployment benefit is potentially lower maintenance costs across institutions, particularly as protocols and patient case-mix drift.Source: §5 · p. 14

Limitations and future directions

• Composite quality is bounded by trajectory verification fidelity; noisy or partially correct traces risk adding suboptimal skills to the library.Source: §5 · pp. 14–15
• Transfer to unseen modalities may expose unbenchmarked failure modes, while the evaluation focuses on predictive metrics rather than prospective human-in-the-loop outcomes.Future work should assess calibration, workflow efficiency, and downstream clinical decision impact.Source: §5 · p. 15