Source-linked AI summary
MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning
Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, Daniel Rueckert
TL;DR
Medical VLMs often lack transparent, genuine reasoning, and SFT may overfit final-answer supervision while providing little incentive to cultivate reasoning. MedVLM-R1 uses GRPO with structured reasoning and rule-based rewards, achieving strong radiology performance and generalization despite limited scale, though important modality, task-format, and reasoning-quality limitations remain.
Problem
Existing medical VLMs often provide final answers or quasi-explanations instead of genuine step-by-step reasoning, while final-answer SFT offers little incentive to cultivate reasoning and can overfit.
Method
MedVLM-R1 trains a 2B-parameter medical VLM with GRPO using final-answer data, structured <think> and <answer> outputs, and rule-based format and accuracy rewards.
Results
MedVLM-R1 shows strong in-domain and out-of-domain radiology performance, improving CT and X-ray results over SFT by 16% and 35%, respectively, while outperforming larger models.
Takeaways & Limitations
The results support reinforcement learning as an approach for combining explicit reasoning with robust and efficient medical VLM development in radiology VQA.
Takeaways & Limitations
The approach is limited to closed-set multiple-choice VQA, degrades substantially on open-ended questions, and fails to converge on some other medical modalities.
Abstract
from arXiv · showhide
Reasoning is a critical frontier for advancing medical image analysis, where transparency and trustworthiness play a central role in both clinician trust and regulatory approval. Although Medical Visual Language Models (VLMs) show promise for radiological tasks, most existing VLMs merely produce final answers without revealing the underlying reasoning. To address this gap, we introduce MedVLM-R1, a medical VLM that explicitly generates natural language reasoning to enhance transparency and trustworthiness. Instead of relying on supervised fine-tuning (SFT), which often suffers from overfitting to training distributions and fails to foster genuine reasoning, MedVLM-R1 employs a reinforcement learning framework that incentivizes the model to discover human-interpretable reasoning paths without using any reasoning references. Despite limited training data (600 visual question answering samples) and model parameters (2B), MedVLM-R1 boosts accuracy from 55.11% to 78.22% across MRI, CT, and X-ray benchmarks, outperforming larger models trained on over a million samples. It also demonstrates robust domain generalization under out-of-distribution tasks. By unifying medical image analysis with explicit reasoning, MedVLM-R1 marks a pivotal step toward trustworthy and interpretable AI in clinical practice. Inference model is available at: https://huggingface.co/JZPeterPan/MedVLM-R1.
1 Introduction
Existing medical VLMs often provide final answers without genuine step-by-step reasoning, while SFT can encourage overfitting and offers little incentive to develop reasoning. MedVLM-R1 addresses this gap with GRPO-trained explicit reasoning using only final-answer data.
- Existing medical VLMs often provide final answers or quasi-explanations rather than genuine step-by-step reasoning.
- SFT can overfit to final-answer supervision, encourage shortcut learning, and weaken performance on out-of-distribution data.
- Direct final-answer supervision provides minimal incentive for cultivating reasoning abilities within medical VLMs.
- MedVLM-R1 is introduced as a medical VLM that generates explicit reasoning alongside final answers using GRPO for radiology VQA.
- MedVLM-R1 uses GRPO with datasets containing only final answers, enabling emergent reasoning without explicit reasoning supervision.
- The compact 2B-parameter model trained on 600 samples generalizes to out-of-distribution modalities and outperforms larger medical VLMs.
2 Related Work
Medical VLM development has largely relied on SFT over costly, large-scale image-text datasets, while reinforcement learning has emerged as an alternative for improving reasoning, interpretability, and robustness.
- Medical VLMs commonly use SFT on final-answer labels, requiring high-quality image-text datasets ranging from 660k to 32M samples.
- Reinforcement learning has emerged as an alternative for improving medical VLM interpretability and robustness.
- Classic PPO-based approaches can reward both answer correctness and the quality of intermediate reasoning steps.
3 Methods
MedVLM-R1 applies GRPO to radiology VQA, structuring outputs as reasoning traces and final answers while using rule-based rewards for format and correctness.
- Overview: MedVLM-R1 employs GRPO to incentivize explicit reasoning capabilities in medical VLMs through efficient reinforcement learning.
- Base Model and Prompt Template: The prompt supplies a radiology image and question, and the VLM outputs a reasoning trace and final answer in designated XML-like tags.
- Group Relative Policy Optimization (GRPO): GRPO samples multiple candidate outputs, computes rewards, and updates the model using group-relative advantages with clipped regularization.
- Group Relative Policy Optimization (GRPO): A KL penalty constrains divergence from the initial reference model to help prevent catastrophic forgetting.
- Reward function: The rule-based reward gives format credit for exactly structured reasoning and answer tags, then scores final-answer correctness as exact, partial, or zero.
- Reward function: The model receives a total reward ri ∈[0, 2] combining format and accuracy rewards.
4 Experiments
Experiments use a combined medical VQA benchmark focused on MRI, CT, and X-ray radiology, with MRI training and held-out in-domain and cross-modality testing. The implementation fine-tunes Qwen2-VL-2B with GRPO and compares it against larger VLMs and an SFT counterpart.
- Dataset: The evaluation dataset combines VQA-RAD, SLAKE, PathVQA, OmniMedVQA, and PMC-VQA into 17,300 multiple-choice image questions.
- Dataset: Training uses 600 MRI image-question pairs, while testing uses 300 MRI, 300 CT, and 300 X-ray pairs.
- Implementation details: Qwen2-VL-2B is fine-tuned with GRPO for 300 steps on two NVIDIA A100 SXM4 80GB GPUs, taking approximately 4 hours.
- Baseline methods and evaluation metric: Baselines include unmodified and larger Qwen2-VL models, HuatuoGPT-Vision-7B, and Qwen2-VL-2B fine-tuned with standard SFT on the same 600 MRI pairs.
5 Results and Discussion
MedVLM-R1 shows strong out-of-domain performance and explicit reasoning across radiology VQA, while its explanations can remain heuristic or superficial.
- Overall Performance: 16% improvement on CT and 35% improvement on X-ray over SFT counterparts demonstrates stronger out-of-domain performance.The model was fine-tuned on MRI data, with CT and X-ray treated as out-of-distribution tests.
- Reasoning Competence and Interpretability: In a chest X-ray example, the reasoning identifies lung markings and the ribcage before selecting the observed anatomy.The answer choices include lungs, bladder, brain, and heart.
- Reasoning Competence and Interpretability: MedVLM-R1 produces explicit reasoning in <think> tags and final decisions in <answer> tags, unlike all baselines.Figure 2 illustrates this format on X-ray, MRI, and CT examples.
- Reasoning Competence and Interpretability: Some complex cases reach correct answers through elimination or unclear causal chains, suggesting heuristic or potentially retrofitted explanations.The authors note that these imperfections limit the evidential strength of the generated reasoning.
- Limitations: The model’s scope is constrained by modality gaps, closed-set dependence, and occasional superficial or hallucinated reasoning.It fails to converge on some pathology or OCT tasks, degrades substantially on open-ended questions, and can provide misleading justifications.
6 Conclusion
MedVLM-R1 integrates GRPO-based reinforcement learning to combine accuracy, interpretability, and robust radiology VQA performance. The authors conclude that RL approaches generalize better than purely SFT methods, particularly under out-of-distribution settings.
- Conclusion: MedVLM-R1 integrates GRPO-based reinforcement learning to address accuracy, interpretability, and robust performance in radiology VQA.The model emphasizes explicit reasoning as a route toward transparency and trustworthiness in high-stakes clinical environments.
- Conclusion: RL-based approaches generalize better than purely SFT methods, particularly under out-of-distribution settings.The authors describe medical VLM reasoning as nascent and still facing considerable challenges.