PaperlayerView exact revision on arXiv

Source-linked AI summary

MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning

Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, Daniel Rueckert

arXiv:2502.19634v2cs.CVcs.AI

TL;DR

Medical VLMs often return final answers or quasi-explanations rather than genuine step-by-step reasoning, although explainable decisions matter to clinicians and patients. MedVLM-R1 uses GRPO-based reinforcement learning to generate explicit reasoning, with reported gains over SFT counterparts on CT and X-ray tasks.

  • Problem

    Existing medical VLMs often return final answers or quasi-explanations rather than genuine step-by-step reasoning, while explainable decisions matter to clinicians and patients.

    Source: §1 · p. 2

  • Method

    MedVLM-R1 applies GRPO-based reinforcement learning to radiology VQA, training a VLM to produce explicit reasoning alongside its final answer.

    Source: §1 · pp. 2–3, §3 · p. 4

  • Results

    16% improvement on CT and 35% improvement on X-ray versus SFT counterparts; despite 2B parameters and 600 samples, MedVLM-R1 outperformed larger models.

    Source: §5 · pp. 6–7

  • Takeaways & Limitations

    The authors conclude RL-based approaches generalize better than purely SFT methods, particularly under out-of-distribution settings.

    Source: §6 · p. 9

    In some cases, MedVLM-R1 gives correct answers with superficial or partial reasoning, and the causal link between reasoning and conclusion can remain unclear.

    Source: §5 · p. 8

Abstract

from arXiv · show

Reasoning is a critical frontier for advancing medical image analysis, where transparency and trustworthiness play a central role in both clinician trust and regulatory approval. Although Medical Visual Language Models (VLMs) show promise for radiological tasks, most existing VLMs merely produce final answers without revealing the underlying reasoning. To address this gap, we introduce MedVLM-R1, a medical VLM that explicitly generates natural language reasoning to enhance transparency and trustworthiness. Instead of relying on supervised fine-tuning (SFT), which often suffers from overfitting to training distributions and fails to foster genuine reasoning, MedVLM-R1 employs a reinforcement learning framework that incentivizes the model to discover human-interpretable reasoning paths without using any reasoning references. Despite limited training data (600 visual question answering samples) and model parameters (2B), MedVLM-R1 boosts accuracy from 55.11% to 78.22% across MRI, CT, and X-ray benchmarks, outperforming larger models trained on over a million samples. It also demonstrates robust domain generalization under out-of-distribution tasks. By unifying medical image analysis with explicit reasoning, MedVLM-R1 marks a pivotal step toward trustworthy and interpretable AI in clinical practice. Inference model is available at: https://huggingface.co/JZPeterPan/MedVLM-R1.

1 Introduction

MedVLM-R1 applies GRPO to radiology VQA so a lightweight medical VLM can generate explicit reasoning alongside answers without reasoning-reference supervision. The paper presents this approach as a way to improve generalization beyond the training domain.

1 Introduction

  • Existing medical VLMs often provide only final answers or quasi-explanations, while SFT can encourage overfitting and offers little incentive to develop reasoning.The authors also note that scaling teacher chain-of-thought data for SFT is prohibitively expensive in healthcare.Source: §1 · p. 2
  • MedVLM-R1 is introduced as a lightweight medical VLM that generates explicit reasoning alongside its final answers.The authors describe it as the first medical VLM with this capability.Source: §1 · pp. 2–3
  • Trained with GRPO on final-answer-only data, MedVLM-R1 demonstrates emergent reasoning without explicit reasoning supervision.This avoids the need for training data containing complex reasoning steps.Source: §1 · p. 3
  • Despite having 2B parameters and training on 600 samples, MedVLM-R1 generalizes to OOD tasks such as MRI-to-CT/X-ray and outperforms Qwen2VL-72B and Huatuo-GPT-Vision-7B.The introduction presents this as superior generalization and efficiency relative to larger models.Source: §1 · p. 3

2 Related Work

Related work frames medical VLM adaptation as dominated by data-intensive SFT that offers limited reasoning transparency and may generalize poorly, while RL is presented as a promising alternative for reasoning and robustness.

Medical VLMs and Their Limitations

  • Medical VLMs such as LLaVA-Med and HuatuoGPT-Vision show strong radiology-task results, but SFT on final-answer labels remains the dominant adaptation strategy.Reported training data requirements range from 660k to 32M samples [32], making high-quality curation costly and vulnerable to noise and privacy concerns.Source: §2 · p. 3
  • SFT provides little visibility into intermediate reasoning and can overfit narrow training distributions, weakening generalization to out-of-distribution clinical scenarios.Limited access to the model’s reasoning is identified as a barrier to clinician trust.Source: §2 · p. 3

Reinforcement Learning for Enhanced Reasoning

  • RL methods including PPO [26] can reward reasoning quality, while GRPO [27] removes PPO’s neural value-function estimator in favor of rule-based group-relative advantage.The passage presents RL as potentially more generalizable than SFT [11] and especially promising for resource-constrained settings such as DeepSeek-R1 [12].Source: §2 · p. 3

3 Methods

MedVLM-R1 uses reinforcement learning with GRPO to encourage explicit, interpretable reasoning in medical VLMs. Its radiology prompts and hierarchical rewards target accurate answers in a prescribed reasoning-and-answer format.

Overview

  • The approach adapts GRPO to radiology data across MRI, CT, and X-ray, using medical-image prompts and custom rewards for explicit reasoning and domain-specific answer formats.The authors describe it as an initial exploration of pure reinforcement learning for a multimodal medical reasoning model.Source: §3 · pp. 3–4

Base Model and Prompt Template

  • The method uses Qwen2-VL-2B [30] and trains it to produce a reasoning trace in <think> tags and a final answer in <answer> tags.Each training sample combines a radiology image with a question and fixed system message; the objective targets accurate, well-formatted answers with transparent reasoning.Source: §3 · p. 4

Group Relative Policy Optimization (GRPO)

  • GRPO samples candidate outputs, normalizes their rewards into group-relative advantages, and updates the model with clipped regularization and a penalty for divergence from its initial checkpoint.Outputs above the group-average reward receive an advantage; the KL penalty is intended to help prevent catastrophic forgetting.Source: §3 · pp. 4–5

Reward function

  • The rule-based reward gives 1 point for correctly formatted output and up to 1 point for answer accuracy, with partial credit for a correct choice stated with extra explanation or as answer content.The answer reward is 1 for an exact choice, 0.5 for a correct choice with additional explanation or choice content, and 0 for an incorrect or missing answer; the format reward requires each of four tags exactly once and no outside content.Source: §3 · p. 5

4 Experiments

Experiments evaluate MedVLM-R1 on MRI in-domain and CT/X-ray out-of-distribution radiology questions, using 600 MRI training pairs and comparisons with model and SFT baselines.

Dataset

  • 600 MRI pairs train MedVLM-R1, while 300 MRI, 300 CT, and 300 X-ray pairs form the in-domain MRI and out-of-distribution CT/X-ray tests.The processed HuatuoGPT-Vision evaluation dataset combines VQA-RAD, SLAKE, PathVQA [14], OmniMedVQA, and PMC-VQA [34], totaling 17,300 image-linked multiple-choice questions with 2–6 choices.Source: §4 · pp. 5–6

Implementation details

  • MedVLM-R1 adapts Qwen2-VL-2B with GRPO for 300 steps on two NVIDIA A100 80GB GPUs, using batch size 2 and six generation candidates.Fine-tuning takes approximately four hours.Source: §4 · p. 6

Baseline methods and evaluation metric

  • Baselines include Qwen2-VL-2B, -7B, and -72B, HuatuoGPT-vision, and same-base Qwen2-VL-2B SFT; scoring rewards correct single-letter answers, with strict answer-tag formatting for MedVLM-R1.SFT uses the same 600 MRI question-answer training pairs and negative log-likelihood; baselines use simple prompts, while MedVLM-R1 uses a reasoning prompt.Source: §4 · p. 6

5 Results and Discussion

MedVLM-R1 shows strong in-domain and out-of-domain performance with compact RL training, while producing explicit but sometimes incomplete or superficial reasoning; its current scope and explanations have important limitations.

Reasoning Competence and Interpretability

  • MedVLM-R1 generates explicit reasoning in <think> tags and final decisions in <answer> tags, a capability absent in the baselines.Examples span X-ray, MRI, and CT; reasoning on simpler questions appears cogent and aligned with medical knowledge.Source: §5 · pp. 7–8, Figure 2 · p. 7

Overall Performance

  • GRPO-trained MedVLM-R1 improves on SFT counterparts by 16% on CT and 35% on X-ray, and outperforms larger VLMs despite its 2B parameters and 600 training samples.MRI fine-tuned models are used for the ID/OOD comparisons; GRPO- and SFT-fine-tuned VLMs also outperform zero-shot general-purpose VLMs on in-domain tasks. The larger-model comparison includes Qwen2-VL-72B and HuatuoGPT-Vision-7B, the latter trained on large-scale medical data.Source: §5 · pp. 6–7, Table 1 · p. 8

Reasoning Competence and Interpretability

  • For complex queries, reasoning can be heuristic or partial, and unclear links between reasoning and conclusions raise the possibility of post-hoc explanations.One example reaches the correct answer by process of elimination rather than detailed medical analysis.Source: §5 · p. 8

Limitations

  • The model fails to converge on pathology and OCT images, and its performance degrades substantially on open-ended questions without predefined answer options.The authors hypothesize that insufficient pre-training exposure to these modalities contributes to the modality gap; closed-set dependence is also common among VLMs.Source: §5 · p. 8
  • Some correct answers have superficial or hallucinated justifications, and the authors identify the 2B-parameter backbone as a potential bottleneck.They plan to evaluate larger VLM backbones to address these concerns.Source: §5 · p. 8

6 Conclusion

MedVLM-R1 combines GRPO-based reinforcement learning with explicit reasoning to target accuracy, interpretability, and robust radiology VQA performance. The authors report better OOD generalization than purely SFT methods, while noting that medical VLM reasoning remains nascent and challenging.

6 Conclusion

  • MedVLM-R1 integrates GRPO-based reinforcement learning with explicit reasoning for radiology VQA.The approach aims to bridge accuracy, interpretability, and robust performance.Source: §6 · p. 9
  • The authors report that RL-based approaches generalize better than purely SFT methods, particularly under OOD settings.They frame explicit reasoning as supporting transparency and trustworthiness in high-stakes clinical environments.Source: §6 · p. 9
  • Medical VLM reasoning remains nascent and faces considerable challenges, despite its potential for safer, more transparent AI-driven healthcare.The authors argue that this potential should be encouraged.Source: §6 · p. 9

Related papers

Loading 2502.19634v2…