PaperlayerView exact revision on arXiv

Source-linked AI summary

Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning

Junha Jung, Minbyul Jeong, Suhyeon Lim, Sungwook Jung, Jaehoon Yun, Taeyun Roh, Mujeen Sung, Jaewoo Kang

arXiv:2606.31825v1cs.CVcs.AI

TL;DR

Outcome-centric post-training provides sparse, delayed supervision for the step-by-step reasoning needed in open-ended medical VQA. MRPO adds step-wise rewards that penalize earlier invalid reasoning more strongly, and across three multimodal backbones it consistently outperforms GRPO and GDPO, surpassing HuatuoGPT-Vision-34B by 2.79 points on Qwen3-VL-8B-Instruct.

  • Problem

    Outcome-centric post-training lacks differentiated supervision for identifying intermediate reasoning failures in open-ended medical VQA.

  • Method

    MRPO incorporates step-wise process rewards and assigns exponentially larger penalties to earlier invalid reasoning steps when the final answer is incorrect.

  • Results

    Across three multimodal backbones, MRPO achieves the highest average performance over GRPO and GDPO, surpassing HuatuoGPT-Vision-34B by 2.79 points on Qwen3-VL-8B-Instruct.

  • Takeaways & Limitations

    MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, indicating improved reasoning failure patterns alongside stronger medical VQA performance.

  • Takeaways & Limitations

    MRPO’s step-wise reward construction relies on gold reasoning annotations from MedThink, limiting direct application to settings without gold rationales.

Abstract

from arXiv · show

Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis-lab/MRPO

1 Introduction

Medical MLLM post-training is limited by sparse, outcome-centric feedback and cascading early reasoning failures. MRPO addresses both issues with step-wise process rewards and exponentially stronger penalties for earlier invalid steps, improving reasoning and answer accuracy across three backbones.

  • Motivation: Medical MLLM post-training remains outcome-centric, causing sparse credit assignment because final or sequence-level rewards cannot identify failed intermediate reasoning steps.The problem is especially severe for free-form generation, where rewards are sparse and delayed until the response is complete.
  • Failure analysis: Early reasoning failures propagate and accumulate into failure cascades, with the first invalid step strongly correlated with an incorrect final answer.The analysis is based on sentence-level reasoning traces from existing MLLMs on open-ended medical VQA benchmarks.
  • Method: MRPO combines answer rewards with step-wise process rewards and assigns exponentially larger penalties to earlier invalid steps when the prediction is incorrect.The method reshapes the GRPO (Shao et al., 2024)-based advantage while preserving successful trajectories and encouraging correction at the first invalid step.
  • Empirical results: MRPO achieves the highest average performance across three multimodal LLM backbones, outperforming GRPO and GDPO with only 13K training samples.The evaluation uses Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct on diverse open-ended medical VQA benchmarks.
  • Empirical results: MRPO reduces early-stage reasoning failures from 64.0% to 13.0% while mitigating downstream failure accumulation.This improvement targets the failure pattern in which early invalid reasoning propagates and derails the final answer.
  • Empirical results: On Qwen3-VL-8B-Instruct, MRPO outperforms the larger HuatuoGPT-Vision-34B by 2.79 points.This comparison is reported as part of the cross-backbone evaluation.

2 Related Works

Medical multimodal language models have progressed from supervised fine-tuning to GRPO-based reinforcement learning for vision-language tasks. Recent methods also move beyond final-answer supervision by evaluating reasoning steps as process rewards, inference-time verifiers, or offline preferences.

3 Preliminary Experiments

Preliminary experiments across four multimodal LLMs and three medical VQA benchmarks show that early reasoning failures are strongly associated with incorrect answers and subsequent failure accumulation. These findings motivate step-wise penalties that target earlier invalid reasoning stages rather than distributing learning signals uniformly across tokens.

  • Models and Benchmarks: The analysis covers four general-purpose or medical-specialized MLLMs evaluated on VQA-RAD, SLAKE, and PathVQA, with MedThink providing aligned gold rationales.The models are Qwen3-VL-8B-Instruct, InternVL3-8B-Instruct, HuatuoGPT-Vision-7B, and Lingshu-7B.
  • Results and Analysis: Earlier first reasoning failures correspond to higher incorrect-answer rates, while incorrect instances show greater failure accumulation across FFP bins.Figure 1 analyzes FFP, defined as the first invalid step’s relative position k/K, and FAR, the proportion of failed remaining steps after that failure.
  • Evaluation Metrics: Step validity is judged at the sentence level by GPT-5-mini using either Groundedness or Answer Contribution, allowing valid reasoning paths that differ from the gold rationale.The protocols assess domain-relevant observations or inferences and whether a step directly supports the ground-truth answer; human evaluation finds substantial agreement with both protocols.
  • Results and Analysis: The observed error cascade motivates MRPO, which evaluates reasoning step-wise and assigns larger penalties to tokens in earlier invalid stages when a trajectory fails.This targets root causes of reasoning failure, unlike standard GRPO-based methods that distribute learning signals uniformly across tokens.

4 Approach

MRPO combines answer, step-wise reasoning, and length rewards with step-aware advantage shaping to correct early reasoning failures in medical VQA. It penalizes earlier invalid steps more strongly only when the final answer is incorrect, while preserving successful reasoning paths.

  • Reward Design: The method combines answer, step-averaged reasoning-process, and length rewards into a total training reward.Answer quality uses lexical overlap and semantic similarity, while the length component regularizes reasoning-trace size.
  • Reasoning Process Reward: Reasoning-process rewards mark a step valid when it aligns with gold findings or contributes directly to the correct answer.This permits alternative reasoning trajectories that reach the same diagnosis, rather than requiring exact gold-path matching.
  • Length Reward: The length reward penalizes traces outside 4–10 sentence-level steps, encouraging reasoning of appropriate length.It penalizes both insufficient traces that omit diagnostic steps and excessively long traces containing redundant information.
  • Policy Optimization: MRPO addresses sequence-level GRPO supervision by differentiating learning signals across reasoning steps, using τ = 0.6 to classify final answers.GRPO (Shao et al., 2024) otherwise applies the same sequence-level signal uniformly across tokens, limiting mitigation of early-stage failures.
  • Step-wise Advantage Shaping: MRPO reshapes token-level advantages so earlier invalid reasoning steps receive exponentially larger penalties when the final answer is incorrect.Correct-answer trajectories are not reweighted, preserving successful reasoning paths while targeting failed traces.

5 Experiments

Across three comparable multimodal LLM backbones, MRPO improves medical VQA performance over baselines and consistently outperforms alternative training methods. Its step-wise optimization also delays failure onset and reduces downstream failure accumulation.

  • Experimental Setup: Experiments train on open-ended medical VQA data from VQA-RAD, SLAKE, PathVQA, and MedThink, evaluate three open-source backbones, and use GPT-5-mini binary answer judgments.Evaluation includes in-distribution and unseen-modality out-of-distribution benchmarks, with human alignment validation reported for the judging protocol.
  • Comparison against MLLMs: MRPO improves all three backbones and reaches the highest evaluated-model average on Qwen3-VL-8B-Instruct, rising from 25.61 to 28.94.It achieves the top score on three of five benchmarks and gains 7.05 points over the baseline on RadImageNet-VQA.
  • Cross-backbone Ablation: MRPO achieves the highest average on all three backbones and surpasses GRPO by 0.73, 0.40, and 1.10 points, respectively.SFT improves in-distribution performance but fails to transfer reliably out of distribution, whereas RL methods improve both distributions.
  • First Failure Point Analysis: MRPO reduces early-stage failures from 64.0% to 13.0%, versus 21.2% for GRPO and 21.4% for GDPO, while shifting late-stage failures to 47.0%.The FFP distribution is averaged across the three backbones and groups failures into Early, Mid, and Late-Stage ranges.
  • Failure Accumulation Analysis: MRPO records the lowest FAR from FFP 0.0 to 0.6, reaching 43.3% in the 0.0–0.2 bin versus 64.6% for the baseline, 62.9% for GRPO, and 58.4% for GDPO.Lower FAR indicates more effective recovery after the first failure, especially when failures begin early.

6 Conclusion

In open-ended medical VQA, early-stage reasoning failures systematically propagate and dominate final prediction errors, revealing a key limitation of outcome-centric approaches. MRPO addresses this by integrating step-wise reasoning rewards and assigning exponentially larger penalties to earlier invalid steps when the final answer is incorrect.

  • 6 Conclusion: Early-stage reasoning failures systematically propagate and dominate final prediction errors in open-ended medical VQA.This exposes a key limitation of outcome-centric approaches.
  • 6 Conclusion: MRPO targets the first point of reasoning failure by integrating step-wise reasoning rewards into policy optimization.When the final answer is incorrect, it assigns exponentially larger penalties to earlier invalid steps.

Limitations … B.1 Human–LLM Evaluator Alignment

The paper positions MRPO within medical multimodal reasoning and process-supervision research, while validating GPT-5-mini’s evaluator alignment with human judgments. It also identifies dependencies on external judging, gold rationales, and medical-VQA-only evaluation as limitations.

  • Limitations: MRPO relies on GPT-5-mini for sentence-level process rewards, adding API costs and dependence on an external judge despite its highest performance ceiling among three evaluated reward models.A detailed cost breakdown is provided in Appendix C.3, and the judge comparison is reported in Appendix D.4.
  • Limitations: MRPO’s step-wise rewards depend on gold reasoning annotations from MedThink (Gai et al., 2024), limiting direct extension to settings without annotated rationales.Future extensions would require annotation-free rewards or weaker forms of step-level supervision.
  • Limitations: Evaluation is confined to medical VQA, leaving empirical verification of MRPO’s potential generalization to scientific question answering and legal reasoning for future work.The proposed cascading-failure and step-wise advantage-reshaping mechanisms may apply to other multi-step reasoning domains, but this remains unverified.
  • A.1 Reasoning in Medical Multimodal Large Language Models: Medical multimodal reasoning has progressed from supervised fine-tuning to GRPO-based reinforcement learning, but most approaches still compute sequence-level advantages.Early systems used curated or synthetic medical data, while later methods adapted GRPO following DeepSeek-R1 (Shao et al., 2024).
  • A.2 Process Supervision for Medical Reasoning: Prior medical process-supervision methods evaluate individual reasoning steps, yet aggregate their signals at the sequence level rather than redistributing learning according to failure location.This gap motivates step-aware credit assignment for reliable clinical reasoning (Fan et al., 2025; Zhi et al., 2025; Yun et al., 2025; Zhou et al., 2025).
  • A.3 Step-wise Credit Assignment: General-domain work has begun exploring token-level credit assignment, including step-wise factual verification that selectively rewards supported reasoning and penalizes hallucinated steps.These approaches provide precedent for more selective learning signals across reasoning trajectories (Tan et al., 2025; Parthasarathi et al., 2025; Xie et al., 2025a; Li and Ng, 2025).
  • B.1 Human–LLM Evaluator Alignment: Cohen’s κ exceeds 0.7 for answer correctness on VQA-RAD and SLAKE, indicating substantial alignment between GPT-5-mini and human evaluation.Agreement rates approach 90% across all three benchmarks, while PathVQA’s lower κ is attributed to more severe class imbalance.

B.2 Cross-Judge Evaluation … C.2 Implementation Detail

Cross-judge evaluations show that MRPO’s accuracy gains and reduction of cascading failures persist across GPT-5-mini, GPT-5.4, and Claude-4.5-haiku. The method is trained and evaluated on open-ended medical VQA data using matched RL settings across three multimodal backbones.

  • B.2 Cross-Judge Evaluation: MRPO consistently outperforms GRPO in answer accuracy across all three backbones and all three judges, preserving the same relative ordering despite differing score strictness.Absolute scores vary across judges, but the MRPO-over-GRPO ordering never changes.
  • B.2 Cross-Judge Evaluation: MRPO has the lowest early-stage failure proportion and highest late-stage proportion under every judge, confirming judge-robust mitigation of cascading failures.The baseline-to-GRPO-to-MRPO progression preserves the same monotonic trend across judges.
  • C.1 Dataset: Training uses open-ended instances from VQA-RAD, SLAKE, and PathVQA, aligned one-to-one with MedThink gold reasoning annotations by exact image, question, and answer matching.Binary and multiple-choice questions are excluded from the training data.
  • C.1 Dataset: Evaluation covers 4,263 open-ended in-distribution samples from VQA-RAD, SLAKE, and PathVQA plus five out-of-distribution medical VQA benchmarks.The in-distribution test sets contain 200, 706, and 3,357 samples respectively.
  • C.1 Dataset: The out-of-distribution evaluation includes PMC-VQA, VQA-Med-2021, Quilt-VQA, RadImageNet-VQA, and MIMIC-Ext-MIMIC-CXR-VQA, filtered as specified for open-ended testing.Examples include 2,000 manually verified PMC-VQA samples, 425 single-answer VQA-Med-2021 pairs, and 724 open-ended Quilt-VQA pairs.
  • C.2 Implementation Detail: Experiments use PyTorch on 8×NVIDIA A100 GPUs with FlashAttention-2 and the VLM-R1 GRPO framework, while SFT uses LoRA with gold reasoning annotations.LoRA uses rank 8, alpha 32, dropout 0.05, learning rate 2 × 10^-5, and 3 epochs; Table 6 compares RL training resources.
  • C Experimental Setup: MRPO, GRPO, and GDPO are trained under identical settings on Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct.Each RL method trains for 1 epoch with batch size 64, learning rate 10^-6, and 8 rollouts per prompt.

C.3 Training Cost and Efficiency … D.3 Advantage Reweighting Strategy

MRPO remains practically affordable while its ablations show that performance depends on cold-start choice, position-aware token shaping, and selective advantage reweighting. Across the tested backbones and medical VQA benchmarks, exponential shaping and selective reweighting provide the strongest reported alternatives.

  • C.3 Training Cost and Efficiency: $215.48 total cost makes MRPO only approximately 12% more expensive than GRPO’s $192.96, because longer reasoning traces—not extra judge queries—increase token usage.MRPO, GRPO, and GDPO each use one API call per rollout, totaling roughly 107K calls per epoch across 13K samples and 8 rollouts.
  • D Ablation Study: Ablations vary SFT cold-start initialization, token-level advantage shaping, advantage reweighting, and process reward models on two backbones across three in-distribution and five out-of-distribution benchmarks.The compared process reward models are MedGemma-27B, Med-PRM, and GPT-5-mini under GRPO and MRPO training.
  • D.1 SFT Cold-Start Initialization: For Qwen3-VL-8B-Instruct, SFT followed by MRPO raises PathVQA from 20.43 to 26.57 and SLAKE from 68.27 to 68.41, producing the highest in-distribution average.The same configuration lowers the out-of-distribution average from 28.94 to 25.05, below MRPO without SFT cold-start.
  • D.1 SFT Cold-Start Initialization: SFT cold-start improves in-distribution performance but degrades out-of-distribution performance, consistent with overfitting to gold reasoning patterns in the training distribution.The effect is especially pronounced on PathVQA, whose test distribution most closely aligns with the MedThink gold reasoning annotations.
  • VQA-RAD SLAKE PathVQA PMC-VQA VQA-Med Quilt-VQA Rad-VQA MIMIC-VQA: Table 8 compares uniform, linear, quadratic, and exponential token-level penalties on Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct across in-distribution and out-of-distribution averages.The exponential function is MRPO’s proposed shaping strategy.
  • D.2 Token-Level Shaping Function: Exponential token shaping gains 2.63 points on Qwen2.5-VL-7B-Instruct and 3.90 points on Qwen3-VL-8B-Instruct over uniform shaping, outperforming linear and quadratic alternatives.All position-aware functions outperform uniform shaping, indicating that stronger penalties for earlier failed reasoning steps are beneficial.
  • D.3 Advantage Reweighting Strategy: MRPO reshapes advantages only when the answer reward falls below threshold τ, whereas full and soft reweighting apply reshaping to every instance, scaled by 1.0 and 0.5 respectively.The ablation compares Soft Reweighting, Full Reweighting, and MRPO’s selective strategy; indiscriminate reshaping harms performance.
  • D.3 Advantage Reweighting Strategy: MRPO’s selective reweighting reaches 29.09 on Qwen3-VL-8B-Instruct, exceeding full reweighting at 27.34 and soft reweighting at 27.29 by 1.75 and 1.80 points.On Qwen2.5-VL-7B-Instruct, MRPO reaches 26.79 versus 25.55 for full and 25.00 for soft reweighting.

D.4 Process Reward Model · E Reasoning Analysis

MRPO generally outperforms GRPO across process reward models, with GPT-5-mini providing the strongest average performance and becoming the default judge. The reasoning analysis examines first-failure-stage distributions and failure accumulation across backbones and training methods, while identifying MedGemma-27B as a viable API-free alternative.

  • D.4 Process Reward Model: The process-reward study compares MedGemma-27B, Med-PRM, and GPT-5-mini under both GRPO and MRPO for computing the stepwise reasoning reward Rproc.MedGemma-27B is a local medical multimodal judge, Med-PRM is specialized for medical reasoning, and GPT-5-mini is a general-purpose API-accessed judge.
  • D.4 Process Reward Model: MRPO beats GRPO with all three process reward models on Qwen3-VL-8B-Instruct, gaining 0.12, 0.75, and 0.40 points with MedGemma, Med-PRM, and GPT-5-mini, respectively.The same trend holds on Qwen2.5-VL-7B-Instruct with MedGemma and GPT-5-mini, but not with Med-PRM.
  • D.4 Process Reward Model: GPT-5-mini achieves the highest average across both backbones and training paradigms, reaching 26.79 on Qwen2.5-VL-7B-Instruct and 29.09 on Qwen3-VL-8B-Instruct.These scores surpass MedGemma at 25.49 and 28.26 and Med-PRM at 23.16 and 24.34, respectively.
  • E Reasoning Analysis: The reasoning analysis compares Early, Mid, and Late-Stage First Failure Point distributions for baseline, GRPO, GDPO, and MRPO across three multimodal backbones.Figure 5 covers Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct.
  • E Reasoning Analysis: Failure Accumulation Rate is analyzed across First Failure Point bins for baseline, GRPO, GDPO, and MRPO on Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct.This analysis reports FAR across FFP bins rather than providing additional numerical values in the supplied passage.
  • D.4 Process Reward Model: GPT-5-mini is adopted as MRPO’s default process reward model because its step-level evaluations provide the highest performance ceiling and align with human judgments at Cohen’s κ above 0.7.MedGemma-27B is described as a weaker judge, while Med-PRM evaluates only the question and reasoning text without the image.
  • D.4 Process Reward Model: MedGemma-27B remains a viable alternative when API access is constrained, while developing dedicated medical VQA process reward models without external API dependence is left for future work.Its results approach those of GPT-5-mini.

E.1 Reasoning Analysis Across Backbones … F.1 Stage-wise failure taxonomy

Across three multimodal backbones, MRPO most strongly suppresses early failures and prevents early errors from cascading, while paired analyses show recovery of GRPO errors and characterize distinct early, mid, and late failure types. MRPO’s newly introduced failures are concentrated later, whereas some comparisons involve surface-level answer mismatches rather than genuine reasoning errors.

  • E.1 Reasoning Analysis Across Backbones: MRPO lowers early-stage failures to 8.0%, 14.3%, and 12.8% across Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct, respectively.Across all three backbones, MRPO achieves the largest reduction relative to the baseline and other RL methods.
  • E.1 Reasoning Analysis Across Backbones: MRPO records the lowest earliest-bin FAR—43.9%, 39.0%, and 47.1%—across the three backbones, indicating stronger prevention of cascading failures.These values correspond to the FFP 0.0–0.2 bin and are below every competing method and the baseline.
  • E.2 Paired Comparison of GRPO and MRPO: Paired instance-level evaluation compares GRPO and MRPO on pooled VQA-RAD, SLAKE, and PathVQA test traces across all three backbones.The comparison isolates the effect of MRPO’s reshaped step-wise advantages on individual predictions.
  • E.2 Paired Comparison of GRPO and MRPO: For MRPO-only-correct instances, the transition analysis shows that MRPO reduces the substantial early- and mid-stage failure concentration observed under GRPO.Table 12 indexes GRPO failure stages by rows and MRPO stages by columns.
  • E.2 Paired Comparison of GRPO and MRPO: Some MRPO-only-correct cases have no GRPO failure point because the reasoning is essentially valid and the incorrect answer reflects a surface-level expression mismatch.These cases fall outside the step-level reasoning behavior analyzed in the paired comparison.
  • E.2 Paired Comparison of GRPO and MRPO: For GRPO-only-correct instances, MRPO’s introduced failures concentrate late at 25.0%, versus 18.9% mid-stage and 8.7% early-stage, while 47.4% have no detected failure point.Because GRPO is correct on these instances, the analysis concerns the stage of MRPO’s failure rather than whether it fails.
  • F.1 Stage-wise failure taxonomy: The failure taxonomy groups 300 manually examined traces into early, mid, and late stages at FFP ranges 0.0–0.4, 0.4–0.7, and 0.7–1.0.Traces with similar FFP positions show similar failure patterns, producing largely homogeneous error types within each range.
  • F.1 Stage-wise failure taxonomy: Early failures corrupt later reasoning through incorrect visual premises, mid failures reflect localized interpretive or diagnostic misjudgments, and late failures are local terminal-answer breakdowns that rarely cascade.Examples include default modality assumptions, wrong organ identification, structural misidentification, and non-committal terminal conclusions.

F.2 Qualitative Case Studies

Qualitative paired case studies show that MRPO improves reasoning by correcting incorrect visual premises before they cascade and by recovering from early errors. Its residual failures are typically late-stage local breakdowns rather than cascading reasoning failures.

  • Case 1: Cascade Correction: MRPO corrects GRPO’s incorrect opening visual premise, preventing subsequent reasoning steps from inheriting the error and leading to a wrong answer.The paired traces use identical inputs with every step annotated as valid or invalid, making the divergence point visible.
  • Case 2: Early Recovery: MRPO can recover from an early failure by re-examining the image and reversing the error before it reaches the answer, whereas GRPO commits to the mistaken premise.In the MRI-weighting example, both traces initially settle on T2 but then diverge in their subsequent behavior.
  • Case 3: MRPO Loss: MRPO’s residual losses are typically local late-stage breakdowns after otherwise valid reasoning, rather than cascading errors.The described failure involves a Terminal Label/Term Mismatch when producing the answer.

G RL Training Plots · H Prompts

MRPO’s training dynamics show stronger reasoning-process rewards than GRPO with stable optimization across three multimodal backbones, while the prompts operationalize answer correctness and step-wise reasoning quality through structured binary evaluations. The step-wise evaluator separately checks alignment with gold reasoning and contribution toward the ground-truth answer, covering stage-specific failures and recovery patterns.

  • G RL Training Plots: MRPO’s higher KL divergence reflects stronger step-wise advantage reshaping, but training remains stable rather than diverging across all three backbones.Completion length varies by backbone and has no consistent ordering between MRPO and GRPO, according to the supplied training-dynamics discussion.
  • Answer Correctness Check Prompt: The answer-correctness prompt compares a generated answer with the medical-image question and ground-truth answer, returning only ‘O’ for correct or ‘X’ for incorrect.It treats sufficiently good answers as correct and requires a single-character output.
  • H.2 Step-wise Reasoning Evaluation Prompt: Gold Alignment checks consistency with extracted modality, context, findings, anatomy, laterality, and diagnostic direction, requiring early premise, middle finding, and late diagnostic agreement.Wrong laterality, contradiction, misdirection, unsupported content, omitted pathology, and specificity mismatch receive alignment failure.
  • H.2 Step-wise Reasoning Evaluation Prompt: Answer Contribution marks steps that directly state or support the ground-truth diagnosis, location, or structure, while generic, irrelevant, evasive, or context-only statements receive zero.The criterion is independent of whether a step matches the gold reasoning.
  • H.2 Step-wise Reasoning Evaluation Prompt: The prompt distinguishes early premise failures, mid-stage structural or pathology errors, and late terminal uncertainty or label mismatches that can cause or limit reasoning cascades.Examples include defaulting to an incorrect modality, misidentifying anatomy, omitting a lesion, hedging, or assigning the wrong final term.
  • H.2 Step-wise Reasoning Evaluation Prompt: Case illustrations show MRPO correcting an erroneous premise, recovering after an initial misread, and occasionally losing through a final terminal term mismatch despite intact preceding reasoning.The cases contrast cascade correction, early recovery, and an MRPO loss.
  • G RL Training Plots: Across three backbones, MRPO achieves a clearer advantage over GRPO on reasoning process reward, a slight advantage on answer reward, and stable KL dynamics that rise early before settling.The plots cover answer reward, reasoning process reward, KL divergence, and completion length for Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and InternVL3-8B-Instruct.
Loading 2606.31825v1…