PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing Visual Reasoning in Chest X-Ray Report Generation Using Reinforcement Learning

Denis Musinguzi, Andrew Katumba, Prasenjit Mitra

arXiv:2609.31911v1cs.CV

TL;DR

Chest X-ray report generation remains challenged by hallucinations and training objectives that emphasize lexical similarity, while prior medical report-generation RL checks final outputs rather than reasoning. The paper trains structured regional reasoning with gated spatial and factual rewards, then refines it with RL. Joint verification of reasoning and final reports yields better performance than verifying either alone.

  • Problem

    Hallucinations remain a challenge because SFT rewards lexical similarity rather than grounding findings in image content, and prior report-generation RL checks final outputs but leaves reasoning unverified.

    Source: Introduction · p. 1

  • Method

    The method trains a model to produce region-level reasoning with anatomical names, bounding boxes, and findings, then applies spatial and gated factual rewards to reasoning and final reports.

    Source: Introduction · pp. 1–2, Method · pp. 3–4

  • Results

    Jointly verifying reasoning and final reports during RL leads to substantially better performance than verifying either component in isolation.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    Structured reasoning traces provide region-level descriptions that ground the final report in the chest X-ray image.

    Source: Method · p. 3

    The framework relies on Chest ImaGenome region annotations, currently available only for chest radiographs, limiting direct applicability to other modalities or anatomical regions without comparable annotations.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Medical report generation has made significant progress with the rise of modern vision-language models and the growing availability of large-scale medical datasets. However, hallucinations remain a major challenge, largely due to the limitations of supervised fine-tuning (SFT), which prioritizes lexical similarity to reference reports rather than clinical correctness. While reinforcement learning has shown strong performance in domains with verifiable rewards such as mathematics and code generation, its application to open-ended medical tasks remains limited. Existing work focuses on evaluating final answers, overlooking the model's reasoning, despite evidence that flawed reasoning can degrade overall performance. In this study, we propose a framework that verifies the model's reasoning process by integrating anatomical regions, bounding boxes, and region-level textual descriptions. We design spatial and factual reward mechanisms to ensure that the model's reasoning is both visually grounded and factually accurate. Starting from Qwen3-VL-8B-Instruct as our base model, we adapt it to the medical domain using supervised fine-tuning, introduce reasoning capability through a cold-start SFT stage, and refine it with reinforcement learning. We find that RL provides performance gains beyond those achievable through SFT alone, and that jointly verifying both reasoning steps and final outputs yields larger improvements than verifying either in isolation. We further identify multiple modes of reward hacking in the RL stage. Finally, the model's structured think traces enhance interpretability, making its outputs easier to audit for clinical use.

Introduction

The paper targets hallucinations and unchecked reasoning in chest X-ray report generation, where lexical similarity objectives do not ensure image-grounded clinical correctness. It proposes staged training and gated spatial and factual rewards to verify both reasoning and final reports.

Introduction

  • Chest X-ray report generation could help address radiology workload, but clinical adoption remains limited by model performance and interpretability concerns.Hallucinations include incorrect findings or omitted abnormalities, and can lead to serious misdiagnoses.Source: Introduction · p. 1
  • Prior RL for medical imaging focused on closed-ended tasks, while report-generation RL checked final outputs without verifying the reasoning that produced them.The paper argues that unchecked reasoning can harm interpretability, reliability, and downstream performance.Source: Introduction · p. 1
  • The training overview has three stages: report-generating SFT, cold-start SFT with explicit reasoning, and GRPO reinforcement learning to refine reasoning.The approach starts from Qwen3-VL-8B-Instruct and uses region annotations and descriptions from Chest ImaGenome with final reports from MIMIC-CXR.Source: Figure 1 · p. 2, Introduction · p. 2
  • Gated spatial and factual rewards verify regional reasoning alongside the final report, addressing reward-hacking patterns seen when either reward is used alone.Spatial-only rewards can produce well-localized but duplicated findings, while factual-only rewards can score correct findings on wrong regions below no reward.Source: Introduction · pp. 1–2

Related Work

Prior vision-language RL work has largely targeted closed-ended tasks, while report generation poses an open-ended verification challenge. This work applies gated spatial and factual rewards to make reasoning itself verifiable.

Multimodal Reasoning

  • Open-ended report generation is harder to verify than closed-ended outputs such as labels or bounding boxes, which can be matched against ground truth.Prior multimodal reasoning methods applied RL to closed-ended tasks, including math answers and localization.Source: Multimodal Reasoning · p. 2

VLMs in Radiology

  • Medical VLM research spans SFT report generation and RL for question answering, grounding, and reports, but prior methods apply spatial and factual rewards in isolation.The paper distinguishes its gated combination as jointly verifying reasoning.Source: VLMs in Radiology · pp. 2–3
  • Because reasoning before answering can underperform when left unchecked, the paper applies spatial and factual rewards to regional reasoning before report synthesis.The stated aim is to make the path to an answer as verifiable as the answer itself.Source: VLMs in Radiology · p. 3

Method

The method progresses from medical-domain report SFT to structured reasoning induction and GRPO reinforcement learning. Its reward cascade checks format, localization, and regional factuality before rewarding factual descriptions.

Method

  • The model first receives medical-domain SFT to generate final reports directly using next-token prediction conditioned on visual tokens and the prompt.The target sequence's conditional likelihood is maximized during this stage.Source: Method · p. 3
  • Cold-start SFT adds dictionary-structured traces containing anatomical regions, bounding boxes, and regional findings before the final report is synthesized.The traces are designed to ground reasoning in the chest X-ray image.Source: Method · p. 3
  • GRPO follows cold-start SFT, sampling candidate completions whose rewards determine relative advantages for updating the model.Completions above the group baseline receive positive advantages and underperforming samples receive negative ones; a KL term regularizes drift from the reference model.Source: Method · p. 3
  • A three-check cascade makes valid format a precondition, scores localization by mean IoU across ground-truth regions, and gates regional factual reward on spatial correctness.Regions below the IoU threshold contribute only their spatial term; factuality is measured with RadGraph-F1 using RadGraph-XL.Source: Method · pp. 3–4

Experiments

Experiments use filtered chest X-ray datasets and a three-stage training pipeline, then compare the model with other systems using lexical and clinical-factuality metrics. The supplied passages describe evaluation datasets and table scopes but do not provide table values or comparative outcomes.

Datasets

  • The experiment uses 213,073 filtered MIMIC-CXR image-report pairs, excluding lateral views and reports without Findings and Impressions.References to prior studies are removed because they are not supported by a single image.Source: Datasets · p. 4
  • Chest ImaGenome supplies 642,703 bounding-box annotations across 38 anatomical regions, used to create region-level reasoning traces paired with final reports.Each trace contains a region, bounding box, and clinical-finding phrase, separated from the report with <think> and <answer> tags.Source: Datasets · p. 4

Implementation Details

  • Training proceeds through medical-domain SFT, cold-start SFT for explicit reasoning, and GRPO reinforcement learning.The implementation passages specify training data and optimization settings for each stage, but do not report comparative results.Source: Implementation Details · p. 4

Baselines

  • The model is compared with a general-purpose model and five domain-specific models, with performance also reported after each training stage.The evaluation protocol uses frontal images from filtered MIMIC-CXR and IU X-ray test sets; unavailable baseline weights are reproduced following their publications.Source: Baselines · pp. 4–5

Evaluation Metrics

  • Evaluation combines BLEU-2, ROUGE-L, METEOR, and BERTScore for report quality with RadGraph-F1 for clinical factuality.Metrics are computed with RadEval and reported as means and standard deviations over three seeds.Source: Evaluation Metrics · p. 5

Results

Qwen3-VL-8B-GRPO combines strong clinical factuality with competitive lexical metrics and transfers well to IU X-ray. Across training stages and reward ablations, jointly verifying reasoning and reports performs best, while structured regional traces make claims auditable.

Results

  • Qwen3-VL-8B-GRPO leads clinical factuality (RadGraph-F1 27.9) and METEOR (32.5), while staying competitive on ROUGE-L 28.5, BLEU-2 23.4, and BERTScore 56.3.General-purpose Qwen3-VL-8B-Instruct trails radiology models; UniRG-CXR leads BLEU-2, while RadVLM-GRPO leads ROUGE-L and BERTScore.Source: Results · p. 5
  • GRPO recovers BLEU-2 to 23.4 and produces the pipeline’s largest factuality gain, reaching RadGraph-F1 27.9 after cold-start SFT lowers BLEU-2 to 14.9.SFT provides a lexical baseline of BLEU-2 22.0 and RadGraph-F1 16.9; cold-start SFT raises factuality to 20.4 while shorter reports incur a brevity penalty.Source: Results · p. 5
  • Trained exclusively on MIMIC-CXR, the model leads four of five IU X-ray metrics, while UniRG-CXR scores slightly higher on METEOR.The four metrics are BLEU-2, ROUGE-L, RadGraph-F1, and BERTScore.Source: Results · p. 5
  • The model traverses anatomical regions, generating boxes and findings before synthesizing the final report; bilateral claims can aggregate left- and right-sided observations, making claims traceable to regions.The final report is not simply a concatenation of the reasoning trace.Source: Results · pp. 5–6

Verification of both the final answer and the reasoning process yields the best performance

  • Joint verification performs best on most metrics; reasoning-only verification beats final-answer-only verification but can list regional findings separately, while combined verification reduces omissions and hallucinations.Reasoning-based objectives provide denser supervision for credit assignment and more stable policy optimization.Source: Verification of both the final answer… · p. 6

Sensitivity Analysis of IoU Gating Threshold for the Factual Reward

Spatial and Factual Rewards

  • Combined spatial and factual rewards perform best; spatial-only rewards can duplicate findings, while factual-only rewards permit wrong region-text associations and underperform no reward.Spatial supervision alone is competitive with the full formulation, whereas the factual reward promotes faithful descriptions once the correct region is identified.Source: RL Enhances Fine-Grained Understanding… · p. 6, Spatial and Factual Rewards · p. 7

Limitations

  • The framework depends on chest-radiograph region annotations, adds rollout-heavy RL cost, and may still exploit reward imperfections; adversarial robustness remains unevaluated.RL requires multiple sampled rollouts per training example.Source: Limitations · p. 7

Conclusion

The proposed RL methodology verifies both the model’s reasoning and final report with spatial and factual rewards. Joint spatial and factual supervision supports stronger performance, clinical faithfulness, and more auditable outputs.

  • Jointly verifying reasoning and final reports during RL leads to substantially better performance than verifying either component alone.Source: Conclusion · p. 7
  • Reasoning verification tracks bounding-box coordinates together with regional findings to prevent identical findings across different regions.Source: Conclusion · p. 7
  • Structured think-traces make model behavior more interpretable and auditable for clinicians.Source: Conclusion · p. 7
Loading 2609.31911v1…