Source-linked AI summary
Pre-training, Reasoning, Benchmarking: X-ray Report Generation on CheXpert Plus Dataset
Xiao Wang, Yuxiang Zhang, Dan Xu, Yuehang Li, Shiao Wang, Bo Jiang, Yaowei Wang, Yonghong Tian, Jin Tang
TL;DR
CheXpert Plus was released without reproducible baseline implementations, leaving a gap in standardized comparison of X-ray report-generation methods. The paper establishes a benchmark and proposes MambaXray-PRB, a staged pre-training and multimodal reasoning framework. Experiments across three datasets report competitive performance and improvements in several clinically oriented metrics.
Problem
CheXpert Plus lacks reproducible baseline implementations, while existing X-ray report-generation methods face efficiency, paired-data, and explicit visual-grounded reasoning challenges.
Source: §1 · p. 2Method
The paper establishes CXPMRG-Bench and proposes MambaXray-PRB, combining autoregressive X-ray pre-training, image-report contrastive learning, and multimodal Chain-of-Thought post-training.
Source: §1 · pp. 2–3Results
Experiments on IU X-ray, MIMIC-CXR, and CheXpert Plus show competitive report-generation performance and improvements in several clinically oriented metrics.
Source: §6 · p. 15Takeaways & Limitations
CXPMRG-Bench provides baseline references and standardized comparison criteria for subsequent X-ray report-generation research on CheXpert Plus.
Source: §1 · pp. 2–3The reasoning traces and localized evidence are automatically constructed and may inherit errors, and experiments are limited to publicly available chest X-ray datasets.
Source: §5.7 · p. 15
Abstract
from arXiv · showhide
X-ray image-based Radiology Report Generation (RRG) constitutes a critical research direction within medical artificial intelligence, with great potential to alleviate clinicians' diagnostic workload and shorten patient waiting periods. Despite substantial advances over recent years, the field faces evident bottlenecks stemming from insufficient standardized benchmarks and inadequate domain adaptation of generic large models. Notably, the newly released CheXpert Plus dataset is provided without accompanying baseline implementations and evaluation results, which impedes standardized training, quantitative evaluation and fair comparison among follow-up algorithms. To mitigate this limitation, we establish a comprehensive benchmark encompassing prevailing X-ray report generation models and Large Language Models on CheXpert Plus. This benchmark delivers a reliable comparative foundation for upcoming methods and enables researchers to rapidly identify state-of-the-art approaches within this domain. Beyond benchmark construction, we rethink X-ray RRG under the paradigm of large models and propose a novel framework termed MambaXray-PRB. Our framework improves report generation performance and enhances model interpretability via multi-stage large-model pre-training and multi-modal Chain-of-Thought reasoning. The pipeline consists of three successive phases: self-supervised auto-regressive modeling, X-ray-report contrastive learning, and post-training optimization for reasoning and report generation. Extensive experiments on IU X-ray, MIMIC-CXR, and CheXpert Plus datasets validate the effectiveness of MambaXray-PRB for radiology report generation. The source code of this paper is available on https://github.com/Event-AHU/Medical_Image_Analysis
1 INTRODUCTION
The introduction identifies both an evaluation gap on CheXpert Plus and methodological challenges in X-ray report generation. It presents CXPMRG-Bench and MambaXray-PRB as complementary contributions, with the latter combining staged pre-training and multimodal reasoning.
1 INTRODUCTION
- The introduction notes that IU X-ray is relatively small, while CheXpert Plus lacks reproducible baseline implementations for fair follow-up comparisons.The paper also broadens its experiments to IU X-ray and MIMIC-CXR and introduces the GREEN metric alongside traditional metrics.Source: §1 · pp. 1–3
- The authors identify high Transformer vision-backbone cost, limited paired-data scale, and limited explicit visual-grounded reasoning as challenges for X-ray report generation.They pose the need for an efficient framework that can use abundant unlabeled X-rays while supporting interpretable, visually grounded reasoning.Source: §1 · p. 2
- MambaXray-PRB combines self-supervised autoregressive X-ray modeling, X-ray–report contrastive learning, and multimodal Chain-of-Thought post-training for report generation.The framework uses Mamba as its vision encoder and a large language model as its decoder; the authors state that it makes visual-linguistic reasoning explicit.Source: §1 · p. 2
- CXPMRG-Bench evaluates 30 mainstream algorithms, 16 large language models, and 9 vision-language models on CheXpert Plus.The authors describe it as a systematic benchmark providing baseline references and standardized comparison criteria for follow-up research.Source: §1 · pp. 2–3, Figure 1 · p. 2
- The paper presents this version as extending its CVPR 2025 predecessor with revised methods, a larger benchmark, and expanded analyses.The listed extensions include a multimodal Chain-of-Thought mechanism intended to improve interpretability and additional experimental validation.Source: §1 · p. 3
2 RELATED WORK
The related-work review covers report-generation methods, multimodal Chain-of-Thought, foundation models, and state-space models. It positions this work around radiology-specific reasoning grounded in disease-relevant image evidence and Mamba-based visual representation learning.
2.1 Radiology Report Generation
- Recent report-generation methods explore temporal modeling, clinical knowledge, retrieval, visual grounding, and alignment, while explicit use of disease-relevant evidence in diagnostic reasoning remains underexplored.This framework incorporates localized disease-relevant image evidence into multimodal reasoning supervision, connecting evidence, reasoning, and report generation.Source: §2.1 · p. 3
2.2 Chain-of-Thought Reasoning
- Prior multimodal Chain-of-Thought methods address varied vision-language tasks; this work instead constructs radiology-specific reasoning supervision linked to disease-relevant image patches.The stated aim is to support evidence-grounded report generation.Source: §2.2 · p. 3
2.3 Large Foundation Models
- Medical and multimodal foundation models increasingly unify heterogeneous inputs, and this work adapts visual and language models through radiographic representation learning, alignment, and evidence-grounded reasoning.The reviewed examples include models handling medical text, 2D and 3D images, video, and other modalities.Source: §2.3 · p. 4
2.4 State Space Model
- Mamba addresses Transformer self-attention's quadratic complexity with linear-complexity sequence modeling, motivating its use for autoregressive X-ray representation learning.The paper describes aligning the learned visual features with clinical semantics and coupling the visual encoder with an LLM for evidence-grounded reasoning and report generation.Source: §2.4 · p. 4
3 METHODOLOGY
MambaXray-PRB progresses from self-supervised visual pre-training to image-report alignment and evidence-grounded reasoning and report generation. It builds reasoning data by linking report-derived chains of thought to disease-localized image patches, then post-trains a model for reasoning and reporting.
3.1 Overview
- The framework progresses from visual structure modeling through image-report semantic alignment to evidence-grounded clinical reasoning and report generation.A Mamba encoder first learns from unpaired X-rays; paired reports are introduced for contrastive alignment before multimodal reasoning post-training.Source: §3.1 · p. 4
3.2 Pre-Training Stage
- Stage I trains a Mamba visual encoder by autoregressively predicting image patches; Stage II contrastively aligns its representations with paired medical reports.The Stage-I encoder initializes the task-specific model used in Stage II.Source: §3.2 · pp. 4–5
3.3 Multimodal Reasoning Generation
- Report-derived CoT follows five reasoning steps, while disease-specific activation thresholds select image patches that are linked to diagnostic mentions in the resulting sequence.The linked patch evidence lets downstream models associate reasoning steps with spatially grounded image evidence.Source: §3.3 · pp. 5–6
3.4 Post-Training Stage
- Post-training jointly optimizes CoT and report generation, and prompt engineering enables CoT-only, report-only, or rationale-plus-evidence outputs from one model.Visual tokens and prompt embeddings are processed autoregressively, with separate losses for reasoning and report tokens.Source: §3.4 · pp. 6–7
4 CHEXPERT PLUS BENCHMARK
The CheXpert Plus benchmark compares mainstream report-generation models and expands evaluation across language models and vision-language models. Reproductions use full reference reports under a unified evaluation protocol.
4.1 Mainstream X-ray Report Generation Models
- The benchmark compares mainstream models spanning CNN-, Transformer-, and Mamba-based visual encoders and recurrent, Transformer, or large-language-model decoders.These baselines provide a broad comparison with MambaXray-PRB on CheXpert Plus.Source: §4.1 · p. 7
- Reproduced methods are evaluated against complete reference reports, without truncation, under a unified protocol.Some original implementations truncated ground-truth reports, potentially making comparisons inconsistent.Source: §4.1 · p. 7
4.2 LLMs and VLMs
- The expanded benchmark covers 25 open-source foundation models: 16 LLMs and 9 VLMs.The LLM comparison adds DeepSeek-R1-Distill-Llama-8B and Phi-4-mini-instruct to the conference-version models.Source: §4.2 · p. 7
- For controlled LLM comparisons, each model replaces R2GenGPT’s Llama2 decoder while the Swin Transformer visual encoder stays unchanged.The VLM benchmark grows from InternVL-2 and MiniCPM-V2.5 to nine models, including Qwen3-VL-8B-Instruct and Qwen3.5-9B.Source: §4.2 · pp. 7–8
- Four of the nine evaluated VLMs are medical-domain models, broadening comparison across language-only, general-purpose VLM, and medically specialized VLM approaches.The medical-domain models are MedGemma-1.5-4B-IT, Hulu-Med-7B, Lingshu-7B, and UniRG-CXR.Source: §4.2 · p. 8
5 EXPERIMENTS
Experiments evaluate MambaXray-PRB on IU X-ray, MIMIC-CXR, and CheXpert Plus using language-generation and clinical metrics. The model is competitive across datasets, while parameter analyses and ablations examine reasoning length, patch budgets, and training components.
5.1 Datasets and Evaluation Metrics
- The evaluations cover MIMIC-CXR, IU X-ray, and CheXpert Plus, using lexical metrics alongside clinical efficacy measures and GREEN error analysis.CheXpert Plus uses Findings as the target report; its splits contain 40,441 training, 5,780 validation, and 11,562 test samples. GREEN characterizes clinically significant errors and matched findings.Source: §5.1 · pp. 8–9, Table 2 · p. 9
5.2 Implementation Details
- Pre-training combines autoregressive visual learning with contrastive alignment on 480,000 image–text pairs, followed by report-generation and reasoning fine-tuning.Downstream training generally freezes the language model and optimizes the visual encoder and mapping layer; MIMIC-CXR Large is the exception, using LoRA adaptation.Source: §5.2 · p. 9
5.3 Comparison on Public Benchmark Datasets
- On MIMIC-CXR, MambaXray-PRB-B raises CIDEr from 0.206 to 0.231 versus MambaXray-VL-Base, while IU X-ray results show modest gains over the conference version.On IU X-ray, Base improves BLEU-1/2/3/4 from 0.479/0.322/0.236/0.179 to 0.492/0.332/0.243/0.187; Large improves BLEU-2/3/4 and slightly improves ROUGE-L and CIDEr.Source: §5.3 · pp. 10–11
- On CheXpert Plus, MambaXray-PRB-B records 3.428 clinically significant errors and 1.805 matched findings, while MambaXray-PRB-L achieves the best F1 score of 0.338.The Base model’s GREEN score is 0.272, second to AM-MRG’s 0.281; errors in categories (c)–(f) remain higher than for several competitors.Source: Table 2 · p. 9, §5.3 · pp. 11–12
5.4 Parameter Analysis
- A 300-token CoT limit yields the highest clinical precision, recall, and F1, while 12 patch indices per disease outperform budgets of 6 or 18.At 300 tokens, precision/recall/F1 are 0.341/0.286/0.300; increasing the patch budget from 6 to 12 raises them from 0.339/0.281/0.295 to 0.341/0.286/0.300, while 18 lowers them to 0.329/0.275/0.290.Source: §5.4 · p. 12, Table 6 · p. 12, Table 7 · p. 12
5.5 Ablation Study
- The complete ablation configuration performs best across reported metrics, with autoregressive pre-training, contrastive alignment, joint CoT generation, and patch-aware reasoning contributing complementary gains.Patch-aware MM-CoT raises precision, recall, and F1 from 0.342/0.264/0.278 to 0.359/0.285/0.303 versus textual CoT; the ablation also reports gains from autoregressive pre-training and contrastive learning over their respective alternatives.Source: §5.5 · pp. 13–14, Table 8 · p. 13
5.6 Visualization
- Qualitative examples associate clinical findings in generated reports with spatially relevant image patches, while comparisons with R2GenGPT show stronger reference agreement on several findings.The visualizations are qualitative evidence of cross-modal correspondence, not precise lesion segmentation; both models sometimes produce unsupported findings or imprecise localization.Source: §5.6 · pp. 14–15, Figure 5 · p. 14, Figure 6 · p. 14
5.7 Limitation Analysis
- The reasoning supervision may inherit errors from its LLM and auxiliary disease recognizer, and grounding is limited to predefined thoracic abnormalities and discrete patch indices.Experiments use public chest X-ray datasets; the authors call for cross-institution and patient-population evaluation and radiologist assessment.Source: §5.7 · p. 15
6 CONCLUSION AND FUTURE WORKS
MambaXray-PRB combines large-scale X-ray pre-training, cross-modal alignment, and evidence-grounded reasoning, with experiments reporting competitive performance and improvements in clinically oriented metrics. Future work targets stronger reasoning supervision, visual grounding, and evaluation of robustness and clinical applicability.
6 CONCLUSION AND FUTURE WORKS
- MambaXray-PRB combines auto-regressive X-ray visual pre-training, X-ray–report contrastive learning, and multimodal Chain-of-Thought post-training, linking disease-relevant image patches to reasoning and generated reports.The paper also establishes a CheXpert Plus benchmark to evaluate representative report-generation methods and foundation models.Source: §6 · p. 15
- Experiments on IU X-ray, MIMIC-CXR, and CheXpert Plus show competitive report generation performance and improvements in several clinically oriented metrics; ablations validate complementary component contributions.The conclusion does not specify individual metric values.Source: §6 · p. 15
- Future work includes expert-verified reasoning supervision, finer-grained visual grounding, cross-institutional and radiologist-centered evaluation, extension to other imaging modalities, and more efficient adaptive reasoning.The authors identify cross-institutional validation and radiologist-centered evaluation as important for assessing robustness and clinical applicability.Source: §6 · p. 15