PaperlayerView exact revision on arXiv

Source-linked AI summary

CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

Chandravardhan Singh Raghaw, Shahid Shafi Dar, Arnav Nishith Sharma, Shrey Modi, Nagendra Kumar

arXiv:2610.13059v1cs.CL

TL;DR

Manual radiology reporting is time-consuming, while automated systems can produce lengthy narratives that miss diagnostic and multi-view context. CliQ-RRG aligns images and text using predicted disease labels, integrates prior scans, and adds graph-grounded clinical evidence before QA restructuring; it outperforms state-of-the-art methods on two CXR benchmarks.

  • Problem

    Manual CXR interpretation is time-consuming, and existing RRG methods can overlook broader context, diagnostic-label supervision, and prior-scan information.

    Source: §1 · pp. 1–2

  • Method

    CliQ-RRG aligns visual and textual representations using predicted disease labels, incorporates prior scans through attention, and uses graph-guided retrieval and an LLM to produce QA-style reports.

    Source: §1 · p. 2, §2.1 · p. 2, §2.2 · p. 3

  • Results

    CliQ-RRG achieves state-of-the-art performance across most metrics on MIMIC-CXR and IU X-Ray, with stronger lexical and clinical alignment than contrastive methods.

    Source: §4 · p. 7

  • Takeaways & Limitations

    CliQ-RRG generates clinically reliable QA-style reports that align visual, textual, and diagnostic-label modalities on the evaluated benchmarks.

    Source: §6 · p. 9

    Training is limited to MIMIC-CXR and IU X-Ray, which may constrain learned representations to their acquisition protocols and population characteristics.

    Source: §7 · p. 9

Abstract

from arXiv · show

Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi-view information. We present Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation (CliQ-RRG), a unified two-stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ-RRG employs a Disease-aware Visual-Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior-Guided Attention Module (PrAM) to capture multi-view dependencies across current and prior scans. In Stage 2, GraphRAG-guided retrieval augments intermediate textual representations with graph-grounded clinical evidence, and a large language model restructures them into concise, interpretable question-answer (QA) pairs with diagnostic summaries. Experiments on two public chest X-ray benchmarks demonstrate that CliQ-RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA-style radiology reports. Code is available at: https://github.com/chandravardhan-singh-raghaw/CliQ-RRG.

1 Introduction

Radiology report generation could ease the time-consuming, cognitively demanding work of CXR interpretation, but existing approaches leave reports lengthy and miss important contextual and diagnostic signals. CliQ-RRG combines disease-aware visual–textual alignment, prior-scan attention, and graph-guided QA generation to address these gaps.

1 Introduction

  • Manual CXR interpretation is time-consuming and cognitively demanding, while many automated RRG methods emphasize disease-specific regions over broader context.The paper frames automated RRG as a promising alternative as imaging volumes rise.Source: §1 · p. 1
  • Prior work includes lengthy, less interpretable outputs, clinically incoherent retrieved text, inefficient alignment, and limited modeling of cross-scan anatomical context.The cited approaches include graph-enhanced regional features, clinical retrieval, cross-modal adapters, and dual-view methods.Source: §1 · p. 1
  • Existing RRG methods can produce long free-text reports, underspecify subtle findings, omit diagnostic-label supervision, and fail to fuse prior scans at the visual-representation level.The paper proposes QA reformulation, intermediate clinical-knowledge injection, label-informed contrastive alignment, and prior-image fusion as responses to these gaps.Source: §1 · p. 2
  • CliQ-RRG consistently outperforms state-of-the-art methods on MIMIC-CXR and IU X-Ray, according to the reported experiments.The framework produces QA-style outputs enriched with clinical knowledge and prior-examination context.Source: §1 · p. 2
  • The framework combines disease-aware visual–textual alignment with prior-guided attention and graph-grounded evidence retrieval followed by LLM-based QA restructuring.The alignment uses predicted disease embeddings; the attention integrates multi-view context, while retrieval enriches intermediate reports with clinical evidence.Source: §1 · p. 2

2 Proposed Approach

CliQ-RRG first aligns chest X-ray and report representations using disease-label supervision and prior scans, then augments an intermediate report with graph-retrieved clinical evidence and restructures it as QA pairs with a summary. The stages are trained jointly with alignment, disease-classification, and report-generation objectives.

2.1 Problem Formulation and Overview

  • CliQ-RRG takes current and prior chest X-rays, associated reports, and predicted disease labels through two stages to produce QA-style reports with a clinical summary.Stage 1 aligns visual and textual representations; Stage 2 adds graph-grounded evidence and uses an LLM to restructure the report.Source: §2.1 · p. 2

2.2 STAGE 1: Disease-aware Visual–Textual Alignment

  • PrAM uses current-image features as an attention query and prior-scan features as keys and values to fuse multi-view context for contrastive alignment.It captures historical context but does not explicitly encode scan order or intervals, so it does not explicitly model longitudinal progression.Source: §2.2 · p. 3
  • Disease-aware alignment uses independently predicted four-state labels for 13 disease categories to supervise visual–textual contrast, pulling label-similar pairs together and separating unrelated pairs.Di-VTC applies bidirectional contrastive learning, using similarity in predicted disease embeddings as supervision.Source: §2.2 · p. 4

2.3 STAGE 2: Knowledge-Guided QA-Style Report

  • GraphRAG grounds the intermediate report and predicted disease labels to graph entities, then retrieves connected subgraphs and multi-hop relation paths as clinical evidence.This retrieves connected evidence rather than isolated knowledge fragments.Source: §2.3 · p. 5
  • A fine-tuned LLM restructures the graph-enhanced report into clinically meaningful question–answer pairs followed by a concise diagnostic summary.Each answer is Yes or No, and the output includes a summary of key findings.Source: §2.3 · p. 5

2.4 Learning Objective

  • Joint training combines disease-aware visual–textual contrast, intermediate report generation, and disease classification losses, with the generative and classification objectives equally weighted.The coefficients for the report-generation and classification losses are each set to 1.0.Source: §2.4 · p. 5

3 Experimental Setting

The study evaluates CliQ-RRG on MIMIC-CXR and IU X-Ray using narrative-report and QA-pair metrics, with comparisons against prior methods. It also reports the benchmark table’s method categories and reproduced-result conventions, alongside implementation settings.

3 Experimental Setting

  • MIMIC-CXR uses 377,110 scans and 227,827 reports with official splits; IU X-Ray uses 7,470 scans and 3,955 reports with 7:2:1 splits.MIMIC-CXR is described as the largest RRG dataset; IU X-Ray includes frontal and lateral scans.Source: §3 · p. 5
  • Narrative reports are assessed with BLEU, METEOR, ROUGE-L, CIDEr, BERTScore, and CheXbert clinical-efficacy metrics; QA pairs use BERTScore, CheXbert-F1, and RadGraph-F1.Inter-annotator agreement is measured with pairwise agreement and Fleiss’ kappa.Source: §3 · pp. 5–6
  • Table 1 compares CliQ-RRG with state-of-the-art methods on both datasets, grouping methods into seven categories and marking published versus reproduced results.Reproduced methods use the same test set for fair comparison.Source: Table 1 · p. 6, §3 · p. 6
  • The framework is implemented in TensorFlow and trained for 30 epochs on a 64 GB NVIDIA GPU using AdamW, batch size 16, learning rate 3e−5, and weight decay 0.01.Additional implementation details are in Appendix E.Source: §3 · p. 6

4 Result and Discussion

CliQ-RRG is reported to outperform prior methods across most benchmark metrics, with additional evaluations of clinical efficacy, QA-pair quality, component contributions, and zero-shot generalization. Expert review and qualitative analyses further assess the generated reports.

Ablation Study

  • On MIMIC-CXR, the full graph-guided model improves NLG by 29.0% and clinical efficacy by 31.9% over the base configuration.The ablation reports mean ± standard deviation over five runs and statistically significant gains for model (g), with p ≤0.05; adding PrAM, label-aware Di-VTC, and Gmed builds toward the full model.Source: Table 3 · p. 7, §4 · p. 7

Comparison with SOTA

  • CliQ-RRG achieves state-of-the-art performance across most metrics on MIMIC-CXR and IU X-Ray, outperforming multi-view, QA-style, recent RRG, and MLLM-based methods.RADAR remains competitive; REVTAF leads METEOR and is second on ROUGE-L on MIMIC-CXR, while DAD-RG performs strongly on IU X-Ray.Source: §4 · p. 7

CE Evaluation

  • On MIMIC-CXR, CliQ-RRG outperforms REVTAF in clinical-efficacy Recall and F1.The evaluation also confirms the disease classifier’s effectiveness in guiding multimodal alignment.Source: §4 · p. 7

QA-pair Evaluation

  • QA-pair evaluation reports strong BERTScore, CheXbert-F1, and RadGraph-F1, while the authors caution that QA scores are not directly comparable with narrative-report scores.Expert evaluation is used as the primary evidence of clinical validity for generated QA pairs.Source: §4 · pp. 6–7, Table 4 · p. 7

Qualitative Analysis

  • For a MIMIC-CXR example, adding PrAM yields more precise findings, disease-aware contrast identifies key observations, and Grad-CAM highlights relevant anatomical and pathological regions.The reported progression moves from a generic base-model statement toward clinically detailed findings and diagnostic grounding.Source: §4 · pp. 7–8, Figure 4 · p. 8

Parameter Sensitivity Analysis

  • Performance peaks at λ1=λ2=1.0 and τ=0.07 in the reported MIMIC-CXR sensitivity analysis.The authors interpret equal generation and classification loss weights as balancing those objectives.Source: Figure 5 · p. 8, §4 · p. 8

Expert Evaluation

  • Across 500 reviewed QA pairs, 99.6% were clinically reliable and 0.4% were hallucinated.A senior radiologist classified 467 pairs as fully acceptable, 31 as clinically acceptable, and 2 as hallucinated; a separate evaluation used Likert ratings from three medical professionals.Source: §4 · pp. 8–9, Table 5 · p. 9

Generalizability Analysis

  • Zero-shot evaluation on ReXGradient-160K found that semantic and clinical metrics remained relatively stable despite a moderate decline in surface-level metrics.MIMIC-CXR-trained models consistently outperformed IU X-Ray-trained models; extension beyond chest X-rays to other anatomies and modalities remains future work.Source: Table 6 · p. 9, Table 7 · p. 9, §4 · p. 9

5 Related Work

The related-work discussion is placed in Appendix A because of space constraints.

5 Related Work

  • Related work is provided in Appendix A due to space constraints in the main text.Source: §5 · p. 9

6 Conclusion

CliQ-RRG combines disease-aware visual–textual alignment, prior-guided attention, graph-guided clinical evidence, and QA generation. The paper reports superiority over state-of-the-art methods on MIMIC-CXR and IU X-Ray.

6 Conclusion

  • CliQ-RRG combines disease-aware visual–textual alignment and prior-guided attention with graph-guided retrieval to generate clinically reliable QA-style radiology reports.Experiments on MIMIC-CXR and IU X-Ray highlight the framework’s superiority over state-of-the-art methods.Source: §6 · p. 9

7 Limitations

The paper identifies limitations in modality and dataset scope, LLM optimization, temporal modeling, and supervision requirements.

7 Limitations

  • Training is limited to MIMIC-CXR and IU X-Ray, potentially constraining learned representations to their acquisition protocols and population characteristics.The study also focuses on 2D chest X-rays, rather than modalities with richer spatial structure.Source: §7 · p. 9
  • PrAM incorporates historical scans but does not encode temporal order or examination intervals, so it does not directly model longitudinal progression patterns.The LLM is a fixed restructuring component, not jointly optimized for multimodal reasoning.Source: §7 · p. 9
  • The framework relies on supervised paired images, reports, and diagnostic labels, which may constrain scalability in data-scarce settings.The authors identify this as a direction for future improvement.Source: §7 · p. 9

Appendix

  • Appendix contents include knowledge-base construction, graph-guided retrieval, ablations, LLM choice, disease classification, sensitivity, expert evaluation, efficiency, practical implications, and future work.The contents list also includes implementation details and reproducibility.Source: Appendix · p. 13

A Related Work

  • Prior work spans alignment, knowledge injection, and QA formatting, while the authors identify longitudinal priors for structured reporting as a gap addressed by CliQ-RRG.They describe their approach as unifying disease-aware contrastive alignment with staged knowledge retrieval and LLM-based QA restructuring.Source: A Related Work · p. 14

B Compared Baselines

  • CliQ-RRG is benchmarked against state-of-the-art baselines classified into seven categories.The supplied baseline passage introduces the comparison but does not detail its categories.Source: B Compared Baselines · p. 14

I: Contrastive-based Methods

The related methods cover contrastive alignment, clinical knowledge injection, LLMs, multiple views, QA generation, retrieval, and multimodal foundation models. The accompanying method details describe disease-linked knowledge retrieval and graph-based evidence integration.

I: Contrastive-based Methods

  • R2GenRL uses generation-metric reinforcement learning, while DCL, MAN, CoFE, SEI, and DART use distinct strategies to align visual and textual information.The described strategies include relation graphs, shared memory, factual-pair contrastive learning, entity-region alignment, and diagnostic-label grounding [Qin and Song, 2022; Li et al., 2023; Shen et al., 2024; Li et al., 2024a; Liu et al., 2024b; Park et al., 2025].Source: §I · p. 14

II: Knowledge Injected-based Methods

  • KiUT integrates a symptom graph into a U-Transformer, and EKAGen combines expert knowledge with attention to focus on pathological regions [Huang et al., 2023; Bu et al., 2024a].PromptMRG uses disease predictions as soft prompts and retrieves similar reports; RADAR injects external clinical knowledge; REVTAF uses a hyperbolic semantic hierarchy to retrieve reference reports [Jin et al., 2024; Hou et al., 2025; Zhou et al., 2025].Source: §II · p. 14

III: LLM-based Methods

  • LLM-based methods include visual-prompt instruction following, cyclic patch–word alignment, knowledge-graph generation, adaptive token fusion, and dual-attention report generation [Liu et al., 2024a; Chen et al., 2024; Li et al., 2024b; Wang et al., 2025; Wu et al., 2026].These are the methods and mechanisms listed in the compared-baseline passage.Source: §III · p. 14

IV: Multi View-based Methods

  • Multi-view methods include preference-adapted reporting, historical constraints for progression-aware generation, and graph-prioritized organ-specific descriptions [Xiao et al., 2025; Liu et al., 2025b; Gu et al., 2025].The passage associates these approaches with MPO, HC-LLM, and ORID, respectively.Source: §IV · p. 14

V: QA Style-based Method

  • CoD uses diagnostic conversations to generate QA pairs that guide a large language model during radiology report generation [Jin et al., 2025].Source: §V · p. 14

VI: RAG-based Method

  • RA-RRG retrieves image-consistent clinical phrases after extracting clinically essential key phrases, with the stated aim of suppressing hallucinations [Park et al., 2026].Source: §VI · p. 15

VII: MLLM-based Methods

  • MAIRA-2 generates grounded and non-grounded chest X-ray reports from current and prior images, reports, and clinical context; MedGemma is a medical vision-language foundation model [Bannur et al., 2024; Sellergren et al., 2025].The passage notes MedGemma’s performance across medical VQA and chest X-ray report generation.Source: §VII · p. 15

C Medical Knowledge Base Construction

  • The knowledge base is built from approximately 5,000 PubMed abstracts and related thoracic-imaging documents, with normalized clinical facts associated with disease labels and embedded using BioWordVec.BioClinicalBERT-based NER identifies entities, and cosine-similarity thresholding filters near-duplicate facts.Source: C Medical Knowledge Base Construction · p. 15

E Implementation Details

Implementation details specify the image input, model dimensions, graph retrieval settings, and QA fine-tuning configuration.

E Implementation Details

  • Images are resized and cropped to 224 × 224; Stage 1 projections use dimension 768, and Stage 2 uses an eight-head Transformer decoder with hidden size 256.Graph retrieval uses 10 seeds and two-hop traversal; QA restructuring fine-tunes Llama-3-8B-Instruct with QLoRA for three epochs.Source: E Implementation Details · p. 16

F Ablation Analysis on IU Dataset

CliQ-RRG’s component analyses report gains from disease-aware alignment, multi-view context, graph-guided retrieval, and QA restructuring. Further analyses examine retrieval choices, disease-state classification, and sensitivity to label noise.

F Ablation Analysis on IU Dataset

  • The full graph-guided framework achieves the best performance (+27.1%), with observations consistent on MIMIC-CXR.On IU X-Ray, adding PrAM yields an 8.2% gain, and Di-VTC improves B1 to 0.498 from the base model’s 0.435.Source: F Ablation Analysis on IU Dataset · p. 16, Table 8 · p. 16

G Impact of LLMs on QA Restructuring

H Impact of Graph-retrieval

  • Two-hop retrieval performs best; removing retrieval seed initialization lowers CheXbert-F1 from 0.610 to 0.591, while removing graph serialization yields RG-L of 0.299.One-hop retrieval limits evidence coverage, while three-hop retrieval introduces irrelevant anatomical associations.Source: H Impact of Graph-retrieval · p. 16, Table 9 · p. 17

I Per-State Disease Classification

  • The disease classifier reaches a macro-average F1 of 0.505 across 13 categories, with uncertain-state classification comparatively more challenging.The four-state setup distinguishes positive, negative, uncertain, and not-mentioned labels rather than reducing prediction to binary classification.Source: I Per-State Disease Classification · p. 17, Table 10 · p. 17

J Noise-injection Sensitivity

K Expanded Expert Evaluation

An expanded expert assessment examined 500 sampled MIMIC-CXR QA pairs and found few hallucinated findings, alongside substantial inter-rater agreement.

K Expanded Expert Evaluation

  • Only two of 500 evaluated QA pairs (0.4%) contained hallucinated findings; pairwise agreement was 0.83 and Fleiss’ κ was 0.77.The stratified sample covered 13 CheXbert disease categories, balanced positive, negative, and uncertain findings, and was grouped by difficulty.Source: K Expanded Expert Evaluation · p. 18, Table 13 · p. 18, Table 12 · p. 18

L Computational Efficiency

CliQ-RRG reports inference costs for its two-stage framework, describes intended clinical workflow benefits, and identifies broader evaluation and modeling directions for future work.

L Computational Efficiency

  • The framework has approximately 170M trainable parameters and averages 180–300 ms per study before QA restructuring, which adds 1.35–2.75 s.Measurements are reported on a single 64 GB GPU; inference time excludes final QA restructuring.Source: L Computational Efficiency · p. 18, Table 14 · p. 18

M Practical Implication

  • The QA-style format is intended to make reports easier to read, review, and query, while prior examinations support longitudinal assessment.The paper also describes disease-aware evidence and clinical knowledge as supporting consistency and contextual coverage.Source: M Practical Implication · p. 18

N Future Work

  • Future work includes evaluation on additional modalities and broader multi-center datasets, prospective assessment, and explicit temporal modeling.The authors also propose open-set VQA, interactive multimodal question answering, and semi- or weakly supervised learning.Source: N Future Work · p. 18
Loading 2610.13059v1…