PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Structured Radiology Report Generation with Rich Clinical Context

Seongjae Kang, Dong Bok Lee, Juho Jung, Dongseop Kim, Won Hwa Kim, Sunghoon Joo

arXiv:2510.00428v1cs.LGcs.AI

TL;DR

Existing structured radiology report-generation systems often omit clinical context that radiologists use, including prior studies, leaving them limited in representing patient-specific information and longitudinal changes. C-SRRG integrates multiple context sources into structured report generation; evaluation reports consistent quality improvements and fewer temporal hallucinations.

  • Problem

    Existing report-generation frameworks often omit clinical context such as indications and prior studies, despite its role in capturing patient-specific properties and longitudinal changes.

    Source: §1 · p. 2

  • Method

    C-SRRG combines structured report generation with multi-view images, clinical indication, imaging technique, and variable-length prior studies with comparisons.

    Source: §1 · p. 2

  • Results

    Across evaluations, incorporating clinical context consistently improves report quality and reduces temporal hallucinations, with context becoming increasingly important as models scale.

    Source: §1 · pp. 2–3, §5 · p. 9

  • Takeaways & Limitations

    C-SRRG provides a contextualized structured-report framework and dataset for benchmarking medical MLLMs, with dataset, code, and checkpoints slated for public release.

    Source: §1 · p. 2

    CheXagent-3B struggles to process and integrate extensive clinical context while maintaining structured-output adherence, limiting its utility when comprehensive context is required.

    Source: B LIMITATION IN CHEXAGENT-3B ON… · p. 20

Abstract

from arXiv · show

Automated structured radiology report generation (SRRG) from chest X-ray images offers significant potential to reduce workload of radiologists by generating reports in structured formats that ensure clarity, consistency, and adherence to clinical reporting standards. While radiologists effectively utilize available clinical contexts in their diagnostic reasoning, existing SRRG systems overlook these essential elements. This fundamental gap leads to critical problems including temporal hallucinations when referencing non-existent clinical contexts. To address these limitations, we propose contextualized SRRG (C-SRRG) that comprehensively incorporates rich clinical context for SRRG. We curate C-SRRG dataset by integrating comprehensive clinical context encompassing 1) multi-view X-ray images, 2) clinical indication, 3) imaging techniques, and 4) prior studies with corresponding comparisons based on patient histories. Through extensive benchmarking with state-of-the-art multimodal large language models, we demonstrate that incorporating clinical context with the proposed C-SRRG significantly improves report generation quality. We publicly release dataset, code, and checkpoints to facilitate future research for clinically-aligned automated RRG at https://github.com/vuno/contextualized-srrg.

1 INTRODUCTION

Radiology reporting is cognitively demanding, yet existing structured report-generation systems often omit clinical context that radiologists use. C-SRRG integrates that context, and the paper reports consistent quality improvements across medical MLLMs.

1 INTRODUCTION

  • Radiology reports combine detailed findings with concise clinical impressions, and producing them is cognitively demanding and time-consuming for radiologists.Automated systems are presented as a way to assist with high imaging volumes and reduce workload.Source: §1 · p. 1
  • Existing SRRG systems often omit clinical context, which can lead to systematic errors, including references to nonexistent prior studies or fabricated temporal comparisons.The paper identifies temporal hallucinations as a prominent example.Source: §1 · pp. 2–3
  • C-SRRG pairs structured report generation with a dataset containing multi-view images, clinical indication, imaging technique, and variable-length prior studies with comparisons.Models can incorporate these context sources depending on their architecture.Source: §1 · p. 2, Figure 2 · p. 2
  • Clinical context consistently improves report quality across evaluated medical MLLMs, with its importance increasing as models scale from 3B to 7B.The benchmark includes CheXagent-3B, MedGemma-4B, and Lingshu-7B.Source: §1 · pp. 2–3
  • The authors plan to release the dataset, code, and benchmarked-model checkpoints to support further C-SRRG research.The contribution summary also reports gains in F1-SRR-BERT and reduced temporal hallucinations.Source: §1 · pp. 2–3

2 RELATED WORK

Automated radiology report generation has developed from image-to-text systems toward structured reporting and multimodal models. Prior context-integration work spans multiple clinical inputs but remains limited in context coverage, temporal scope, and report format.

2 RELATED WORK

  • Radiology report generation progressed from vision encoders paired with language decoders to memory-driven, domain-specialized, cross-modal, and region-guided architectures.The paper focuses on extending structured report generation with clinical context aligned with radiologists’ workflow.Source: §2 · p. 3
  • Medical MLLMs combine visual understanding and natural-language generation, and the paper benchmarks medical-specific models for contextualized structured report generation.The related work names CheXagent, MedGemma, and Lingshu as medical-specific MLLMs.Source: §2 · p. 3
  • Figure 3 illustrates a temporal hallucination: a report says “new from prior exam” despite no prior study being provided.The example highlights the risk of making temporal comparisons without prior-study input.Source: Figure 3 · p. 4, §2 · p. 4
  • Prior context-integration approaches have used multi-view images, indications, clinical history, and previous studies, but often cover only partial context or a narrow temporal range and target free-form reports.The described limitations include relying only on the immediately preceding image or report.Source: §2 · pp. 3–4

3 METHOD

C-SRRG is designed to reflect radiologists’ use of clinical context, which can be missing or extensive across patients. It combines context-aware dataset curation with prompts and training that adapt to available information.

3.1 CURATION OF CONTEXTUALIZED CLINICAL CONTEXT

  • C-SRRG incorporates multi-view images, indication, technique, and prior studies with comparisons to reflect radiologists’ diagnostic workflow.The paper motivates this design in part by the risk of temporal hallucinations when prior studies are absent.Source: §3.1 · p. 4
  • The dataset builds on MIMIC and CheXpert Plus, ordering studies by patient and date and extracting indication, technique, and comparison from SRRG components parsed from reports.Available views are integrated using dataset-specific view-position fields.Source: §3.1 · p. 5
  • Prior-study availability and the number of studies per patient have long-tailed distributions, ranging from no history to extensive longitudinal sequences.This variability requires models to handle missing information while using context when it is available.Source: §3.1 · p. 6

3.2 CONTEXTUALIZED RADIOLOGY REPORT GENERATION (C-SRRG)

  • Prompts cover findings and impression both with and without prior studies, combining current-study context and images with prior-study information when available.The response format is standardized for structured report generation.Source: §3.2 · p. 6
  • Medical MLLMs are fine-tuned with contextualized prompt–response pairs using autoregressive next-token prediction, inserting prior studies only when available.This design is intended to produce context-aware reports when history is present and avoid hallucinated temporal comparisons when it is absent.Source: §3.2 · p. 6

4 EXPERIMENTS

Across findings and impression tasks, clinical context generally improves report metrics, while context used in both training and evaluation performs better than using it in only one phase. The experiments also find fewer temporal hallucinations when models are trained with context.

4.1 EXPERIMENTAL SETUPS

  • The experiments compare CheXagent-3B, MedGemma-4B, and Lingshu-7B using text-quality and clinical-accuracy metrics.CheXagent-3B impression training uses indication only after training failure with the full context; the two most recent prior studies are used, subject to model-specific image limits.Source: §4.1 · p. 7

4.2 EXPERIMENTAL RESULTS

  • C-SRRG substantially improves findings and impression results across evaluated models, with a few metric-specific exceptions.The findings and impression comparisons are reported in Tables 3 and 4.Source: Table 3 · p. 7, §4.2 · p. 7, Table 4 · p. 8
  • Using clinical context in only training or evaluation provides limited improvement or slight degradation; performance is highest when context is used in both phases.The study also reports that all context components contribute incrementally in most ablation cases, with the best performance using all available context.Source: §4.2 · p. 8, Table 5 · p. 8, Table 6 · p. 8
  • 22.9% to 10.7% (−12.2%) for Findings and 43.8% to 25.8% (−18.0%) for Impression are the temporal hallucination rate changes with C-SRRG.The evaluation counts temporal-reference indicators in reports produced without clinical context.Source: Table 9 · p. 9, §4.2 · p. 9

5 CONCLUSION, LIMITATIONS, AND FUTURE WORK

The paper concludes that contextualized SRRG aligns with radiologists’ workflow and improves reported generation outcomes. It also identifies annotation, model, and context-window constraints that bound the current approach.

5 CONCLUSION, LIMITATIONS, AND FUTURE WORK

  • C-SRRG integrates multi-view images, indication, imaging technique, and prior studies with comparisons, and the authors report improved text quality and diagnostic accuracy with fewer temporal hallucinations.The authors state that clinical context becomes increasingly critical as models scale up.Source: §5 · p. 9
  • The dataset’s synthetic LLM annotations may introduce biases and subtle hallucinations, while evaluation is limited to models up to 7B parameters and restricted image and context-window capacity.Recency-based selection may occasionally omit important historical context.Source: §5 · p. 9
  • Future work proposed by the authors includes larger models with extended context, learned or retrieval-based context selection, radiologist-feedback preference learning, and broader clinical modalities.Source: §5 · p. 9

REPRODUCIBILITY STATEMENT

The work reports reproducibility measures spanning public data sources, patient-level splits, experimental settings, and planned release materials.

REPRODUCIBILITY STATEMENT

  • C-SRRG is built from publicly available MIMIC-CXR and CheXpert-Plus, with data processing documented, including longitudinal history extraction and multi-view integration.Source: REPRODUCIBILITY STATEMENT · p. 10
  • The authors specify model hyperparameters, patient-level data splits, evaluation metrics, and fixed random seeds for experiments run on one NVIDIA H100 GPU.Source: REPRODUCIBILITY STATEMENT · p. 10
  • The authors commit to releasing code, the context-annotated dataset, trained checkpoints, and dataset-recreation documentation.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

The work uses existing de-identified public datasets and collects no additional patient data, while its synthetic reports may carry hallucinations, inconsistencies, and language-model biases.

ETHICS STATEMENT

  • C-SRRG uses de-identified public datasets, collects no new patient data, and maintains the source datasets’ privacy protections.The authors report no new ethical concerns and note that the source datasets received appropriate IRB approvals.Source: ETHICS STATEMENT · p. 10
  • The dataset and models are intended for research, not clinical decision-making without appropriate validation and regulatory approval.Source: ETHICS STATEMENT · p. 10

A.1 SYNTHETIC DATASET AND ANNOTATIONS

  • LLM reformulation of free-form reports may introduce hallucinations, inconsistencies, and biases that affect report diversity and clinical accuracy.C-SRRG builds on the SRRG dataset, which was generated by reformulating free-form radiology reports with large language models.Source: §A.1 · p. 19

A.2 MODEL ARCHITECTURE AND SCALE LIMITATIONS

Evaluation is limited by model scale and architecture, context selection, and supervised fine-tuning with greedy decoding. These constraints may affect integration of longitudinal and multimodal clinical information and the fidelity of generated reports.

A.2 MODEL ARCHITECTURE AND SCALE LIMITATIONS

  • Evaluation is limited to backbones up to 7B parameters, and two evaluated models were restricted to two images per case.The authors say the scale constraint likely underestimates achievable performance; MedGemma-4B supports multi-image processing and used fewer tokens per image.Source: §A.2 · p. 19
  • The authors identify larger foundation models, mixture-of-experts variants, and multimodal attention as possible directions for improving integration of temporal and multimodal information.Source: §A.2 · p. 19

A.3 CLINICAL CONTEXT INTEGRATION AND SELECTION LIMITATIONS

  • The approach limits the number of images and prior studies, prioritizes recent studies, and relies mainly on public datasets, potentially omitting useful history and limiting context diversity.The authors identify context-window limits and computational overhead as reasons for restricting longitudinal history.Source: §A.3 · p. 19

A.4 TRAINING METHODOLOGY AND DECODING LIMITATIONS

  • Training uses supervised fine-tuning with greedy decoding, which may not capture expert decision-making or learning from comparative feedback and iterative refinement.The authors propose radiologist-feedback preference learning and retrieval-conditioned decoding as directions for future work.Source: §A.4 · p. 20

B LIMITATION IN CHEXAGENT-3B ON C-SRRG-IMPRESSION

With full clinical context, CheXagent-3B often fails to follow the structured impression format, and its validation metrics drop sharply. The authors suggest difficulty processing long inputs may explain the degradation.

B LIMITATION IN CHEXAGENT-3B ON C-SRRG-IMPRESSION

  • With full clinical context, CheXagent-3B often produces single-word or generic phrases instead of structured, detailed impression sections.Examples include “Pneumothorax” and “Pneumonia” in place of multi-point impressions.Source: B LIMITATION IN CHEXAGENT-3B ON… · p. 20
  • BLEU fell from 9.44 to 2.57, ROUGE-L from 34.03 to 21.76, and BERTScore from 61.82 to 40.10 on validation with full clinical context.The reported drops are for CheXagent-3B on C-SRRG-Impression.Source: Table 11 · p. 20, B LIMITATION IN CHEXAGENT-3B ON… · p. 20
  • The authors suggest the degradation reflects difficulty processing extensive clinical context while maintaining structured output, possibly after training primarily on shorter sequences.Source: B LIMITATION IN CHEXAGENT-3B ON… · p. 20

C DETAILED DATASET STATISTICS

The appendix describes patient-level dataset splits, prompt formats, and instruction-tuning examples that integrate clinical context. The examples distinguish detailed findings with temporal comparison from impression-level clinical synthesis.

C DETAILED DATASET STATISTICS

  • Training and evaluation splits have zero patient overlap; findings and impression test sets contain 274 and 423 patients, respectively.The test-reviewed splits contain 173 findings patients and 172 impression patients, with 106 and 108 respectively shared with the test split.Source: C DETAILED DATASET STATISTICS · p. 21, Figure 10 · p. 21

D PROMPT DESIGNS

E INSTRUCTION TUNING DATASET PROMPT EXAMPLE

  • Instruction-tuning examples integrate medical history, imaging techniques, previous findings, temporal comparisons, and multi-view information.The examples preserve temporal, multi-view, and metadata context for report generation.Source: E INSTRUCTION TUNING DATASET PROMPT… · p. 23

E.1 FINDINGS GENERATION EXAMPLE

  • The findings example requires detailed anatomical observation and temporal comparison across multiple studies.Source: §E.1 · p. 23

E.2 IMPRESSION GENERATION EXAMPLE

  • The impression example demonstrates clinical synthesis and diagnostic reasoning.Source: §E.2 · p. 23

F HALLUCINATION ANALYSIS

The analysis traces temporal hallucinations from context-free training data to generated reports, and describes the authors’ limited use of LLMs for editing.

F HALLUCINATION ANALYSIS

  • The analysis quantifies temporal hallucinations by counting temporal statements in reports generated on evaluation sets without clinical context.Source: F HALLUCINATION ANALYSIS · p. 23
  • Reports trained without clinical context can reproduce temporal comparisons indiscriminately, including for patients with no prior imaging history.Such statements may be appropriate when prior studies are available, but become hallucinations without that context.Source: F HALLUCINATION ANALYSIS · pp. 23–24
  • MedGemma-4B trained without clinical history generates comparative language despite having only the current study, including “No significant interval change.”Source: F HALLUCINATION ANALYSIS · pp. 24, 32

G THE USE OF LLMS

  • LLMs were used only for light editing, including grammar correction and wording polish, not for ideation, experiments, analysis, or substantive writing.Source: G THE USE OF LLMS · p. 34

Related papers

Loading 2510.00428v1…