PaperlayerView exact revision on arXiv

Source-linked AI summary

Transition-Aware best-of-N sampling for Longitudinal Chest X-ray Reports

Halil Ibrahim Gulluk, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert

arXiv:2606.28393v1cs.CV

TL;DR

Chest X-ray report generators and best-of-N selection usually treat exams independently, despite reports emphasizing change from prior studies. The paper introduces transition-aware best-of-N sampling and finds it consistently outperforms random selection, with the largest gains on Impressions.

  • Problem

    Most chest X-ray report generators and best-of-N pipelines score each exam independently, although longitudinal reports emphasize changes from prior studies.

  • Method

    Transition-aware best-of-N represents prior-to-current report changes as directional set vectors and scores candidates against cached ground-truth training transitions.

  • Results

    Every transition-vector aggregation beats random on every headline metric; Impressions gain up to +8.5% ROUGE-L with Novelty/kNN.

  • Takeaways & Limitations

    Across nine model-prompt configurations, transition-aware selection consistently improves both report sections, especially the change-oriented Impressions section.

  • Takeaways & Limitations

    The framework uses a single prior visit, while longer histories remain an extension for future work.

Abstract

from arXiv · show

In longitudinal clinical practice, every chest X-ray is read in the context of the patients prior exam, and much of what the radiologist communicates is the change from one visit to the next. To the best of our knowledge, we present the first training-free best-of-N sampling scheme for pre-trained chest X-ray report generators that is explicitly aware of this longitudinal prior to current transition. We call it transition-aware best-of-N sampling, each report is split into sentences and embedded into an unordered set in Rd; each (prior, current) pair is reduced to a fixed-dim directional vector via a set-to-set distance designed to encode the change between the two sets; and candidates are scored by cosine distance from their candidate transition vector to a cached bank of ground-truth training transition vectors, aggregated as min or kNN. We instantiate the framework with four directional set distances (mean-shift, novelty residual, directed-Hausdorff anchor, and cost-weighted optimal transport) and evaluate on a multi-visit AP-PA cohort, running inference under three prompts on three vision-language generators. Transition-aware best-of-N outperforms random selection across the board, with the largest relative gains on the Impression section.

1 Introduction

Chest X-ray report generators can draft credible reports, but existing best-of-N pipelines usually score candidates without longitudinal context. This work selects candidates by modeling clinically meaningful transitions from prior to current studies.

  • Motivation: Chest radiography is widely performed, and automated report generation could reduce radiologist workload and improve reporting consistency.
  • Motivation: Best-of-N sampling draws N candidate reports per image and selects the candidate favored by a preference or performance criterion.
  • Problem: Most chest X-ray generators and best-of-N pipelines treat test images independently, despite follow-up exams being interpreted around changes since prior studies.
  • Novelty: The proposed selection strategy evaluates longitudinal candidates through prior-to-current transitions, extending longitudinal modeling to best-of-N selection.Prior work showed that feeding longitudinal context to the decoder can improve generation quality, but this context had not been exploited at selection.
  • Method: Reports are sentence-embedded as unordered sets, converted into fixed-dimensional directional transition vectors, and cosine-scored against a cached bank of ground-truth training transitions.Transition-space scoring penalizes candidates that silently drop chronic findings or hallucinate unsupported acute worsening.

2 Related work

Related work spans early CNN–sequence-decoder and retrieval systems, newer vision-language report generators, and methods incorporating prior examinations. The proposed approach differs by scoring best-of-N candidates in transition space using sentence-set representations rather than image-independent or candidate-level scores.

  • Early systems used CNN encoders with LSTM/Transformer decoders or retrieved canned sentences from training-report memories.
  • Recent systems pair domain-specific visual encoders with general-purpose LLMs, including R2GenGPT, CheXagent, and MAIRA-2.CheXagent uses a curated mixture of 28 CXR datasets, while MAIRA-2 adds per-finding grounding.
  • Several methods use prior studies during generation, conditioning or fusing prior–current examinations for temporal tasks and longitudinal report generation.These approaches target the decoder and may use a learned placeholder for an absent prior or a CXR-BERT semantic-similarity reward.
  • Sentence-set distances such as Chamfer, Hausdorff, optimal transport, and Hungarian matching provide continuous, permutation-invariant comparisons between reports.SDR applies sentence-set distances to GRPO rewards and single-image best-of-N scoring but remains image-independent and does not encode change.
  • Standard best-of-N sampling draws N stochastic candidates and keeps the highest-scoring report, whereas this pipeline scores candidates in transition space.

3 Method

The method represents Findings and Impression sentences as unordered embedding sets, then encodes prior-to-current changes with directional, vector-valued set distances. It selects generated reports whose section transitions are closest to cached ground-truth training transitions.

  • Report representation: Each report section is split into sentences and embedded with frozen all-mpnet-base-v2, producing unordered Findings and Impression sets in R^d.The sentence embedding dimension is d = 768.
  • Transition representation: A directional set distance maps prior and current embedding sets to a fixed-dimensional vector encoding an asymmetric change from prior to current.Its output dimension d′ does not depend on the numbers of sentences in the two sets and supports cosine comparisons across transitions.
  • Directional set distances: The framework instantiates four transition designs: Mean-Shift, Novelty, Dir-Hausdorff, and Cost-OT.They represent centroid shift, current-sentence novelty, directed-Hausdorff-anchor displacement, and cost-weighted optimal-transport displacement, respectively.
  • Transition bank: Ground-truth transition vectors are pre-computed from the frozen encoder and directional design, then cached as section-specific training banks.The banks are cached once per training corpus.
  • Candidate scoring and selection: Each generated candidate is embedded into section transition vectors, scored against its section bank with Dmin or DkNN, and selected by minimum summed distance across Findings and Impression.Dmin tests resemblance to any single training transition, whereas DkNN provides a noise-robust soft alternative.

4 Experimental setup

The study evaluates transition-aware report selection on a multi-visit AP/PA cohort with 22,745 training and 1,727 test transitions. It compares five-candidate sampling across three vision–language models and three prompts, scoring selected reports against current-visit ground truth for Findings and Impressions.

  • Dataset: 22,745 training transitions populate the transition bank, while 1,727 test transitions form the evaluation set.The cohort retains patients with at least two qualifying exams, samples one AP or PA view per patient, and forms one transition per consecutive visit pair.
  • Models and prompts: Three vision–language models are evaluated: Gemini-2.5-Flash-Lite, Gemini-3.1-Flash-Lite-preview, and Mistral-Small-2603.Each model is tested under three prompts: image-only zero-shot, few-shot with five random training reports, and zero-shot conditioned on the prior ground-truth report.
  • Sampling: For every model–prompt configuration and test transition, K = 5 candidate reports are sampled with distinct seeds.The prior-report prompt is the only configuration that places longitudinal context in the generator’s input.
  • Evaluation: Selected responses are scored against the current-visit ground-truth report using NLP overlap and clinical-content metrics, separately for Findings and Impressions.The full metric list and references are provided in Appendix D.
  • Baseline: The random baseline uniformly selects one of the five candidates and averages results over five seeds.Table 1 reports headline Impression metrics averaged across runs in which every listed method has data.

5 Results

Transition-aware best-of-N sampling outperforms random selection across Impressions and Findings, with the largest gains on Impressions. Gains remain consistent across most model–prompt configurations and across Gemini and Mistral.

  • Impressions: Impressions achieve +8.5% ROUGE-L with Novelty/kNN, +13.6% RadGraph averaged-F1 with Mean-Shift/min, and +4.4% METEOR with Dir-Hausdorff/kNN.These short, change-oriented summaries are the regime where transition signals should carry the most ranking information.
  • Findings: Every min and kNN aggregation of every transition vector beats random on every headline Findings metric.The strongest BERTScore-F1 and RadGraph averaged-F1 results come from Cost-OT/min, while Mean-Shift/kNN leads CheXbert F-14.
  • Findings: +3.1% BERTScore-F1 and +5.1% RadGraph averaged-F1 are the strongest Findings results, both from Cost-OT/min.Mean-Shift/kNN gives the best CheXbert F-14 at +2.2%.
  • Configuration breakdown: Transition-aware best-of-N beats random on nearly every Impressions configuration and most Findings configurations, with larger gains on weaker prompts.The trend is consistent across both Gemini and Mistral.

6 Conclusion · A Prompts · P1 – Zero-shot, image-only

The paper introduces a training-free transition-aware best-of-N sampler that scores candidate chest X-ray reports using prior-to-current transition vectors and cached ground-truth transitions. The appendix specifies the prompts, including zero-shot image-only generation with a concise Findings/Impression format.

  • 6 Conclusion: The method reduces each prior–current report pair to a fixed-dimensional directional vector and scores candidates against cached ground-truth training transition vectors.It is presented as the first training-free best-of-N scheme for pre-trained chest X-ray report generators that explicitly conditions scoring on the patient’s prior exam.
  • 6 Conclusion: The pipeline is evaluated across nine (model, prompt) configurations on a multi-visit AP/PA cohort.The supplied conclusion passage states that the pipeline’s results were consistent across these configurations, but the excerpt does not provide the associated quantitative values.
  • 6 Conclusion: Proposed extensions include learnt directional encoders, larger candidate pools N, histories longer than one prior visit, and combining prior-conditioned generation with prior-conditioned selection.The extensions target alignment with a clinical metric, broader candidate search, longer longitudinal context, and complementary prior-based interventions.
  • A Prompts: The appendix reproduces verbatim the three prompts used in the experiments.This appendix section documents the experimental prompting setup rather than introducing a new model component.
  • A Prompts: Few-shot examples in P2 are sampled randomly from the training corpus for every test query.This specifies how the few-shot prompt is instantiated at inference time.
  • A Prompts: P3 fills {prior_findings} and {prior_impression} at query time with the patient’s prior ground-truth Findings and Impression.The placeholders are populated separately from the prior report’s Findings and Impression sections.
  • P1 – Zero-shot, image-only: P1 instructs the model to generate a concise clinical report for the given image in the exact Findings: <text> Impression: <text> format.The zero-shot prompt is image-only and requires professional radiology language without explanations, disclaimers, or extra text.

P2 – Few-shot with random training examples · P3 – Zero-shot with prior report as context

P2 uses randomly selected few-shot reports to constrain concise, clinically accurate chest-X-ray generation in a fixed Findings/Impression format. P3 instead supplies the prior report as context, requiring the model to describe the new image and interval changes without copying prior text.

  • P2 – Few-shot with random training examples: P2 instructs the model to output only Findings and Impression sections.The prompt prohibits explanations, disclaimers, medical-advice language, and additional commentary.
  • P2 – Few-shot with random training examples: P2 requires concise, clinically accurate reports that match the examples’ writing style, tone, and structure.
  • P2 – Few-shot with random training examples: P2 provides randomly selected example reports covering findings and impressions such as cardiomegaly, pneumonia, peribronchial thickening, and no active cardiopulmonary disease.
  • P3 – Zero-shot with prior report as context: P3 gives the model the patient’s prior chest-X-ray report as clinical context for generating a report of the new image.The prompt applies to patients with at least one prior chest-X-ray examination.
  • P3 – Zero-shot with prior report as context: P3 directs the model to describe current findings and changes relative to the prior, including new findings, worsening, improvement, and resolution.
  • P3 – Zero-shot with prior report as context: P3 enforces the same Findings/Impression-only structure and excludes explanations, disclaimers, extra text, and medical-advice phrases.
  • P3 – Zero-shot with prior report as context: P3 prohibits verbatim copying of the prior report and requires professional radiology language that remains concise and structured.

B Directional set distances

This section defines four directional set-to-set distances that convert pairs of sentence-embedding sets into fixed-dimensional transition vectors for cosine comparison.

  • Directional set distances: Four distances—Mean-Shift, Novelty, Dir-Hausdorff, and Cost-OT—are defined for directional set-to-set transitions.These distances are summarized in Table 3 and defined in Section 3.2.
  • Directional set distances: Each distance maps a pair of sentence-embedding sets to a single fixed-dim vector that can be cosine-compared across transitions.

C Multi-visit cohort statistics

Table 4 reports patient, visit, and transition counts for each split of the multi-visit AP/PA cohort used throughout Section 4.1.

  • Table 4 reports per-split patient, visit, and transition counts for the multi-visit AP/PA cohort used throughout Section 4.1.

D Evaluation metrics in detail

Selected responses are evaluated against the current-visit ground-truth report separately for Findings and Impressions using text-overlap, semantic, and clinical-content metrics. Headline tables use seven representative scores spanning these evaluation categories.

  • Responses are scored separately for Findings and Impressions against the current-visit ground-truth report.
  • The evaluation suite includes BLEU-1–4, ROUGE-1/2/L precision, recall, and F-measure, METEOR, BERTScore-F1, COMET, chrF++, RadGraph F1, and CheXbert F1.BERTScore-F1 uses roberta-large with baseline rescaling; CheXbert is reported for 14 and 5 classes.
  • The seven headline scores are BLEU-1, ROUGE-1 F-measure, ROUGE-L F-measure, METEOR, BERTScore-F1, RadGraph averaged-F1, and CheXbert F-14.
  • These headline metrics span shallow text overlap, soft lexical or semantic similarity, and clinical-content correctness.BLEU-1 and ROUGE represent overlap; METEOR and BERTScore-F1 represent lexical or semantic similarity; RadGraph and CheXbert represent clinical content.

E Per-(model, prompt) absolute values · F Per-(model, prompt) delta tables

Across nine (model, prompt) runs, Novelty/min beats random on ROUGE-L and RadGraph averaged-F1 for Impressions, with gains generally extending to Findings. The delta tables then enumerate absolute method-versus-random differences across four transition designs and two aggregations.

  • E Per-(model, prompt) absolute values: Tables 5 and 6 report absolute headline values for Novelty/min versus random, with one row per (model, prompt) configuration.
  • E Per-(model, prompt) absolute values: Novelty/min beats the matched random baseline on every (model, prompt) configuration for ROUGE-L and RadGraph averaged-F1 on Impressions.
  • E Per-(model, prompt) absolute values: On Findings, Novelty/min improves ROUGE-L and RadGraph averaged-F1 on the majority of configurations.
  • E Per-(model, prompt) absolute values: The largest absolute gains occur under weaker prompts P1 and P2, while P3 compresses but does not invert the method gain.Under P3, the prior is already in the prompt and random starts from a substantially higher absolute value.
  • E Per-(model, prompt) absolute values: Both Gemini and Mistral show positive deltas, indicating that gains are not tied to a single generator’s failure modes.
  • F Per-(model, prompt) delta tables: The delta tables report method-versus-random differences for BERTScore-F1 and RadGraph averaged-F1 across four transition designs and min or kNN aggregation.They report Impressions first and Findings second for every (model, prompt) configuration.
Loading 2606.28393v1…