PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty as a Proxy for Semantic Correctness in Diffusion-Based Medical Image Synthesis

Yuxuan Ou, Konstantinos Kamnitsas, OxAAA Study, AICT Consortium, Regent Lee, Vicente Grau

arXiv:2610.03224v1cs.CVcs.AI

TL;DR

Visually realistic generated medical images may still have incorrect anatomy, and image-similarity metrics do not directly measure semantic correctness. This study tests uncertainty as a proxy for correctness in NCCT-to-CECT synthesis using jointly generated lumen segmentations and six uncertainty methods. Uncertainty is informative across spatial scales and for OOD detection, with MCDropout among the leading methods and requiring no extra training when the base model already uses dropout.

  • Problem

    Visual realism and image-similarity metrics do not directly establish whether generated anatomy is correct, and prior uncertainty evaluations have not established its correspondence with semantic errors.

    Source: §I · p. 1

  • Method

    The study compares six uncertainty methods in NCCT-to-CECT synthesis, evaluating uncertainty against segmentation-derived semantic correctness at pixel, region, and image levels and for OOD cases.

    Source: §I · p. 2

  • Results

    Uncertainty is informative about semantic correctness at all three scales and supports external image-level failure detection and OOD identification, while MCDropout ranks among the leading methods at every scale.

    Source: §I · p. 2

  • Takeaways & Limitations

    Uncertainty can serve as a practical signal for quality filtering, reliability assessment, and OOD detection in NCCT-to-CECT synthesis.

    Source: §VII · p. 7

    The experiments cover only abdominal-aorta NCCT-to-CECT synthesis, so generalizability to other organs, modalities, and generation problems remains to be validated empirically.

    Source: §VII · p. 8

Abstract

from arXiv · show

Diffusion models can synthesise contrast-enhanced CT (CECT) from non-contrast CT (NCCT), avoiding contrast administration and its environmental and patient-access costs. However, visually realistic images are not necessarily anatomically correct, and the pixel-intensity and feature-space similarity metrics used to assess generation quality do not directly measure anatomical correctness. In this work, we investigate whether uncertainty can serve as a proxy for semantic correctness in diffusion-based medical image synthesis. We study NCCT-to-CECT synthesis using AortaDiff, a multitask diffusion framework that jointly generates CECT images and lumen segmentations. The segmentation output provides an explicit representation of the generated vascular anatomy, enabling segmentation-derived errors to be used as a quantitative measure of generation correctness. Six methods spanning weight (Ensemble, HyperDiff, BayesDiff), architecture-perturbation (MCDropout), generative-stochasticity (RDS) and input-perturbation (TTA) uncertainty are compared at the pixel, region and image levels, and for detection of clinically relevant out-of-distribution (OOD) cases. Uncertainty proves informative at all three spatial scales, remains informative on an external multi-centre dataset under distribution shift, and supports OOD detection. MCDropout stands out among the six: it ranks among the leading methods at every scale, generalizes well on the external dataset, and can be enabled at inference on any model already trained with dropout, so reliable uncertainty comes at no extra training cost. Uncertainty reliably flags severe failures but discriminates poorly among already high-quality images. These findings support uncertainty as a practical and computationally economical signal for quality filtering, reliability assessment and OOD detection in NCCT-to CECT synthesis.

I. INTRODUCTION

The introduction frames trustworthy NCCT-to-CECT synthesis as important because contrast administration has environmental and patient-access costs, yet visual similarity does not establish anatomical correctness. It presents a segmentation-based framework for testing whether uncertainty tracks semantic error across scales and under distribution shift.

I. INTRODUCTION

  • Synthesizing CECT from NCCT has potential environmental, resource-resilience, and patient-access benefits because contrast administration adds costs and may be withheld from patients with impaired renal function.Iodinated contrast media are poorly removed by conventional wastewater treatment and are now widely detected as environmental contaminants.Source: §I · p. 1
  • Visual realism and pixel- or feature-space similarity metrics do not directly establish whether generated anatomy is correct, leaving generation correctness difficult to assess.Small aortic structural errors may affect downstream analysis and clinical decisions.Source: §I · p. 1
  • Uncertainty is informative about semantic correctness at pixel, region, and image levels, is most informative for severe failures, and supports external image-level failure detection and OOD identification.MCDropout ranks among the leading methods at every scale and, when the base model was trained with dropout, requires no additional training.Source: §I · p. 2
  • The proposed framework evaluates uncertainty against semantic correctness measured by agreement with ground-truth anatomy, using the lumen segmentation jointly predicted by a multitask diffusion model.The evaluation spans pixel, region, and image levels, external multi-centre data under distribution shift, and clinically relevant OOD cases such as vascular stents.Source: §I · p. 2
  • The study compares six diffusion uncertainty methods spanning four sources of uncertainty.These sources are weight, architecture perturbation, generative stochasticity, and input perturbation.Source: §I · p. 2

II. RELATED WORK: UNCERTAINTY QUANTIFICATION METHODS

Diffusion uncertainty methods differ in where they introduce variability. The related work groups methods by these sources and describes three approaches to weight uncertainty.

II. RELATED WORK: UNCERTAINTY QUANTIFICATION METHODS

  • The broader method taxonomy locates uncertainty in model parameters, the network computation, diffusion sampling, or input conditions.It covers weight uncertainty, architecture perturbation, generative stochasticity, and input perturbation.Source: §II · p. 3

A. Weight Uncertainty

  • Weight uncertainty represents variability in learned parameters, based on the assumption that parameter disagreement propagates to uncertainty in generated outputs.The paper considers BayesDiff, HyperDiff, and deep ensembles as complementary implementations.Source: A. Weight Uncertainty · p. 3
  • BayesDiff fits a Gaussian posterior around the pretrained final layer using a last-layer Laplace approximation, then propagates predictive variance through reverse diffusion to produce a pixel-wise uncertainty map.The covariance term is estimated by Monte Carlo sampling.Source: A. Weight Uncertainty · p. 3
  • HyperDiff uses a Bayesian hypernetwork to generate plausible denoiser weights, estimating epistemic uncertainty across sampled weights and aleatoric uncertainty within each weight configuration.Source: A. Weight Uncertainty · p. 3
  • Deep ensembles approximate weight uncertainty by independently training multiple models and measuring disagreement among their predictions.Source: A. Weight Uncertainty · p. 3

B. Input Perturbation Uncertainty

The paper describes three uncertainty methods that introduce variability through transformed inputs, stochastic network computation, or repeated diffusion sampling.

B. Input Perturbation Uncertainty

  • Test-time augmentation estimates input-perturbation uncertainty from prediction variability across randomly transformed inputs while keeping model parameters fixed.Source: B. Input Perturbation Uncertainty · p. 3

C. Architecture Perturbation Uncertainty

  • MCDropout estimates architecture-perturbation uncertainty by keeping dropout active during inference, producing different effective subnetworks across repeated forward passes.Uncertainty is measured from the variability of their predictions.Source: C. Architecture Perturbation Uncertainty · p. 3

D. Generative Stochasticity

  • Repeated deterministic DDIM sampling estimates generative stochasticity from outputs generated with different initial noise realizations.Because the reverse process is deterministic given the initial noise, variation across samples reflects differences in that noise.Source: D. Generative Stochasticity · p. 3

III. METHODOLOGY

A multitask diffusion model jointly generates a synthetic CECT image and lumen mask from an NCCT slice, allowing segmentation-derived errors to serve as a semantic correctness target. The methodology evaluates generation uncertainty against those errors at multiple spatial scales.

A. Multitask Diffusion Framework

  • The multitask model jointly predicts synthetic CECT and a lumen segmentation mask from NCCT, making the segmentation an explicit representation of generated lumen structure.Segmentation-derived errors provide a semantic proxy for generation correctness rather than appearance similarity measured by PSNR, SSIM, or LPIPS.Source: A. Multitask Diffusion Framework · p. 3

B. Generation Uncertainty

  • Uncertainty maps are estimated from the CECT generation branch, while the segmentation branch defines the semantic correctness target.Source: B. Generation Uncertainty · p. 3

C. Semantic Correctness Definition and Evaluation

1) Pixel Level:

  • Pixel-wise AUROC measures whether uncertainty separates correct from incorrect pixels within the aortic region; sparsification removes highest-uncertainty pixels and recomputes retained Dice.AUSE is the gap from the oracle curve, with lower values better; AUSC measures retained Dice, with higher values better.Source: 1) Pixel Level · p. 3

C. Semantic Correctness Definition and Evaluation

  • Regional analysis partitions the aortic region into SLIC superpixels and measures each region’s error as the proportion of incorrectly predicted pixels.Regional uncertainty is the mean pixel uncertainty within each superpixel.Source: 2) Region Level · p. 3

2) Region Level:

Regional uncertainty is used to rank superpixels for removal and assess retained Dice. The evaluation also tests whether image-level uncertainty identifies stent-containing slices as clinically relevant OOD cases.

2) Region Level:

  • Regional sparsification removes superpixels in descending mean uncertainty and recomputes retained Dice as increasing fractions of aortic pixels are removed.Curves are summarized by AUSE and AUSC; Spearman correlation measures rank agreement between regional uncertainty and error.Source: 2) Region Level · pp. 3–4

D. Out-of-Distribution Detection

  • Stents, absent from training, define the OOD test: image-level scores aggregated from pixel-wise maps classify stent-containing versus non-stent slices using AUROC.The external evaluation uses previously unseen structures to test uncertainty under distribution shift.Source: D. Out-of-Distribution Detection · p. 4
  • The semantic-correctness and OOD evaluations together assess uncertainty as a reliability signal for semantic errors and clinically relevant distribution shifts.Source: D. Out-of-Distribution Detection · p. 4

IV. DATASETS

The Oxford dataset supports model training and in-distribution testing, while AICT provides external evaluation across clinical sites and scanners. Both datasets contain paired NCCT and CECT scans.

A. Oxford Dataset

  • The Oxford dataset contains 264 registered NCCT–CECT cases; 20 lumen-annotated cases were held out for testing, yielding 1,547 paired axial slices before filtering.The 199 cases without lumen annotations and 45 annotated cases were used for training and validation.Source: A. Oxford Dataset · p. 4
  • Registration-Dice filtering above 0.90 reduced the Oxford test set to 1,316 slices.Source: A. Oxford Dataset · p. 4

B. AICT Dataset

  • AICT provides external evaluation using paired scans from multiple clinical centers, scanners, and acquisition protocols.It is sourced from 10 hospitals across 3 continents; only 61 cases passed preprocessing and quality control, yielding 5,950 slices, including 1,575 stent-containing slices.Source: B. AICT Dataset · p. 4
  • Because AICT lacked expert lumen annotations, Oxford-trained automated segmenters supplied masks that were manually inspected or reviewed, and cases had to meet image-quality and registration criteria.Cases were retained only when at least one automated method produced an acceptable aortic mask and registered aortic masks had Dice above 0.90.Source: B. AICT Dataset · p. 4

V. UNCERTAINTY ESTIMATION IMPLEMENTATION DETAILS

The study uses a multitask diffusion model’s lumen segmentation as a measure of generated anatomy and tests whether it supports uncertainty evaluation. The segmentation agrees with independent segmenters, and registration stratification suggests residual misalignment is unlikely to dominate the uncertainty–error relationship.

V. UNCERTAINTY ESTIMATION IMPLEMENTATION DETAILS

  • The implementation section summarizes four uncertainty sources and six representative methods, with BayesDiff using a Laplace approximation and HyperDiff reporting epistemic uncertainty.BayesDiff generation uncertainty is the final propagated variance; HyperDiff’s aleatoric and epistemic estimates perform comparably at all evaluation levels.Source: §V · p. 5

VI. RESULTS

  • Results proceed from validation of the segmentation proxy to pixel-, region-, and image-level uncertainty analysis, followed by external-data and OOD evaluation.Source: §VI · p. 5

A. Using Segmentation as a Semantic Proxy for Generation Correctness

  • 25.37 vs. 25.12 PSNR, 0.84 vs. 0.80 SSIM, and 0.06 vs. 0.08 LPIPS: the multitask framework improves on a two-stage pipeline at equal segmentation Dice of 0.90.The comparison uses DDIM inference with 10 steps on the refined test set.Source: A. Using Segmentation as a Semantic… · p. 5
  • Mean Dice of 0.95 with three independent segmenters supports the multitask segmentation as a faithful representation of synthesized vascular anatomy and a basis for measuring semantic error.The independent models were nnU-Net, UNet++ and Attention U-Net, trained on real CECT annotations.Source: A. Using Segmentation as a Semantic… · p. 5
  • Across registration-Dice strata, pixel-level AUROC differed by at most 0.03 and pixel-level AUROC and region-level Spearman rankings were identical, making residual misalignment unlikely to dominate the uncertainty–error relationship.The strata were above 0.95 (n = 791) and 0.90–0.95 (n = 525).Source: 1) Robustness to Registration Quality · p. 5

1) Robustness to Registration Quality:

Uncertainty methods differ in how closely their rankings track semantic errors across pixel and region scales. MCDropout is among the strongest pixel-level methods, while HyperDiff and MCDropout lead region-level results.

B. Relationship Between Generation Uncertainty and Semantic Correctness

  • MCDropout, HyperDiff, and RDS form a close top cluster for pixel-level AUROC, while BayesDiff is no better than random guessing (AUROC 0.493).MCDropout’s pixel-level uncertainty corresponds most closely with semantic errors on AUSE, AUSC, and AUROC; MCDropout and RDS are lightweight and retraining-free, unlike HyperDiff’s trained hypernetwork.Source: B. Relationship Between Generation… · p. 5
  • MCDropout tracks the oracle throughout pixel-wise sparsification, whereas HyperDiff’s large early gains weaken at high removal fractions despite identifying highly uncertain error pixels.The figure compares retained Dice as pixels are removed by uncertainty ranking against theoretical-best and random removal.Source: B. Relationship Between Generation… · p. 5, Figure 2 · p. 5
  • HyperDiff and MCDropout outperform the other methods and are effectively tied across all three region-level metrics, while BayesDiff’s negative correlation is −0.24 ± 0.28.The region-level ranking differs from the pixel-level ranking, indicating the two scales capture distinct properties.Source: 2) Region-Level Evaluation · p. 5

1) Pixel-Level Evaluation:

Uncertainty estimates identify semantic errors at region and image levels, but their effectiveness depends on scale and failure severity.

2) Region-Level Evaluation:

  • Figure 3 evaluates region-level sparsification by progressively removing SLIC superpixels in decreasing order of average uncertainty, recomputing retained Dice at each step, and comparing uncertainty-guided curves with oracle and random removal.Source: Figure 3 · p. 6
  • Region averaging lets HyperDiff flag superpixels containing error pixels that its pixel-level uncertainty under-flags.High uncertainty in neighbouring error pixels can outweigh under-flagged pixels within the same superpixel.Source: 2) Region-Level Evaluation · p. 6, Figure 4 · p. 6

3) Image-Level Evaluation:

  • For severe failures (Dice < 0.65), MCDropout, TTA, RDS, and Ensemble exceed 0.9 AUROC and remain above 0.8 to roughly 0.85.AUROC falls as the failure threshold rises from 0.6 to 0.9, so uncertainty is most informative for severe failures.Source: 3) Image-Level Evaluation · p. 6
  • HyperDiff follows the image-level oracle most closely, retaining about 0.93 Dice after removing 95% of the most uncertain images, but no method ranks optimally.The four uncertainty aggregation strategies yield similar trends and do not change the method ranking.Source: 3) Image-Level Evaluation · p. 6

4) External Validation Under Distribution Shift:

On external data, uncertainty supports image-level failure detection and identifies stent-related OOD cases, with method performance varying by task.

4) External Validation Under Distribution Shift:

  • On external AICT data, MCDropout exceeds 0.8 AUROC across most aggregation strategies and nearly the full threshold range, while other methods largely retain their internal ranking.External AUROC curves are flatter, so failure detection varies less across Dice thresholds under distribution shift.Source: 4) External Validation Under… · p. 7, Figure 8 · p. 7
  • In representative external cases, uncertainty concentrates where generated anatomy deviates from ground-truth CECT, though not necessarily on the stent itself.The deviations correspond to generation errors induced by the unseen structure.Source: 4) External Validation Under… · p. 7, Figure 7 · p. 7

5) Out-of-Distribution Detection:

  • For stent OOD detection, all methods except BayesDiff perform similarly at 0.75–0.85 across aggregation strategies; BayesDiff ranges from 0.48 with mean aggregation to 0.82 with maximum aggregation.The A KNN feature-space detector achieves 0.43–0.44, below every uncertainty-based method.Source: 5) Out-of-Distribution Detection · p. 7

VII. DISCUSSION

The discussion presents uncertainty as a multi-scale reliability signal, while emphasizing method-specific strengths and the study’s narrow empirical scope.

VII. DISCUSSION

  • MCDropout performs strongly across scales and under distribution shift, and requires no additional training when the base model already contains dropout.The paper describes it as a practical default for reliability assessment.Source: §VII · p. 7
  • Uncertainty mechanisms align with different strengths: HyperDiff concentrates variability in spatially coherent regions, while MCDropout’s neighbouring-pixel disagreement is smooth.The authors suggest BayesDiff’s final-layer-only posterior may explain its weaker performance and negative region-level correlation.Source: §VII · p. 7
  • Method choice depends on the intended clinical application: TTA sensitivity to input changes aligns with OOD performance, while RDS variability tends to indicate lower-quality generations.Training-free methods match or exceed retraining-based methods at every scale, avoiding retraining costs for deployment-time assessment.Source: §VII · p. 8
  • The experiments cover only abdominal-aorta NCCT-to-CECT synthesis; generalization to other organs, modalities, and generation problems remains empirically unvalidated.The authors expect the framework to translate to tasks with quantifiable anatomical structures, but state this as an expectation.Source: §VII · p. 8
Loading 2610.03224v1…