Source-linked AI summary
Uncertainty Quantification in Deep MRI Reconstruction
Vineet Edupuganti, Morteza Mardani, Shreyas Vasanawala, John Pauly
TL;DR
Deep-learning MRI reconstruction needs uncertainty assessment because models can produce realistic artifacts on data unlike their training examples. This paper combines probabilistic VAE reconstruction, Monte Carlo variance maps, and SURE-based risk estimation. Its evaluations report greater uncertainty with adversarial loss and lower uncertainty with recurrent blocks, while SURE estimates remain correlated with risk.
Problem
Deep-learning MRI reconstruction can be limited on data unlike training examples and may introduce realistic artifacts that mislead radiologists.
Source: §III · p. 2Method
The paper uses a probabilistic VAE, Monte Carlo sampling for pixel variance maps, and SURE to estimate reconstruction risk without ground truth.
Source: §I · p. 1Results
Adversarial loss increases pixel uncertainty while better recovering high frequencies, recurrent blocks decrease uncertainty, and SURE correlation has R² = 0.97 for 2-fold and R² = 0.84 for 16-fold undersampling.
Source: §I · p. 1, §VII · p. 9, F. SURE results · p. 9Takeaways & Limitations
SURE serves as a tool for assessing reconstruction risk when the fully sampled ground-truth image is unavailable.
Source: §I · p. 1The reported SURE measure is global; extending it to local, pixel-wise risk remains future work.
Source: §VII · p. 9
Abstract
from arXiv · showhide
Reliable MRI is crucial for accurate interpretation in therapeutic and diagnostic tasks. However, undersampling during MRI acquisition as well as the overparameterized and non-transparent nature of deep learning (DL) leaves substantial uncertainty about the accuracy of DL reconstruction. With this in mind, this study aims to quantify the uncertainty in image recovery with DL models. To this end, we first leverage variational autoencoders (VAEs) to develop a probabilistic reconstruction scheme that maps out (low-quality) short scans with aliasing artifacts to the diagnostic-quality ones. The VAE encodes the acquisition uncertainty in a latent code and naturally offers a posterior of the image from which one can generate pixel variance maps using Monte-Carlo sampling. Accurately predicting risk requires knowledge of the bias as well, for which we leverage Stein's Unbiased Risk Estimator (SURE) as a proxy for mean-squared-error (MSE). Extensive empirical experiments are performed for Knee MRI reconstruction under different training losses (adversarial and pixel-wise) and unrolled recurrent network architectures. Our key observations indicate that: 1) adversarial losses introduce more uncertainty; and 2) recurrent unrolled nets reduce the prediction uncertainty and risk.
I. INTRODUCTION
Deep-learning MRI reconstruction can produce biased predictions and artifacts on unseen patients, motivating tools to assess reconstruction risk. The paper introduces probabilistic VAE reconstruction and SURE-based risk estimation, against a background of limited uncertainty analysis for medical image recovery.
I. INTRODUCTION
- Deep-learning reconstructions may be biased toward training data and miss novel pathology, while artifacts can mislead radiologists and affect diagnoses.The paper motivates automated risk assessment as part of the imaging pipeline.Source: §I · p. 1
- The study develops a probabilistic VAE for MR recovery, uses Monte Carlo sampling to analyze output variation, and applies SURE as an MSE surrogate when ground truth is unavailable.The Monte Carlo approach requires ground truth to assess bias directly, motivating the use of SURE.Source: §I · p. 1
- Adversarial loss increases pixel uncertainty while better recovering high frequencies, whereas cascaded networks produce higher-confidence reconstructions and SURE approximates MSE without ground truth.These are the paper's key reported observations across its reconstruction and risk-assessment methods.Source: §I · p. 1
II. RELATED WORK
- GAN-based reconstruction is noted for image quality and realistic detail, while VAE probabilistic interpretability had not yet been demonstrated for medical images.The paper situates its VAE approach against these strands of prior work.Source: §II · p. 2
- Prior uncertainty work includes Monte Carlo output statistics, invertible networks, bootstrapping, and data-based error maps, but these approaches differ in model scope and data requirements.Some bootstrap and jackknife methods can identify risky regions without ground truth.Source: §II · p. 2
- SURE has been used in imaging regularization and denoising but had not been widely applied to uncertainty estimation or medical imaging.The paper explores SURE in that less-established setting.Source: §II · p. 2
III. PRELIMINARIES AND PROBLEM STATEMENT
MRI reconstruction from undersampled k-space is ill-posed and requires both realistic-image priors and measurement consistency. This work learns a projection onto their intersection and evaluates uncertainty in its reconstructions.
III. PRELIMINARIES AND PROBLEM STATEMENT
- Undersampling accelerates MRI acquisition but makes recovery of the true image from k-space measurements challenging.The paper notes diagnostic-guided surgery and pediatric scanning as potential applications of accurate, rapid reconstruction.Source: §III · p. 2
- High-quality recovery requires intersecting the realistic-image manifold with the data-consistent subspace, where multiple admissible solutions may have different likelihoods.The paper illustrates the admissible solutions in Fig. 2.Source: §III · p. 2
- On data unlike training examples, deep-learning models can produce realistic hallucinated artifacts that mislead radiologists and risk incorrect diagnoses.This motivates analysis of uncertainty and robustness in MR imaging.Source: §III · p. 2
- The objective is to learn a deep-learning projection onto the intersection of the real-image manifold and data-consistent subspace, then evaluate reconstruction uncertainty.Source: §III · p. 2
IV. VAES FOR MEDICAL IMAGE RECOVERY
The paper uses a VAE to model realistic medical images probabilistically, then enforces data consistency on generated reconstructions. It also studies cascaded VAE and data-consistency blocks as a recurrent architecture.
IV. VAES FOR MEDICAL IMAGE RECOVERY
- The VAE learns a distribution of realistic images, enabling latent-code sampling and data-consistent exploration of the intersection between image priors and measurements.The paper presents this as a medical-image recovery application of VAEs.Source: §IV · p. 3
- The encoder estimates a Gaussian posterior over a low-dimensional latent variable, and decoding sampled latent codes produces reconstructions under a unit-normal prior.Source: §IV · p. 3
- Training balances pixel-wise ℓ2 reconstruction error against a KL-divergence term that encourages the posterior to follow the unit-normal prior.Increasing the KL weight preserves latent-code sampling structure at the expense of reconstruction quality.Source: §IV · p. 3
- At test time, sampled latent codes are decoded and an affine projection based on the undersampling mask enforces data consistency on every output.The paper reports data consistency as essential to obtaining high SNR.Source: §IV · p. 3
- Cascaded models repeat VAE and data-consistency stages with shared weights, while the one-block model is the unrepeated baseline.The architecture is examined to assess robustness and can improve reconstruction quality according to prior work.Source: §IV · p. 3
V. UNCERTAINTY ANALYSIS
The uncertainty analysis combines Monte Carlo sampling of VAE reconstructions with SURE, because variance maps alone do not capture bias. Monte Carlo yields pixel-wise statistics, while SURE addresses risk without requiring ground truth.
V. UNCERTAINTY ANALYSIS
- Monte Carlo variance maps show which pixels vary across reconstructions, but do not capture bias against the fully sampled image.Bias-dependent statistics require ground-truth knowledge, motivating SURE.Source: §V · p. 3, A. Monte Carlo sampling · p. 4
A. Monte Carlo sampling
- Monte Carlo output statistics have seen limited use in inverse problems and medical image recovery, despite their potential to characterize model uncertainty.Source: A. Monte Carlo sampling · p. 4
- For each aliased input, Monte Carlo sampling draws latent codes from encoder statistics, decodes them, and aggregates k reconstructions into pixel-wise mean and variance maps.The method analyzes the output distribution for a fixed input.Source: A. Monte Carlo sampling · p. 4
B. Denoising SURE
SURE provides a ground-truth-independent proxy for reconstruction MSE, using assumptions about the noise and a degrees-of-freedom term based on network sensitivity. The section also describes approximations intended to make risk estimation practical.
B. Denoising SURE
- The SURE derivation assumes zero-mean Gaussian noise relating the zero-filled input to the ground-truth image.The text states that under this assumption Stein’s formula is used to estimate risk.Source: B. Denoising SURE · p. 4
- SURE estimates reconstruction risk without the ground-truth image by expressing it through residual and degrees-of-freedom terms.The degrees-of-freedom term uses the trace of the end-to-end network Jacobian, which measures sensitivity to small input perturbations.Source: B. Denoising SURE · p. 4
- The method estimates noise variance assuming reconstruction error is small and approximates the Jacobian trace to reduce computation.The trace approximation uses a Gaussian noise vector and a small perturbation value; the authors report validating their assumption when undersampling is not very large.Source: B. Denoising SURE · p. 4
C. Gaussian noise with density compensation
Density compensation is introduced to make the residuals more compatible with SURE’s zero-mean Gaussian-noise assumption. The method forms a compensated input, reconstructs from it, and estimates risk without access to ground truth.
C. Gaussian noise with density compensation
- Density compensation enforces zero-mean residuals, makes artifacts independent of the underlying image, and significantly increases residual normality.These properties address SURE’s zero-mean Gaussian-noise assumption.Source: C. Gaussian noise with density… · p. 4
- The practical procedure reconstructs from the compensated input, then estimates noise variance, approximates degrees of freedom, and obtains SURE risk.The text notes that ground truth is unavailable in practice, so the method relies on an approximation for noise variance.Source: C. Gaussian noise with density… · p. 5
- The compensated zero-filled image is formed by applying the inverse sampling-density weighting before inverse Fourier transformation.The passage defines the density-compensated image using the sampling mask and its expected density.Source: C. Gaussian noise with density… · pp. 4–5
- Generalized SURE can handle colored noise distributions from the exponential family, but the text says it is harder to compute in practice.Source: C. Gaussian noise with density… · p. 5
VI. EMPIRICAL EVALUATIONS
The evaluations use Knee MR images and a patient-stratified split, with aliased inputs created using five-fold variable-density undersampling. The dataset comprises volumes from 19 patients scanned on a 3T GE MR750.
VI. EMPIRICAL EVALUATIONS
- The Knee MRI dataset contains volumes from 19 patients scanned with a 3T GE MR750 scanner.Source: §VI · p. 5
- Each volume has 320 two-dimensional slices of size 320 × 256, split into training, validation, and test sets at 70/15/15 by patient.Source: §VI · p. 5
- A five-fold variable-density undersampling mask with radial view ordering produces aliased inputs for reconstruction.The mask is designed to preserve low-frequency structural elements.Source: §VI · p. 5
A. Adversarial Loss
The adversarial setup pairs a VAE generator with a discriminator to encourage realistic high-frequency detail. Increasing GAN loss enhances realistic components but also raises uncertainty.
A. Adversarial Loss
- The VAE generator is trained against a multi-layer CNN discriminator that distinguishes reconstructions from fully sampled images and feeds back to the generator.The authors use adversarial loss to better capture high-frequency details.Source: A. Adversarial Loss · p. 5
- Adding adversarial training leaves the VAE training process otherwise unchanged, with a weighted discriminator-feedback loss term.Source: A. Adversarial Loss · p. 5
- As GAN loss weight λ increases, modeling of realistic image components is enhanced but uncertainty rises.Source: A. Adversarial Loss · p. 6
B. Network architecture
The architecture combines a VAE generator with an optional CNN discriminator and can repeat the VAE in recurrent blocks. In sample reconstructions, more recurrent blocks improve quality, while increasing GAN loss sharpens outputs but introduces artifacts and lowers SNR.
B. Network architecture
- The VAE encoder uses four convolutional layers and represents latent-space mean and standard deviation with fully connected layers; the decoder upsamples through five transpose-convolution layers.Skip connections are used to improve gradient flow.Source: B. Network architecture · p. 6
- When adversarial loss is used, an 8-layer CNN discriminator distinguishes reconstructions from fully sampled images and feeds its feedback to the VAE generator.Multiple recurrent blocks repeat the VAE using the same model parameters without changing the discriminator architecture.Source: B. Network architecture · p. 6
- Figure 6 compares mean reconstruction, pixel-wise variance, bias, and error across one-, two-, and four-block configurations under five-fold undersampling without adversarial loss.Source: Figure 6 · p. 7
C. Individual reconstructions
- Adding a second recurrent block improves sample reconstructions by roughly 1 dB SNR; increasing GAN loss sharpens outputs but introduces artifacts and lowers average SNR.With limited GAN loss and additional recurrent blocks, SNR is close to 20 dB.Source: C. Individual reconstructions · p. 7
D. Variance, bias, and error maps
The paper examines Monte Carlo uncertainty maps and density-compensated SURE across undersampling rates and model settings. Density compensation improves residual normality, and SURE remains strongly correlated with MSE, while variance, bias, and error vary with GAN loss and recurrent-block count.
D. Variance, bias, and error maps
- Monte Carlo analysis finds that variance, bias, and error increase with higher GAN-loss weight and fewer recurrent blocks; GAN-related variance extends to image structures.A reasonably conservative GAN weight is associated with substantially lower risk, and more recurrent blocks can lower variance and error.Source: D. Variance, bias, and error maps · p. 7
E. Noise distribution with density compensation
- Density compensation makes residuals better match a zero-mean normal distribution, supporting use of denoising SURE across the tested undersampling rates.Without compensation, residuals are not perfectly normal, which can limit SURE's effectiveness and accuracy.Source: E. Noise distribution with density… · p. 8, Figure 7 · p. 8
F. SURE results
- R^2 is 0.97 for 2-fold and 0.84 for 16-fold undersampling in SURE–MSE correlations, which remain strongly linear across tested conditions.The paper reports that SURE can effectively estimate reconstruction risk even with relatively high undersampling.Source: Figure 8 · p. 8, F. SURE results · p. 9
- Across hyperparameters, increased GAN loss lowers SURE values while increasing RSS and DOF; more recurrent blocks yield higher SURE values and lower RSS and DOF.These trends align with the Monte Carlo analysis.Source: F. SURE results · p. 9
- Variance maps localize uncertainty, while SURE provides a more statistically sound global metric; the paper reports a relationship between SURE values and variance magnitude.Source: F. SURE results · p. 9, Figure 9 · p. 9
VII. CONCLUSIONS
The paper presents a probabilistic VAE and Monte Carlo sampling for pixel-wise uncertainty maps, alongside density-compensated SURE for assessing reconstruction risk. Its Knee MRI evaluations find increased uncertainty with greater adversarial loss and decreased uncertainty with multiple recurrent blocks; the authors identify broader uncertainty sources and local risk estimation as future work.
VII. CONCLUSIONS
- The VAE posterior supports Monte Carlo pixel-variance maps, while density-compensated SURE estimates risk under zero-mean Gaussian residuals without requiring fully sampled ground truth.The methods are evaluated on Knee MR images.Source: §VII · p. 9
- Increased adversarial loss is associated with larger uncertainty, whereas multiple recurrent blocks decrease uncertainty and suggest an approach to promoting robustness.Source: §VII · p. 9
- The study focuses on model uncertainty; data and knowledge uncertainty, different MRI datasets and acquisition strategies, and pathological cases remain future directions.Source: §VII · p. 9
- The authors propose extending SURE from global risk to local, pixel-wise risk so it could provide spatial information in uncertainty maps.Source: §VII · p. 9