Source-linked AI summary
Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution
Ryutaro Tanno, Daniel E. Worrall, Aurobrata Ghosh, Enrico Kaden, Stamatios N. Sotiropoulos, Antonio Criminisi, Daniel C. Alexander
TL;DR
Medical image super-resolution is ill-posed, so CNNs need ways to represent uncertainty in their predictions. The paper combines heteroscedastic noise modeling with variational dropout in 3D CNNs, achieving strong diffusion-MRI reconstruction results, tractography benefits, and uncertainty maps that may flag inaccurate or unfamiliar regions.
Problem
Ill-posed low-resolution-to-high-resolution medical image mappings create ambiguity, while conventional enhancement methods typically provide no measure of prediction reliability.
Method
The paper extends subpixel CNN super-resolution to 3D and combines per-patch heteroscedastic noise modeling for intrinsic uncertainty with variational dropout for parameter uncertainty.
Results
Combining heteroscedastic modeling and variational dropout produces the top two interior-region performances on HCP and Lifespan data and reduces exterior errors to baseline-comparable or better levels.
Takeaways & Limitations
Uncertainty maps correlate highly with reconstruction errors and highlight pathology, while reduced reconstruction errors benefit downstream tractography.
Abstract
from arXiv · showhide
In this work, we investigate the value of uncertainty modeling in 3D super-resolution with convolutional neural networks (CNNs). Deep learning has shown success in a plethora of medical image transformation problems, such as super-resolution (SR) and image synthesis. However, the highly ill-posed nature of such problems results in inevitable ambiguity in the learning of networks. We propose to account for intrinsic uncertainty through a per-patch heteroscedastic noise model and for parameter uncertainty through approximate Bayesian inference in the form of variational dropout. We show that the combined benefits of both lead to the state-of-the-art performance SR of diffusion MR brain images in terms of errors compared to ground truth. We further show that the reduced error scores produce tangible benefits in downstream tractography. In addition, the probabilistic nature of the methods naturally confers a mechanism to quantify uncertainty over the super-resolved output. We demonstrate through experiments on both healthy and pathological brains the potential utility of such an uncertainty measure in the risk assessment of the super-resolved images for subsequent clinical use.
1 Introduction and Background
Medical imaging can provide richer information, but scan cost, acquisition time, and poor image quality hinder clinical translation and downstream analysis. Existing CNN enhancement methods generally make single predictions without reliability measures, motivating Bayesian CNN-based image quality transfer with uncertainty estimation.
- High imaging cost and lengthy acquisition can hinder clinical translation, while poor image quality may reduce analysis accuracy and contribute to false clinical conclusions.
- Supervised image enhancement methods learn mappings from low-quality clinical images to high-quality experimental images, including random-forest super-resolution of brain diffusion tensor images.
- CNNs have been applied to MRI super-resolution and contrast-resolution enhancement, but some approaches require clinically impractical long-scan training data.
- Current methods typically provide a single prediction without a measure of prediction reliability, whereas Bayesian IQT uses a Bayesian random forest to quantify predictive uncertainty.
- The paper proposes Bayesian image quality transfer with CNNs, extending a 2D subpixel CNN to 3D volumes and designing architectures that estimate distinct uncertainty components.
- The study evaluates CNN accuracy and uncertainty modeling on HCP and Lifespan data, tractography effects on MAP-MRI, and uncertainty maps in healthy and tumour brains.
2 Method
The method performs patch-wise 3D super-resolution with an efficient subpixel CNN and models both intrinsic and parameter uncertainty. Heteroscedastic likelihoods estimate spatially varying output uncertainty, while variational dropout approximates Bayesian parameter inference through repeated samples.
- Patch-wise 3D super-resolution: The method formulates super-resolution as patch-wise regression, splitting a low-resolution image into overlapping sub-volumes that are sequentially enhanced.
- Baseline 3D-ESPCN: The baseline 3D-ESPCN performs convolutions in low-resolution space before upsampling, reducing feature-map computation and memory costs compared with high-resolution refinement.
- Baseline 3D-ESPCN: The 3D shuffling operation remaps channels into spatial locations, producing an upsampled image and implementing learned interpolation after the final convolution.
- Baseline 3D-ESPCN: Each low-resolution subvolume predicts a corresponding high-resolution subvolume, and shifted predictions are tessellated to reconstruct the whole high-resolution volume.
- Intrinsic uncertainty: The baseline minimizes per-pixel MSE, equivalent to Gaussian negative log likelihood with isotropic variance σ2, treating high-resolution patches as deterministic functions of low-resolution patches corrupted by noise.
- Intrinsic uncertainty: The heteroscedastic model replaces constant uncertainty with a covariance estimated by a second 3D-ESPCN, using its diagonal to quantify intrinsic uncertainty across output components.
- Intrinsic uncertainty: The heteroscedastic loss combines a covariance-weighted MSE term with a log-determinant term that controls the spread of the estimated covariance.
- Parameter uncertainty: Variational dropout models parameter uncertainty by injecting learned noise into weights and estimating predictive mean and covariance from samples of the approximate posterior.
3 Experiments and Results
The experiments compare CNN-based and established upsampling methods for DTI super-resolution across reconstruction accuracy, tractography, and predictive uncertainty. Combining heteroscedastic modeling with variational dropout improves regional performance, supports sharper tractography, and produces uncertainty maps that track reconstruction error and flag pathology.
- Performance comparison: The evaluation tests ×2 DTI upsampling on HCP and Lifespan data, measuring RMSE separately in interior and exterior regions because boundary estimation differs and matters for tractography.The Lifespan data test generalisability across acquisition protocols, lower resolution, and a different age range.
- Performance comparison: 8.5%/39.8% reduction in RMSE on HCP interior/exterior regions is achieved by 3D-ESPCN relative to BIQT-RF, while whole-brain reconstruction takes <10 s on CPU and ∼1 s on GPU.The CNN also outperforms RF methods on both regions in the Lifespan dataset.
- Performance comparison: Heteroscedastic modeling significantly improves interior performance on both datasets (p < 10^-3), whereas variational dropout best handles exterior performance and reduces its errors toward baseline levels.Combining heteroscedastic modeling with variational dropout yields the top two interior results across both datasets.
- Performance comparison: The loss function emphasizes lower-noise regions, while variational dropout reduces exterior error through regularization against overfitting.The model allocates more resources to regions where the LR-to-HR mapping is less ambiguous.
- Tractography with MAP-MRI SR: On Prisma tractography, CNN reconstruction avoids a false-positive tract and recovers small gyral white-matter pathways more sharply than linear interpolation and RF-IQT.The experiment super-resolves MAP-MRI coefficients from 2.5 mm data before predicting 1.25 mm HR DWIs.
- Visualisation of predictive uncertainty: Uncertainty maps from 200 predictive samples are highly correlated with error maps, capture subtle MD variations, and show high uncertainty in a tumour absent from training data.The tumour case lacks ground truth, but the uncertainty map flags the tumour as a potential low-accuracy area.
4 Discussion
The paper presents 3D subpixel-CNN super-resolution for diffusion MRI with strong accuracy and reconstruction efficiency. It links combined intrinsic and parameter uncertainty modeling to predictive performance and uncertainty maps that correlate with errors and highlight pathology, while identifying unfamiliar test environments as future work.
- Discussion: The proposed 3D subpixel-CNN achieves state-of-the-art accuracy and reconstruction efficiency on diffusion MRI datasets.The discussion frames the method as a super-resolution algorithm for diffusion MRI.
- Discussion: Applying the method to MAP-MRI coefficients benefits downstream tractography compared with previous methods.
- Discussion: Combining intrinsic and parameter uncertainty leads to the best predictive performance, while uncertainty maps correlate highly with reconstruction errors and highlight pathologies.
- Discussion: Understanding uncertainty behavior in unfamiliar environments such as pathologies remains important future work for developing a more generalisable method.