Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging

1 Introduction

• Scientific imaging uses image-to-image regression for tasks such as algorithmically improving imaging resolution, but researchers need ways to detect incorrect predictions.The introduction notes that these systems may fail silently and are often deployed without statistical reliability measures.Source: §1 · p. 1
• The MRI example shows a reconstructed hallucination alongside high uncertainty at that patch, where the feature is absent from the ground truth.The figure maps an undersampled knee image to a sharp reconstruction and calibrated uncertainty; red indicates high and blue low uncertainty.Source: Figure 1 · p. 2
• The method provides rigorous uncertainty guarantees for any image dataset and regression model, regardless of the number of data points used to construct the interval.It is designed to alert scientists to potential hallucinations in image-valued predictions.Source: §1 · pp. 1–2
• Pixelwise quantile regression consistently performs best among the evaluated uncertainty algorithms, often by a large amount.The paper reports smaller uncertainty intervals than other methods across its imaging tasks.Source: §1 · p. 2

1.1 Notation and Goal

• The method constructs an interval around each predicted pixel, with coverage set by user-chosen risk α and error δ for a fresh test point from the calibration-data distribution.With probability at least 1 − δ, the intervals contain at least 1 − α of the ground-truth pixel values.Source: §1.1 · pp. 2–3
• The interval procedure is modular, allowing users to retain their chosen predictor while adding intervals that communicate prediction quality.The paper says users can use complex, cutting-edge methods to learn the predictor.Source: §1.1 · p. 3

2 Methods

• The two-phase procedure learns heuristic lower and upper widths, scales them using calibration data, and returns a Risk-Controlling Prediction Set for each new image.The scaling factor is shared across the heuristic widths, yielding the final intervals.Source: §2 · pp. 3–4

2.1 Picking a Heuristic Notion of Uncertainty

• The method compares four heuristics: residual magnitude, a Gaussian standard deviation, softmax distributions over pixel values, and pixelwise quantile regression.Each heuristic is trained with its own loss, and calibration is applied afterward because the heuristics may be inaccurate.Source: §2.1 · p. 4

2.1.1 Magnitude of the Residual

• Residual-magnitude fitting targets each pixel's absolute prediction error, but yields only symmetric intervals and may inflate set size or underestimate test errors under overfitting.Without RCPS calibration it provides no guarantee of a valid prediction set; an extra data split can address the stated overfitting concern.Source: §2.1.1 · p. 5

2.1.2 One Gaussian Per Pixel

• The Gaussian heuristic models each pixel with the prediction as its mean and a learned standard deviation, but produces symmetric intervals without coverage guarantees absent strong assumptions.Unlike residual magnitude and quantile regression, it cannot use data splitting to avoid overconfidence due to overfitting.Source: §2.1.2 · p. 5

2.1.3 Softmax Outputs

• The softmax heuristic discretizes pixel values into K bins and learns a distribution at each pixel, but its endpoints are restricted to multiples of 1/K.For K=256, the model output has size M × N × 256, which can make memory infeasible for large images.Source: §2.1.3 · pp. 5–6

2.1.4 Pixelwise Quantile Regression

• Pixelwise quantile regression estimates the α/2 and 1−α/2 conditional quantiles to form lower and upper heuristic interval widths.For a 90% interval, the passage describes using the estimated 5% and 95% quantiles.Source: §2.1.4 · p. 6

2.2 Calibrating Heuristic Notions of Uncertainty

• Calibration selects the smallest scaling factor whose upper confidence bound on the fraction of pixels outside the intervals falls below α, and the resulting set satisfies the stated RCPS guarantee.The guarantee holds except with probability δ over the calibration data; the bound is computed using Hoeffding's upper-confidence bound.Source: §2 · p. 4, §2.2 · pp. 6–7

3.1 Evaluation Metrics

• Evaluation measures calibration risk, average interval size, size-stratified risk, and validation MSE of the jointly trained point prediction.Smaller intervals are more actionable; similar risk across interval-size quartiles checks whether difficult pixels are systematically missed.Source: §3.1 · p. 7

3.2 Experimental Details

• All heuristics share a training and evaluation pipeline; experiments use an 8-layer U-Net, 10 epochs, and RCPS settings α = δ = 0.1.The softmax heuristic uses K = 50 because larger choices require too much memory.Source: §3.2 · p. 8

3.3 Quantitative Phase Microscopy of Leukocytes

• QPI predicts reference phase maps from two obliquely illuminated leukocyte images, reducing inputs relative to four-image DPC while retaining interpretable phase values.The BSCCM split uses 1,800 training, 100 calibration, and 100 validation images.Source: §3.3 · p. 9
• Risk was controlled across QPI, MRI, and TEM; quantile regression led QPI/MRI metrics, while TEM intervals were reasonable and roughly balanced.Source: §3.3 · p. 9, Figure 4 · p. 9, §3.4 · p. 10

3.4 Fast Magnetic Resonance Imaging

• FastMRI maps images reconstructed from fourfold k-space undersampling to fully sampled images, using knee slices split across training, calibration, and validation.The experiment uses 27,993 training, 3,474 calibration, and 3,474 validation slices.Source: §3.4 · p. 10

3.5 Drosophila Brain Transmission Electron Microscopy

• TEM super-resolution maps fourfold-downsampled 4 nm images to their originals; the 26 TB fly-brain dataset was evaluated only with pixelwise quantile regression.Roughly 2 million images were used for training, with 25,000 each for calibration and validation.Source: §3.5 · pp. 10–11
• In TEM, intervals become zero-length every fourth pixel because those target values are known from the downsampled input; high-contrast regions appear more uncertain.The results illustrate intervals identifying both uncertain regions and locations where the model performs reliably.Source: §3.5 · p. 11

• Softmax is prohibitively memory-intensive and slightly worse than quantile regression; Gaussian training is harder, and GaussianNLL harms prediction and heuristic quality.Source: §4 · pp. 11–12
• Pixelwise quantile regression is recommended for asymmetric intervals and better empirical performance; residual-magnitude regression is another strong-performing method.Source: §4 · p. 12

5 Related Work

• Image-to-image regression developed from classical interpolation and spline methods to learning-based approaches, and now also includes denoising, style transfer, and colorization.The paper builds on U-Net work adapting these techniques to biomedical imaging.Source: §5 · p. 12
• Quantile regression, proposed by Koenker and Bassett, has an asymptotic conditional-coverage guarantee under certain weak regularity conditions.Source: §5 · p. 12
• Bayesian and approximately Bayesian methods, including MC-Dropout, and ensemble methods [63] are common uncertainty heuristics without finite-sample guarantees.The paper places these heuristics outside its discussion scope.Source: §5 · p. 12
• Conformal prediction provides finite-sample coverage from any heuristic without retraining, and the paper builds on conformalized quantile regression (CQR) using fixed-sequence testing [13].The authors are unaware of prior work studying distribution-free uncertainty quantification for image-to-image regression.Source: §5 · p. 12