Interval Neural Networks: Uncertainty Scores

1. Introduction

• DNNs support computational imaging tasks including image translation, super-resolution, denoising, and image synthesis.Medical inverse problems such as CT and MRI are identified as promising application areas.Source: §1 · p. 1
• DNN opacity limits prospects for adoption in medical settings.The introduction notes that optimization and further automation could positively affect the quality and coverage of medical care.Source: §1 · p. 1
• The paper presents INN, a non-Bayesian, optimization-based uncertainty-quantification method intended to alleviate limitations in existing methods.It is an interval-valued network trained to encompass targets while remaining bound to the underlying prediction network.Source: §1 · p. 2

2. Related Work

• Monte Carlo dropout estimates a DNN prediction's mean and variance from stochastic inference passes, while ProbOut trains a network to estimate Gaussian mean and variance.Both are used as baselines in this paper's experiments.Source: §2 · p. 2
• Popular DNN uncertainty methods have Bayesian roots, while other approaches use confidence intervals, including conformal prediction.The passage notes conformal prediction's exchangeability assumption may rarely hold in practical settings.Source: §2 · p. 2
• Most deep-learning inverse-problem approaches focus on empirical reconstruction performance rather than uncertainty quantification.A cited exception uses a Bayesian framework to consider reconstruction uncertainties [1].Source: §2 · p. 2

3. Research Needs

• Existing DNN uncertainty scores do not reveal whether an output may be higher or lower, and can become detached from the prediction model under data perturbations.A confident prediction may still be right or wrong.Source: §3 · pp. 2–3
• Figure 1 shows point-interval inputs propagated through interval-valued weights and biases to produce lower and upper output bounds for pixel-wise uncertainty scores.The intervals contain the underlying network's prediction at each layer.Source: Figure 1 · p. 3
• INN propagates intervals to form an output-space box that contains the prediction and is fitted to encompass target data.Its scores are described as providing prediction-error and directional information, greater robustness to input and output noise, analytic coverage bounds, and competitive or improved runtime.Source: §3 · p. 3
• The paper motivates uncertainty quantification through medical-imaging inverse problems, where reconstruction requires recovering an unknown signal from measurements.The measurement model represents observations as a forward operation on the signal plus noise; examples include denoising, MRI, and CT.Source: §3 · p. 3
• Deep-learning reconstructions can outperform earlier model-based regularization schemes, but lack theoretical recovery guarantees and may be susceptible to unwanted artifacts.The paper identifies reliability information as indispensable for sensitive medical-imaging applications.Source: §3 · p. 4

4. Interval Neural Networks

• The INN uses the underlying DNN's architecture and initializes its parameters as point intervals.Input components are also treated as point intervals before propagation.Source: §4 · p. 4
• Constraining INN parameter intervals to contain the underlying model's parameters guarantees that its output interval contains the underlying network's prediction.Output interval size therefore upper-bounds absolute error.Source: §4 · p. 4
• INN propagates interval-valued inputs through interval-valued weight matrices using interval arithmetic, with a simplified operation for nonnegative activations such as ReLU.The passage also describes a representation for point-interval inputs that can process negative values.Source: §4 · pp. 4–5
• The interval loss penalizes targets outside bounds by squared distance to the interval boundary and adds a linear penalty on interval size.Without the size penalty, output intervals would expand to cover the whole target range.Source: §4 · p. 5
• The tightness parameter β controls outlier sensitivity, and choosing it near the underlying model's mean absolute error is described as a practical heuristic.The interval-parameter bounds are optimized by backpropagation in existing deep-learning frameworks.Source: §4 · p. 5

5. Qualities of Interval Neural Networks

• INNs are presented as offering theoretically justified coverage bounds, adaptive behavior under noise, and directional information.For very deep INNs, interval growth can create training stability issues; the authors suggest a small learning rate or training interval parameters only in the last several layers.Source: §5 · pp. 5, 8
• The illustrative 1DDeconv experiment uses simulated blurred measurements from piecewise-constant signals and divides 2,000 pairs into training, validation, and test sets.The split is 1,600 training, 200 validation, and 200 test pairs.Source: §5 · p. 5
• 89% of ground-truth values fell inside INN intervals on 1DDeconv, with coverage bounds justified under an expected-zero-gradient assumption for the training distribution.The authors note that the training set must sufficiently represent the true data distribution.Source: §5 · p. 7
• With Gaussian noise (σ = 0.05), INN intervals capture deviations that MCDrop misses, and mean interval size increases as noise levels rise.The authors attribute noise capture to interval bias parameters in the last layer.Source: Figure 2 · p. 6, §5 · p. 7, Figure 3 · p. 8
• INN directional uncertainty yields direction accuracy 12 to 25 percentage points above chance, unlike symmetric MCDrop and ProbOut scores.INNs require 2 × K operations, compared with T × K for MCDrop and C + K for ProbOut; for K ≫ C, INNs lie between them.Source: §5 · p. 8

6.3. Experiment.

• The Error Correlation experiment evaluates how uncertainty scores correspond to prediction error, using interval size for INN and standard deviation for MCDrop and ProbOut.Its performance-weighted correlation coefficient weights correlation by mean squared error to discourage rewards for poor prediction models with high uncertainty everywhere.Source: §6 · p. 8, §6.3 · p. 10

6.1. Data.

• The experiment uses 512 × 512 human CT scans from ten patients, split into eight training patients, one validation patient, and one test patient.The data come from the AAPM Low Dose CT Grand Challenge; the Radon transform and filtered backprojection use a thirty-degree missing wedge.Source: §6.1 · p. 10

6.2. Neural Network Architectures and Training Procedures.

• All methods use a U-Net prediction model; INN interval parameters are trained for 15 epochs on the last twelve layers, while ProbOut is trained for 400 epochs and MCDrop uses 128 forward passes.The UQ methods are evaluated over four training runs.Source: §6.2 · p. 10, §6.3 · p. 10

6.4. Results.

• On CT examples, INN and MCDrop identify an error-prone region and an edge artifact, while INN also highlights regions with high local intensity variations.Figure 5 displays input and target, predictions, uncertainty scores, and absolute errors for the methods.Source: §6.4 · p. 10, Figure 5 · p. 11
• 76 ± 6% of CT test targets fall inside INN intervals, while directional accuracy rises from 57% to 72% as the interval-direction threshold grows.Figure 4 reports the CT results as means and standard deviations across four experimental runs.Source: Figure 4 · p. 9, §6.4 · p. 12

• The paper introduces a non-Bayesian interval-propagation method that computes upper and lower bounds and uncertainty maps for pre-trained neural networks.Source: §7 · p. 12
• In comparisons on a CT reconstruction data set, interval neural networks give the most consistent results among the methods analyzed.Source: §7 · p. 12
• INNs provide additional insights about interval coverage, noise behavior, and directional information for the ground truth.Source: §7 · p. 12

Appendices

• The supplement adds a formal derivation of the coverage-bound argument, a Bayesian treatment of INNs, and further randomly sampled results from the 1D Deconvolution and CT reconstruction experiments.Source: Appendices · p. 15

A. INN Coverage Bound

• The coverage-bound derivation applies the Markov inequality to bound the probability that an input’s target is outside the interval bounds.The supplement identifies this as a formal derivation of the main paper’s argument.Source: A. INN Coverage Bound · pp. 15–16
• For λ > 0 and α > 0, at least 1 − 1/(λα) of the input probability mass has targets with probability at least 1 − α of lying inside the interval.Source: A. INN Coverage Bound · p. 16

B. INNs and the Bayesian View

• Variational Bayesian methods approximate the posterior over network parameters with qθ and train by maximizing the evidence lower bound (ELBO).The section describes weights as random variables and gives a Gaussian likelihood whose mean is the network output.Source: B. INNs and the Bayesian View · pp. 16–17
• The INN view allows qθ to be any distribution of weights and biases supported within the specified interval parameters.The interval parameters are upper and lower bounds for an L-layer network’s weights and biases.Source: B. INNs and the Bayesian View · p. 17
• For a fixed input, the interval bounds define the range of possible network outputs under weights distributed according to qθ.The passage introduces best- and worst-approximation weight choices within this output range.Source: B. INNs and the Bayesian View · p. 17
• With β = 1, minimizing the INN loss corresponds to maximizing a lower bound for one ELBO component; the KL term corresponds to weight regularization.The gap between the ELBO bounds is determined by P², and output-interval size corresponds to the gap from the true ELBO when using the INN loss.Source: B. INNs and the Bayesian View · p. 17
• At inference, the true posterior may lie anywhere between the bounds, while the probability of the target lying far outside the INN interval decays quickly.Figure 6 schematically marks the INN prediction interval with vertical lines; the decay observation is stated to align with an Appendix A Markov-bound result.Source: B. INNs and the Bayesian View · p. 18, Figure 6 · p. 18

C. 1DDeconv Experiments: Additional Samples

• Figure 7 contrasts the same 1DDeconv sample without noise and with Gaussian noise (σ = 0.05), showing uncertainty estimates and uncertainty-versus-absolute-error plots for MCDrop, ProbOut, and INN.The upper graphs show target, network output, and uncertainty; the lower graphs plot uncertainty against absolute error.Source: Figure 7 · p. 19
• Figure 8 presents the same 1DDeconv sample without noise and with Gaussian noise (σ = 0.05), with uncertainty estimates and uncertainty-versus-absolute-error plots for MCDrop, ProbOut, and INN.The upper graphs show target, network output, and uncertainty; the lower graphs plot uncertainty against absolute error.Source: Figure 8 · p. 20
• Figure 9 presents the same 1DDeconv sample without noise and with Gaussian noise (σ = 0.05), with uncertainty estimates and uncertainty-versus-absolute-error plots for MCDrop, ProbOut, and INN.The upper graphs show target, network output, and uncertainty; the lower graphs plot uncertainty against absolute error.Source: Figure 9 · p. 21

D. CT Experiments: Additional Samples

• Figure 10 shows CT inputs and targets, predictions, uncertainty estimates, and absolute errors for the uncertainty methods.MCDrop and ProbOut uncertainty is shown as standard deviation, while INN uncertainty is shown as interval size.Source: Figure 10 · p. 22
• Figure 11 shows CT inputs and targets, predictions, uncertainty estimates, and absolute errors for the uncertainty methods.MCDrop and ProbOut uncertainty is shown as standard deviation, while INN uncertainty is shown as interval size.Source: Figure 11 · p. 23
• Figure 12 shows CT inputs and targets, predictions, uncertainty estimates, and absolute errors for the uncertainty methods.MCDrop and ProbOut uncertainty is shown as standard deviation, while INN uncertainty is shown as interval size.Source: Figure 12 · p. 24