PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Evidential Regression

Alexander Amini, Wilko Schwarting, Ava Soleimany, Daniela Rus

arXiv:1910.02600v2cs.LGcs.NEstat.ML

TL;DR

The paper addresses the need for calibrated uncertainty in regression without requiring predefined OOD data or inference-time sampling. It learns evidential distributions for deterministic neural networks, achieving calibrated uncertainty across vision tasks while exposing regularization and prior-selection limitations.

  • Problem

    Regression uncertainty methods need calibrated OOD uncertainty from in-distribution training data, while continuous regression lacks a well-defined evidential regularization distance.

    Source: §1 · p. 2

  • Method

    The method learns evidential distributions over Gaussian regression likelihoods to estimate aleatoric and epistemic uncertainty without sampling or OOD training data.

    Source: §1 · p. 2

  • Results

    The method demonstrates calibrated uncertainty on benchmark and complex vision regression tasks, including OOD and adversarial robustness evaluations; evidential calibration error is 0.033 versus 0.048 for ensembling.

    Source: §1 · p. 2, §4.2 · p. 7, §6 · p. 8

  • Takeaways & Limitations

    The approach provides a scalable, broadly applicable route to uncertainty estimation for regression across temporal forecasting, property prediction, control learning, and vision tasks.

    Source: §1 · p. 2, §6 · pp. 8–9

    The main limitations are tuning the regularization coefficient, removing non-misleading evidence during calibration, and determining the effects of alternative variance priors.

    Source: §6 · p. 9

Abstract

from arXiv · show

Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust, and efficient measures of uncertainty are crucial. In this paper, we propose a novel method for training non-Bayesian NNs to estimate a continuous target as well as its associated evidence in order to learn both aleatoric and epistemic uncertainty. We accomplish this by placing evidential priors over the original Gaussian likelihood function and training the NN to infer the hyperparameters of the evidential distribution. We additionally impose priors during training such that the model is regularized when its predicted evidence is not aligned with the correct output. Our method does not rely on sampling during inference or on out-of-distribution (OOD) examples for training, thus enabling efficient and scalable uncertainty learning. We demonstrate learning well-calibrated measures of uncertainty on various benchmarks, scaling to complex computer vision tasks, as well as robustness to adversarial and OOD test samples.

1 Introduction

The paper addresses uncertainty estimation for deterministic regression networks used in safety-critical settings. It proposes learned evidential distributions to estimate both aleatoric and epistemic uncertainty without sampling or OOD training data.

  • Safety-critical regression applications require calibrated uncertainty to interpret confidence, detect OOD inputs, and recognize likely model failure.Source: §1 · p. 1
  • Aleatoric uncertainty describes data uncertainty, whereas epistemic uncertainty describes uncertainty in the prediction.Source: §1 · p. 1
  • Existing evidential methods primarily target classification and rely on uncertain-prior distances or OOD training, while regression lacks a well-defined regularization distance.Source: §1 · p. 2
  • The proposed approach trains a network to predict higher-order evidential-distribution parameters over likelihood parameters such as (µ, σ2).Source: Figure 1 · p. 1, §1 · p. 2
  • The method learns both uncertainty types without sampling during inference or training on OOD data, and evaluates calibration, robustness, and benchmark performance.Source: §1 · p. 2

2 Modelling uncertainties from data

Deterministic regression commonly predicts an average target and may model aleatoric uncertainty through a Gaussian likelihood, but it does not capture predictive epistemic uncertainty. The paper places higher-order evidential priors over likelihood parameters to estimate both forms of uncertainty.

  • Squared-error regression encourages a model to learn the average correct answer but does not explicitly model noise or uncertainty in the data.Source: §2.1 · p. 2
  • Maximum likelihood regression assumes targets arise from a Gaussian with mean and variance parameters θ = (µ, σ2), optimizing negative log likelihood.Source: §2.2 · p. 2
  • Learning Gaussian likelihood parameters models aleatoric uncertainty but leaves the model oblivious to predictive epistemic uncertainty.Source: §2.2 · p. 2
  • The proposed method directly learns evidence for predictions by placing higher-order prior distributions over parameters governing the observation distribution.Source: §2.2 · pp. 2–3
  • An evidential distribution represents confidence over likelihood parameters, and its realizations can generate lower-order likelihoods over observed data.Source: Figure 2 · p. 3

3 Evidential uncertainty for regression

The method places a Normal Inverse-Gamma evidential prior over Gaussian regression likelihoods and trains neural networks to infer its hyperparameters, representing both uncertainty types without inference sampling. Training combines model-evidence maximization with an error-scaled regularizer that reduces misleading evidence.

Evidential distribution

  • The Normal Inverse-Gamma distribution approximates the posterior over unknown Gaussian mean and variance through factorized Gaussian and Inverse-Gamma components.The prior is parameterized by m = (γ, υ, α, β), with γ ∈ R, υ > 0, α > 1, and β > 0.Source: §3.1 · p. 3
  • Its hyperparameters determine likelihood location and dispersion, while total evidence Φ = 2υ + α counts inferred virtual observations.Increasing υ and α concentrates the evidential distribution around its inferred likelihood function.Source: §3.1 · pp. 3–4

Neural prediction

  • A neural network maps each input to evidential hyperparameters, enabling simultaneous regression, aleatoric uncertainty, and epistemic uncertainty estimation without OOD training data or sampling.The maximum-likelihood Gaussian is computed analytically, and epistemic uncertainty is obtained from the variance of the inferred evidential distribution.Source: §3.1 · p. 4

Learning objective

  • Training maximizes model evidence by marginalizing the Gaussian likelihood parameters under the NIG prior, yielding an analytically evaluable objective.With a NIG prior, the marginal likelihood has an analytical solution and is expressed through a Student-t distribution.Source: §3.3 · p. 4
  • A second objective minimizes evidence on incorrect predictions by scaling an error penalty with total inferred evidence.The regularizer addresses the undefined KL divergence to a zero-evidence NIG prior and avoids penalizing evidence everywhere in the unbounded regression output space.Source: §3.3 · p. 5
  • The combined loss trades off model fit against uncertainty inflation, with λ = 0 producing over-confidence and excessively large λ causing over-inflation.The network outputs four parameters per target and uses constrained activations for υ, α, and β.Source: §3.3 · p. 5

4 Experiments

Experiments compare evidential regression with established uncertainty methods on toy, benchmark, depth-estimation, OOD, and adversarial settings. The results show competitive predictive performance, calibrated uncertainty, and increasing uncertainty under distribution shift and perturbations.

Benchmark regression

  • On benchmark regression tests, evidential models outperform dropout sampling and ensembling on NLL and inference speed across all datasets.RMSE remains competitive despite the loss not explicitly optimizing accuracy.Source: §4.1 · p. 5, Table 1 · p. 6

Monocular depth estimation

  • Depth estimation scales evidential learning to over 27k indoor RGB-to-depth pairs using a U-Net-style neural network tested on disjoint scenes.The task predicts depth at every pixel, making the target high-dimensional.Source: §4.2 · p. 6

Toy uncertainty estimation

  • In toy cubic regression, regularized evidential regression enables precise in-distribution predictions and conservative epistemic uncertainty where training data is absent.The comparison includes Gaussian MLE and ensembling baselines, with and without evidential regularization.Source: §4.1 · p. 5, Figure 3 · p. 6

Monocular depth estimation

  • Evidential depth models localize prediction errors and produce confidence whose increasing levels correspond to steadily decreasing error.Dropout underestimates uncertainty, while ensembling occasionally overestimates it.Source: §4.2 · pp. 6–7
  • Evidential uncertainty achieves calibration error 0.033, outperforming ensembling at 0.048 and dropout at 0.126.Ideal calibration follows y = x.Source: Figure 4 · p. 6, §4.2 · p. 7

OOD uncertainty

  • On unseen OOD driving images, evidential models show a positive entropy shift relative to in-distribution data, separating the uncertainty distributions.The evaluation uses entropy CDFs, interquartile comparisons, histograms, and sample predictions.Source: §4.3 · p. 7, Figure 5 · p. 7

Adversarial robustness

  • Under increasing adversarial noise, evidential entropy rises and its spatial concentrations remain closely aligned with prediction error.The study uses FGSM perturbations and tracks error, epistemic uncertainty, entropy distributions, and pixel-wise visualizations.Source: §4.3.1 · p. 8

5 Related work

The paper situates evidential regression relative to Bayesian neural networks, ensembles, dropout, evidential deep learning, and Prior Networks. Its distinguishing setup places uncertainty priors over predictive likelihoods and avoids sampling or OOD training data.

Prior networks and evidential models

  • Prior evidential and classification methods either require regularization toward a fixed prior, OOD training data, or are limited to discrete classification problems.The paper frames continuous regression as lacking a well-defined distance measure for regularizing the inferred evidential distribution.Source: §5 · p. 8

Bayesian deep learning

  • Bayesian neural networks place priors over weights and use sampling-based approximations, while ensembles and dropout require multiple sampled inferences to estimate predictive variance.These approaches are described as computationally expensive compared with a single-pass deterministic approach.Source: §5 · p. 8

6 Conclusions, limitations, and scope

The paper presents evidential regression as a scalable approach for joint uncertainty estimation and reports calibration, OOD, and vision-task results. It identifies regularization and evidence removal as central limitations, while calling for further study of variance priors.

Conclusions and scope

  • The method places evidential priors over regression likelihood outputs to learn both epistemic and aleatoric uncertainty.The paper positions this as a novel approach for regression problems.Source: §6 · p. 8
  • The authors demonstrate scalability to complex vision tasks, calibrated uncertainty on OOD data, and broad applicability across regression tasks.Examples include temporal forecasting, property prediction, and control learning.Source: §6 · pp. 8–9

Limitations and future work

  • The primary limitations are tuning the regularization coefficient and effectively removing non-misleading evidence during uncertainty calibration.The authors suggest dual optimization and alternative variance priors as directions for further investigation.Source: §6 · p. 9

Broader Impact

The paper emphasizes calibrated uncertainty for neural networks deployed in safety-critical decision systems. It also warns that stronger model performance and uncertainty estimation may increase human trust, motivating continued redundancy.

Societal importance

  • Uncertainty estimation is intended to help interpret confidence, detect OOD domain shift, and recognize when a model is likely to fail in safety-critical applications.The motivation spans computer vision, robotics, control, and downstream decision systems.Source: Broader Impact · p. 9
  • Superior calibration over baselines is presented as supporting trust in algorithms and understanding when they indicate uncertainty.The paper frames this as complementary to improving neural-network precision and accuracy.Source: Broader Impact · p. 9

Societal challenges

  • Greater model performance and uncertainty capabilities may increase human trust, so the authors recommend redundancy to help catch and correct mistakes independently.This is identified as a potential societal challenge of uncertainty-aware systems.Source: Broader Impact · p. 9

S1 Derivations

The derivations model unknown Gaussian mean and variance with a Normal-Inverse-Gamma distribution, whose moments provide prediction and aleatoric and epistemic uncertainty. The KL-based zero-evidence regularizer is not directly evaluable, so the paper uses a small-evidence alternative but finds it highly sensitive.

S2 Benchmark regression tasks

Synthetic regression experiments examine uncertainty estimation, regularization, and disentanglement of aleatoric and epistemic uncertainty. The evidential regularizer controls OOD epistemic uncertainty while preserving aleatoric uncertainty, and separates high noise from epistemic uncertainty in-distribution.

S2.1 Experimental setup

S2.1.3 Impact of the evidential regularizer

  • Reducing the evidential regularizer weight causes OOD uncertainty to decay toward zero, whereas stronger regularization inflates it while maintaining constant aleatoric uncertainty.Source: S2.1.3 Impact of the evidential… · p. 15

S2.1.4 Disentangled uncertainties

S2.1.3 Impact of the evidential regularizer

S3 Depth estimation evaluations

Depth-estimation experiments evaluate evidential models against dropout and ensembles on predictive accuracy, efficiency, calibration, and robustness. The evidential method maintains competitive predictive performance while using fewer parameters and faster sampling-free inference, with uncertainty increasing on OOD and adversarial inputs.

S3.1 Depth estimation setup

S3.2 Depth estimation performance metrics

S3.3 OOD uncertainty

  • Evidential uncertainty shows a positive entropy-CDF shift on OOD images and clear separation between ID and OOD uncertainty distributions.Source: Figure S6 · p. 18

S3.4 Aleatoric uncertainty estimation on depth

Limitations

Related papers

Loading 1910.02600v2…