Source-linked AI summary
Evidential Deep Learning to Quantify Classification Uncertainty
Murat Sensoy, Lance Kaplan, Melih Kandemir
TL;DR
Standard deterministic neural networks can predict accurately while remaining ignorant of prediction confidence, limiting uncertainty estimation for unfamiliar or adversarial inputs. The paper explicitly models class-probability uncertainty with a Dirichlet distribution whose parameters are produced by a deterministic neural network. The method outperforms state-of-the-art Bayesian neural networks on out-of-distribution uncertainty and shows greater robustness to adversarial attacks.
Problem
Deterministic neural networks may achieve high accuracy without representing prediction confidence, leaving uncertainty estimation for unrelated and adversarial inputs inadequately addressed.
Method
The method places a Dirichlet distribution over class probabilities and uses neural-network outputs to learn the evidence underlying these subjective opinions.
Results
The method outperforms state-of-the-art Bayesian neural networks by a large margin, approaches maximum entropy more closely on out-of-distribution inputs, and is more robust to adversarial attacks.
Takeaways & Limitations
The resulting predictor provides a more detailed uncertainty model than the point estimate produced by standard softmax-output networks.
Abstract
from arXiv · showhide
Deterministic neural nets have been shown to learn effective predictors on a wide range of machine learning problems. However, as the standard approach is to train the network to minimize a prediction loss, the resultant model remains ignorant to its prediction confidence. Orthogonally to Bayesian neural nets that indirectly infer prediction uncertainty through weight uncertainties, we propose explicit modeling of the same using the theory of subjective logic. By placing a Dirichlet distribution on the class probabilities, we treat predictions of a neural net as subjective opinions and learn the function that collects the evidence leading to these opinions by a deterministic neural net from data. The resultant predictor for a multi-class classification problem is another Dirichlet distribution whose parameters are set by the continuous output of a neural net. We provide a preliminary analysis on how the peculiarities of our new loss function drive improved uncertainty estimation. We observe that our method achieves unprecedented success on detection of out-of-distribution queries and endurance against adversarial perturbations.
1 Introduction
Deep neural networks achieve high accuracy, but reliable uncertainty estimation remains important for detecting unfamiliar inputs and resisting adversarial attacks. The paper addresses this gap by modeling classification predictions as Dirichlet distributions through a Theory of Evidence perspective.
- Deep learning achieves strong accuracy across applications, increasing the need for robustness, sample efficiency, security, and interpretability.
- High test accuracy does not answer whether a network can recognize unrelated data or say “I do not know” about unfamiliar inputs.Bayesian neural networks address these questions by estimating predictive uncertainty through posterior-predictive moments.
- The paper represents classification predictions as a distribution over possible softmax outputs by replacing categorical parameters with Dirichlet parameters.The resulting density models uncertainty around point estimates and is trained by minimizing a specific loss with standard backpropagation.
- The proposed method outperforms state-of-the-art Bayesian neural networks by a large margin on two uncertainty-modeling applications.It approaches maximum entropy more closely on out-of-distribution inputs and is more robust to adversarial attacks on two benchmark datasets.
- After about 50° of input rotation, the method sharply lowers classification probabilities and raises uncertainty, unlike standard softmax.
2 Deficiencies of Modeling Class Probabilities with Softmax
Softmax converts neural-network activations into point estimates of class probabilities, but its maximum-likelihood training does not model predictive variance. This can produce inflated confidence and unreliable uncertainty, especially when inputs are rotated away from the training setting.
- Softmax maps neural-network outputs to class probabilities on a simplex, and the resulting multinomial likelihood is optimized through negative log-likelihood.The negative log-likelihood is commonly known as cross-entropy loss.
- Cross-entropy training has a maximum-likelihood interpretation and cannot infer the variance of the predictive distribution.
- Exponentiation in softmax can inflate the predicted class probability, making distance from the predicted label uninformative beyond class-to-class comparison.
- When a rotated MNIST digit 1 is misclassified, LeNet assigns it to classes such as 2 or 5 according to the rotation angle.For small rotations, it correctly classifies the digit as 1 with high probability.
- Figure 1 compares softmax classification probabilities with the proposed method’s probabilities and uncertainty across rotations from 0° to 180°.
3 Uncertainty and the Theory of Evidence
The paper uses subjective logic and Dirichlet distributions to represent classification predictions as opinions that include evidence and uncertainty, rather than single probability estimates.
- Theory of Evidence: Subjective Logic formalizes Dempster–Shafer belief assignments as Dirichlet distributions, enabling belief masses and uncertainty to be quantified.The framework assigns masses to individual classes and an overall uncertainty mass to the full frame of possible states.
- Theory of Evidence: For class k, evidence e_k determines belief and uncertainty, with Dirichlet parameters α_k = e_k + 1 and uncertainty inversely proportional to total evidence.When evidence is absent, uncertainty is one and the belief mass for every singleton is zero.
- Theory of Evidence: A Dirichlet distribution represents a density over possible class-probability assignments, thereby modeling second-order probabilities and uncertainty.Standard classifiers output one probability assignment, whereas the Dirichlet representation models distributions over those assignments.
- Theory of Evidence: With no belief mass in a 10-class problem, the prior is D(p|⟨1, . . . , 1⟩), a uniform distribution expressing total uncertainty.The uniform Dirichlet prior contains no observed evidence and no class preference.
- Theory of Evidence: The network interprets α_ij − 1 as evidence for assigning sample i to class j, allowing epistemic uncertainty to be computed from the predicted Dirichlet parameters.The parameters account for the evidence collected for each class.
4 Learning to Form Opinions
The method replaces softmax with non-negative evidence outputs and trains a neural network to form Dirichlet opinions, using a loss that balances data fit with predictive variance and regularized uncertainty.
- Model and Training: The model replaces softmax with a non-negative activation whose output is an evidence vector for the predicted Dirichlet distribution.For sample i, α_i = f(x_i|Θ) + 1, and the Dirichlet mean α_i/S_i estimates class probabilities.
- Model and Training: The Dirichlet parameters are treated as a prior over class probabilities, and the marginal likelihood is minimized with respect to those parameters using Type II Maximum Likelihood.This integrates out the class probabilities before optimizing the resulting negative log marginal likelihood.
- Loss Function: The selected loss was more stable than two alternatives, which produced excessively high belief masses and comparatively less stable performance.The authors chose the final option based on empirical findings and leave theoretical analysis of the alternatives for future work.
- Loss Function: The loss jointly minimizes prediction error and the variance of the Dirichlet distribution generated for each training sample.Its design prioritizes data fit over variance estimation.
- Loss Function: The error term decreases when evidence is added to the correct class and when excessive evidence is removed from an incorrect class.These properties encourage evidence for correct labels while reducing misleading evidence associated with misclassification.
- Loss Function: A KL-divergence term regularizes predictions toward the uniform Dirichlet distribution when divergences do not contribute to data fit.Annealing increases this regularization gradually, helping avoid premature convergence to the uniform distribution for samples that may later be classified correctly.
5 Experiments
The experiments compare EDL with deterministic, dropout, ensemble, and Bayesian uncertainty methods on in-distribution, out-of-distribution, and adversarial settings. EDL remains comparable in MNIST classification while assigning substantially greater uncertainty to unseen CIFAR10 classes.
- Experimental setup: The study compares L2, Dropout, Deep Ensemble, FFG, MNFG, and EDL on MNIST and CIFAR10 prediction uncertainty.Adversarial examples are generated with the Fast Gradient Sign method.
- Figures: Figure 2 varies accuracy with EDL’s uncertainty threshold, while Figures 3–5 visualize entropy-based uncertainty across out-of-distribution and adversarial settings.Figure 3 reports empirical CDFs for notMNIST and held-out CIFAR10 categories; Figures 4 and 5 plot accuracy and entropy against perturbation ϵ.
- In-distribution performance: EDL achieves MNIST test classification performance comparable to competing methods.All methods use the same LeNet-based architecture for the MNIST comparison.
- Uncertainty evaluation: The experiments evaluate uncertainty with predictive entropy and, for adversarial examples, the ratio of entropy to maximum entropy.Maximum entropy is log(10) for MNIST and log(5) for CIFAR5.
- Out-of-distribution detection: EDL assigns much more uncertainty than other methods to CIFAR10 samples from classes excluded during training.Training uses the first five CIFAR10 categories, while testing uses the last five, for which no prediction is correct.
6 Related Work
The related work situates uncertainty-aware prediction in Gaussian processes and Bayesian parameter-uncertainty methods. Bayesian neural networks account for uncertainty through distributions over synaptic weights and posterior predictive moments.
- Gaussian processes: Gaussian processes provide both accurate predictions and uncertainty measures across contexts including transfer learning and deep learning.
- Bayesian approaches: A major uncertainty-modeling approach places prior distributions on parametric model parameters and infers posterior predictive uncertainty.
- Bayesian neural networks: Bayesian neural networks model parameter uncertainty by placing priors on synaptic connection weights.Their predictive uncertainty is derived from higher-order moments of the resulting posterior predictive distribution.
7 Conclusions
The paper models classification predictions as Dirichlet distributions whose parameters come from neural-network outputs. It fits this predictive distribution using Bayes risk with L2 loss and an information-theoretic complexity regularizer.
- Conclusion: EDL assigns a Dirichlet distribution to class probabilities and sets its parameters using neural-network outputs.The resulting predictor provides a more detailed uncertainty model than a point prediction.
- Conclusion: The predictive distribution is fitted by minimizing Bayes risk under L2-Norm loss with an information-theoretic complexity regularizer.
Appendix: Propositions for the Loss Function in (5)
The propositions analyze how the loss changes when evidence is added to or removed from Dirichlet parameters. They establish behavior for correct-class evidence and for removing evidence from an incorrect class with the largest parameter.
- Proposition 2: For a sample with correct label j, Proposition 2 analyzes the error loss Lerr_i and its update after evidence removal.The supplied proposition states positivity under ˆp_il > ˜p_il ≥ ˆp_ik for k ≠ j, with maximization as ˆp_il increases.
- Proposition 2: Adding evidence to the correct-class parameter α_ij decreases Lerr_i, whereas removing evidence from α_ij increases it.
- Proposition 3: Proposition 3 compares Lerr_i before and after removing evidence from α_il through the corresponding expected probabilities.The pre-removal loss is introduced alongside the post-removal comparison.
- Proposition 3: Removing evidence from the largest incorrect-class Dirichlet parameter α_il decreases Lerr_i.The updated expected probability for class l decreases, while the other class probabilities increase by compensating amounts that sum to the decrease.