Source-linked AI summary
Confidence estimation in Deep Neural networks via density modelling
Akshayvarun Subramanya, Suraj Srinivas, R. Venkatesh Babu
TL;DR
Adversarial changes can produce high-confidence false predictions, raising questions about confidence estimates. The paper proposes density modelling and reports that it mostly outperforms softmax, with performance limits in some distortion cases.
Problem
Adversarial perturbations can produce high-confidence false predictions, while softmax-based confidence can rise with input magnitude and ignore dataset structure.
Source: §1 · p. 1, §3.1 · p. 2, §3.2 · p. 2Method
The paper estimates class-conditional densities of final-layer activations and uses Bayes’ rule to obtain predictive confidence.
Source: §1 · p. 1, §3.3 · p. 3Results
In most cases, the diagonal Gaussian works well and mostly outperforms a softmax-based approach, but performance suffers with Gaussian blurring and partly with adversarial examples.
Source: §5 · p. 5Takeaways & Limitations
Density modelling is presented as a general confidence-estimation approach, with more sophisticated models proposed as a direction for high-dimensional inference.
Source: §5 · pp. 4–5The approach assumes P(z) is approximately P(X), an assumption the authors say may be debatable but empirically appears to hold for many natural images.
Source: §3.3 · p. 3
Abstract
from arXiv · showhide
State-of-the-art Deep Neural Networks can be easily fooled into providing incorrect high-confidence predictions for images with small amounts of adversarial noise. Does this expose a flaw with deep neural networks, or do we simply need a better way to estimate confidence? In this paper we consider the problem of accurately estimating predictive confidence. We formulate this problem as that of density modelling, and show how traditional methods such as softmax produce poor estimates. To address this issue, we propose a novel confidence measure based on density modelling approaches. We test these measures on images distorted by blur, JPEG compression, random noise and adversarial noise. Experiments show that our confidence measure consistently shows reduced confidence scores in the presence of such distortions - a property which softmax often lacks.
1. INTRODUCTION
Because adversarial changes can yield high-confidence false predictions [4], the paper asks whether confidence estimation needs a better approach. It proposes density modelling and reports better confidence estimates than softmax.
- Softmax is often retained at test time as a proxy for label probabilities, although prior work finds it ineffective for uncertainty estimates [5].Source: §1 · p. 1
- Adversarial perturbations can induce imperceptible, high-confidence false predictions, threatening forensic and biometric uses [4].Source: §1 · p. 1
- The paper discusses pathological softmax behavior and proposes density modelling as a way to estimate predictive confidence.Source: §1 · p. 1
- The proposed density-modelling measure is reported superior to softmax at producing confidence estimates.Source: §1 · p. 1
2. RELATED WORKS
Prior uncertainty methods and adversarial-example studies frame two related concerns: softmax-based confidence estimates may be unreliable, and neural networks can be fooled by small perturbations.
- Gal et al. [5] use test-time dropout averaging and Bendale et al. address open-set recognition, but both use softmax to compute uncertainty.Source: §2 · p. 1
- Studies describe adversarial perturbations and fooling images, including universal perturbations [10] and eyeglass frames that tamper with face recognition.Source: §2 · pp. 1–2
- These examples raise security concerns for commercial neural-network applications, while good uncertainty estimates are presented as a possible remedy.Source: §2 · p. 2
3. CONFIDENCE ESTIMATION
The section defines predictive confidence as estimating label probabilities from network activations, then shows why softmax can be misleading. It proposes class-conditional density modelling, addresses high-dimensional Gaussian behavior, and describes the resulting confidence pipeline.
3. CONFIDENCE ESTIMATION
- Confidence estimation seeks P(y|X) from pre-softmax activations and should ideally track classifier accuracy across samples.Low accuracy paired with high confidence indicates confident incorrect predictions.Source: §3 · p. 2
3.1. Pathologies with Softmax and Neural Networks
- Softmax confidence rises when input magnitude increases, so large-norm inputs can receive higher confidence regardless of data structure.The section identifies this magnitude dependence as a pathology that should be avoided.Source: §3.1 · p. 2
3.2. How to estimate better confidence?
- For better confidence, compare an input with points from each class: large-norm points can lie far from a separating hyperplane while ignoring dataset structure.The section proposes distances to class points as a more suitable intuition for confidence.Source: §3.2 · p. 2
3.3. Proposed method: Density Modelling
- The method models P(z|y_i) for each class using final-layer activations, then applies Bayes’ rule with class priors to estimate P(y_i|z).It assumes P(z) ∼ P(X), an empirically supported but potentially debatable approximation for many natural images; Gaussian density choices simplify computation.Source: §3.3 · p. 3
3.4. Gaussian in High Dimensions
- High-dimensional Gaussian probability concentrates in a thin annulus, giving points similarly small densities; scaling covariance by d produces varied, non-vanishing density values.The section presents this covariance adjustment as a way to support Bayes-rule computations.Source: §3.4 · p. 3
3.5. Overall process
- Training remains softmax-based; afterward, training data estimates density parameters, and test-time confidence comes from all class densities followed by Bayes’ rule.The predicted label is still the maximum entry of the pre-softmax vector, equivalent to the maximum softmax output.Source: §3.5 · pp. 3–4
4. EXPERIMENTS
The experiments test confidence measures on MNIST and ImageNet under image distortions, including noise, blur, JPEG compression, and adversarial examples. The reported results vary by distortion and dataset.
- The experiments use LeNet-5 on MNIST and VGG-16 on the ILSVRC12 validation set, testing whether confidence falls on distorted, unfamiliar images.Source: §4 · p. 4
- For Gaussian noise, the proposed method has a monotonically decreasing confidence profile on both MNIST and ImageNet, unlike softmax.Source: §4 · p. 4
- For Gaussian blur, the proposed confidence behavior appears on MNIST but not ImageNet.Source: §4 · p. 4
- Both softmax and the proposed method show the expected confidence profile as JPEG quality increases.Source: §4 · p. 4
- Adversarial evaluation counts confidence increases on ImageNet validation examples as failures for each method, with results assigned to Table 1.Adversarial examples were generated using.Source: §4 · p. 4, Table 1 · p. 4
5. DISCUSSION AND CONCLUSION
The paper presents density modelling as a confidence measure and reports that a diagonal Gaussian works well in most cases, mostly outperforming softmax. Results vary across distortions, with high-dimensional density estimation proposed as a possible difficulty for blur and some adversarial examples.
- The proposed confidence measure uses density modelling, and the authors note that the approach can use models more sophisticated than a diagonal-covariance Gaussian.Source: §5 · pp. 4–5
- Softmax confidence is sensitive to input scale, which the authors describe as undesirable.Source: §5 · p. 4
- On ImageNet, the proposed approach is qualitatively better than softmax for distortion in Figure 4(a), while both behave similarly in Figures 4(b–c).Both measures are scaled so clean-image confidence equals 1.Source: Figure 4 · p. 5
- As distortions increase, the proposed approach's unnormalized confidence drops, whereas softmax confidence does not show this property.Source: Figure 5 · p. 5
- The diagonal Gaussian works well in most cases and mostly outperforms softmax, while blur and partly adversarial performance may suffer from high-dimensional density estimation.The authors suggest more sophisticated models may work better for high-dimensional inference.Source: §5 · p. 5
Appendix
The appendix proves Lemma 3.1 by comparing s_i(kz) with s_i(z) and ruling out the assumption that scaling by k makes s_i smaller.
Appendix
- Assuming s_i(kz) < s_i(z) leads to a contradiction, so the premise is false.The proof concludes that s_i(kz) is not less than s_i(z).Source: Appendix · p. 6
- The comparison relies on each element z_j being independent of the others.This assumption is stated when deriving the condition for s_i(kz) > s_i(z).Source: Appendix · p. 6
- The contradiction uses k > 1 and z_i being the maximum element of z to reject the two alternative conditions.The proof rejects k < 1 and z_j > z_i.Source: Appendix · p. 6