Source-linked AI summary
Learning Confidence for Out-of-Distribution Detection in Neural Networks
Terrance DeVries, Graham W. Taylor
TL;DR
The paper addresses neural networks’ poor ability to recognize incorrect predictions and unfamiliar inputs. It trains interpretable confidence estimates, uses them for out-of-distribution detection, and finds improvements over output thresholding and ODIN while using misclassified in-distribution examples for calibration.
Problem
Neural networks often produce highly confident incorrect predictions and cannot reliably identify out-of-distribution inputs.
Method
The method trains a confidence estimate jointly with the classifier by adjusting predictions toward targets according to confidence and combining task and confidence losses.
Results
The technique improves upon output thresholding and ODIN for out-of-distribution detection across the reported tests.
Takeaways & Limitations
Misclassified in-distribution examples can calibrate the detector as a proxy for out-of-distribution examples, avoiding separate OOD examples for calibration.
Takeaways & Limitations
Related uncertainty formulations do not transfer well to classification tasks.
Abstract
from arXiv · showhide
Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safely perform. To jointly address these issues, we propose a method of learning confidence estimates for neural networks that is simple to implement and produces intuitively interpretable outputs. We demonstrate that on the task of out-of-distribution detection, our technique surpasses recently proposed techniques which construct confidence based on the network's output distribution, without requiring any additional labels or access to out-of-distribution examples. Additionally, we address the problem of calibrating out-of-distribution detectors, where we demonstrate that misclassified in-distribution examples can be used as a proxy for out-of-distribution examples.
1. Introduction
Modern neural networks can be highly confident when wrong and generally cannot recognize unfamiliar inputs. The paper proposes learned confidence estimates for out-of-distribution detection, reporting broad improvements over prior methods and proxy-based calibration without requiring OOD examples.
- Motivation: Neural network classifiers often produce highly confident incorrect predictions and cannot identify inputs unlike those seen during training.This can cause silent failures on nonsensical or adversarial inputs.
- Approach: The paper trains classifiers to output confidence estimates for each input and uses them to distinguish in-distribution from out-of-distribution examples.The approach is designed to be simple, interpretable, and computationally lightweight.
- Calibration: Misclassified in-distribution examples can serve as a proxy for out-of-distribution examples when calibrating the detection threshold.This avoids collecting or generating separate out-of-distribution examples for calibration.
2. Confidence Estimation
The method adds a confidence branch that adjusts predictions toward targets during training, balancing task and confidence losses. Training refinements preserve meaningful confidence estimates, while data augmentation helps retain difficult and misclassified examples.
- Architecture: The model adds a confidence branch parallel to the class-prediction branch, producing a scalar confidence estimate between 0 and 1.Confidence near 1 indicates the network expects a correct prediction, while confidence near 0 indicates uncertainty.
- Training objective: During training, prediction probabilities are interpolated toward the target distribution according to the network’s confidence.The task loss is computed from these modified probabilities.
- Training objective: The confidence loss penalizes low confidence, and its weighted sum with the task loss prevents the model from always choosing c = 0.The weighting hyperparameter λ balances the task and confidence losses.
- Training dynamics: When c →1, predictions approach the original probabilities; when c →0, they approach the targets while the confidence loss becomes very large.Intermediate confidence trades reduced task loss against increased confidence loss.
- Training refinements: A budget parameter β and adaptive λ preserve confidence meaning by encouraging c →1 for correct samples and c →0 for incorrect samples.Reasonable β values between 0.1 and 1.0 do not significantly affect out-of-distribution detection performance.
- Training refinements: Aggressive data augmentation creates difficult examples that help prevent overfitting and retain misclassified examples needed to learn confidence.The experiments use random cropping, flipping, and Cutout.
3. Out-of-Distribution Detection
The method uses learned confidence estimates, optionally enhanced by input preprocessing, to separate in- and out-of-distribution examples. Inputs with confidence c less than or equal to threshold δ are marked out-of-distribution.
- The learned confidence estimates are used directly to perform out-of-distribution detection after model training.
- The detector marks an input as out-of-distribution when its confidence estimate c is less than or equal to threshold δ.Detector quality can vary widely with small changes in δ.
- 3.1. Input Preprocessing: Input preprocessing perturbs images toward greater confidence, with in-distribution examples increasing in confidence more than out-of-distribution examples.The perturbation magnitude is represented by ϵ and is computed from gradients of the confidence loss.
4. Experiments
The experiments first examine learned confidence on a toy 2D dataset, then evaluate its ability to separate in- and out-of-distribution examples using replicated prior experiments.
- The evaluation combines a toy 2D confidence visualization with replicated out-of-distribution detection experiments.The replications originate from experiments by Hendrycks & Gimpel (2017) and Liang et al. (2018).
4.1. Visualizing Learned Confidence Estimates
On a noisy 2D XOR dataset, the learned confidence branch assigns low confidence near mixed-class boundaries and high confidence in single-class regions. Increasing noise raises the confidence lower bound for a fixed budget.
- The XOR visualization uses 500 training samples per dataset across progressively increasing noise levels.The model is a three-layer MLP with parallel classification and confidence branches, trained for 30 epochs with budget β = 0.3.
- The confidence branch outputs low confidence in noisy regions containing both classes and high confidence in regions containing a single class.These regions correspond respectively to true class boundaries and areas dominated by one class.
- As dataset noise increases with a fixed budget, the lower bound on the confidence estimate increases.The authors indicate that noisier datasets may require a larger budget to keep confidence near 0 for out-of-distribution and misclassified examples.
4.2. Out-of-Distribution Detection
The evaluation tests learned confidence for out-of-distribution detection across image datasets, metrics, architectures, and training settings. Learned confidence generally improves separation over softmax thresholding and can outperform ODIN, while misclassified in-distribution examples provide a conservative calibration proxy.
- Evaluation metrics: Detection quality is measured using FPR at 95% TPR, detection error, AUROC, and AUPR.Detection error is minimized over thresholds, while AUROC summarizes ranking of in-distribution versus out-of-distribution examples.
- Model training: The study evaluates DenseNet, WideResNet, and VGGNet models, including architectures with and without skip connections.Models are trained with standard augmentation and Cutout, and experiments use five runs per model.
- Comparison with baseline: Thresholding learned confidence yields better in- versus out-of-distribution separation than softmax prediction probability for almost all tested architectures and datasets.The comparison uses the baseline method of Hendrycks and Gimpel; Table 1 reports results averaged over five runs.
- Comparison with ODIN: Confidence estimates with input preprocessing outperform ODIN in all tested settings except DenseNet on CIFAR-10.The ODIN comparison uses the All Images dataset as out-of-distribution and temperature scaling with T = 1000.
- Selecting a detection threshold: Misclassified in-distribution holdout examples can calibrate the detection threshold without access to out-of-distribution examples, producing a more conservative detector.Their detection-error curves resemble those of true OOD examples; performance is reduced in most cases but remains acceptable.
5. Related Work
The work relates learned confidence to uncertainty estimation, while distinguishing its normalized classification confidence from prior approaches. Regression-oriented formulations do not transfer well to classification, and a separate classification technique uses logit noise to estimate uncertainty.
- Kendall & Gal and Gurevich & Stuke train networks to output both a prediction and an uncertainty estimate.
- Their loss formulations, like this work’s, downweight the main prediction loss using the estimated uncertainty.
- Regression uncertainty formulations do not transfer well to classification, motivating a separate logit-noise approach for classification uncertainty.The noise magnitude is proportional to the estimated uncertainty.
- Unlike the comparison method’s unbounded uncertainty, this work produces normalized confidence estimates that may better suit human-in-the-loop applications.
6. Conclusion
The paper introduces an intuitive, label-free method for learning neural-network confidence estimates. It applies those estimates to out-of-distribution detection, reports improvements over established baselines, and uses misclassified examples to calibrate detectors.
- The method learns intuitive neural-network confidence estimates without labels for supervised confidence training.
- It evaluates predicted confidence for out-of-distribution detection and reports improvements over output thresholding and ODIN.
- Misclassified examples can be used to calibrate out-of-distribution detectors.
- Future work targets learned confidence beyond classification, including semantic segmentation and natural language understanding.
A. Supplementary Materials
The supplementary materials include XOR confidence visualizations and comparison tables for confidence-based thresholding. Figure 5 varies data noise and confidence budget, while Tables 4 and 5 compare methods on SVHN and CIFAR-10.
- Figure 5 varies XOR data noise from left to right and confidence budget β from top to bottom.The training set illustrates how the confidence decision boundary extends across noisy regions.
- The figure shows confidence predictions alongside the training set to illustrate the confidence decision boundary’s coverage of noisy regions.
- Tables 4 and 5 compare baseline and confidence-based thresholding on SVHN and CIFAR-10, respectively.