PaperlayerView exact revision on arXiv

Source-linked AI summary

Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, Jasper Snoek

arXiv:1906.02530v2stat.MLcs.LG

TL;DR

Predictive uncertainty is difficult to assess when deployment data shift from training data, despite its importance for deciding when model outputs should be trusted. The paper benchmarks uncertainty methods across classification tasks and modalities, finding that post-hoc calibration often fails under shift while some model-marginalizing methods perform strongly. The study also identifies scalability and computational cost as practical boundaries.

  • Problem

    Existing uncertainty methods lacked a rigorous large-scale empirical comparison under dataset shift, where calibrated confidence is important for assessing risk and model trustworthiness.

  • Method

    The paper evaluates popular probabilistic deep-learning approaches on classification benchmarks spanning image, text, and categorical data under i.i.d., shifted, and OOD conditions.

  • Results

    Post-hoc calibration can harm calibration under dataset shift, while ensembles and some other model-marginalizing methods perform strongly across multiple tasks.

  • Takeaways & Limitations

    Uncertainty methods should be evaluated under dataset shift rather than relying only on i.i.d. calibration, with model-marginalizing approaches showing promising results.

  • Takeaways & Limitations

    The benchmark focuses on uncertainty quality, while computational and memory costs remain important because better-performing methods tend to be more expensive.

Abstract

from arXiv · show

Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}. Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In such settings, well calibrated uncertainty estimates convey information about when a model's output should (or should not) be trusted. Many probabilistic deep learning methods, including Bayesian-and non-Bayesian methods, have been proposed in the literature for quantifying predictive uncertainty, but to our knowledge there has not previously been a rigorous large-scale empirical comparison of these methods under dataset shift. We present a large-scale benchmark of existing state-of-the-art methods on classification problems and investigate the effect of dataset shift on accuracy and calibration. We find that traditional post-hoc calibration does indeed fall short, as do several other previous methods. However, some methods that marginalize over models give surprisingly strong results across a broad spectrum of tasks.

1 Introduction

Deep neural networks are increasingly used in high-stakes applications where accurate predictive uncertainty matters, especially when deployment data shift from training data. The paper benchmarks uncertainty methods under dataset shift because prior comprehensive comparisons were lacking.

  • Motivation: High-stakes applications require confidence estimates alongside class predictions to quantify predictive uncertainty.Such applications include medical imaging diagnosis and self-driving cars.
  • Motivation: Dataset shift can make deployed data differ substantially from training data, threatening reliable confidence assessment.Shift may arise from time of day, seasonality, or popular trends.
  • Contribution: The paper presents a large-scale benchmark of probabilistic deep-learning approaches across diverse classification benchmarks and distributional settings.The evaluation focuses on both i.i.d. data and distributional shift across image, text, and categorical modalities.
  • Research questions: The study asks whether uncertainty and accuracy co-vary under dataset shift and whether i.i.d. calibration transfers to shifted data.It also asks whether any methods perform consistently well in this regime.

2 Background

The paper formulates classification uncertainty evaluation by comparing predictions on in-distribution data with corrupted, perturbed, and fully OOD inputs. It focuses on predictive-distribution methods to enable fairer comparisons across modeling assumptions.

  • Problem setup: The setup models k-class labels with a neural-network conditional distribution pθ(y|x), evaluated on in-distribution and shifted test data.The true distribution p*(x,y) is unknown and observed through training samples.
  • Shift types: Covariate shifts use corrupted or perturbed inputs whose labels remain among the k classes, with uncertainty expected to increase as accuracy degrades.These shifts are intended to test behavior under progressively altered inputs.
  • Shift types: Fully OOD datasets contain instances whose labels are outside the k modeled classes, so evaluation relies on prediction-only uncertainty diagnostics.The paper examines whether predictive uncertainty increases on these new instances.
  • Method taxonomy: The authors distinguish conditional-distribution methods, joint-distribution models, and methods adding an OOD-detection component to p(y|x).They focus on methods described by the first category to support more meaningful comparisons across differing assumptions.

3 Methods and Metrics

The benchmark compares scalable uncertainty methods using proper scoring rules, calibration, entropy, and prediction-confidence diagnostics. It also documents important metric limitations and practical constraints on method selection.

  • Methods: The evaluated methods include vanilla softmax, temperature scaling, Monte-Carlo dropout, ensembles, SVI, and Bayesian or dropout variants for the last layer.The selected approaches were chosen for prevalence, scalability, and practical applicability.
  • Practical scope: The methods were selected partly because they scale to training and prediction, although some alternatives were difficult to train on very large or high-class-count datasets.The appendix notes challenges on the 37M-example Criteo dataset and 1000-class ImageNet.
  • Metrics: Brier score and negative log-likelihood are proper scoring rules whose optimum corresponds to a perfect prediction.Brier score decomposes into uncertainty − resolution + reliability, with reliability measuring calibration.
  • Metrics: Expected calibration error measures the average gap between bucketed accuracy and predicted probability, while entropy summarizes predictive uncertainty.ECE can have varying estimation error across bins and need not increase monotonically as predictions approach ground truth.
  • OOD diagnostics: For fully OOD inputs, the evaluation reports confidence and predictive-entropy histograms plus accuracy-versus-confidence plots.Confidence is defined as the maximum predicted class probability.

4 Experiments and Results

The benchmark evaluates predictive uncertainty across increasingly shifted and OOD data in image, text, and categorical tasks. Calibration and uncertainty behavior degrade under shift, while model-marginalizing methods—especially ensembles—often remain more reliable than post-hoc calibration.

  • Experimental setup: Across image, text, and categorical datasets, the experiments evaluate accuracy, calibration, confidence, entropy, and proper scoring rules under increasing shift and OOD inputs.The benchmark follows standard training, validation, and testing protocols while additionally evaluating increasingly shifted data and an OOD dataset.
  • MNIST: On MNIST, accuracy degrades for all methods under shift, but SVI achieves lower Brier scores and more cautious predictive distributions despite lower validation and test accuracy.SVI has lower overall confidence and higher predictive entropy under shift and OOD data.
  • Calibration under shift: Calibration on an i.i.d. validation set does not guarantee calibration under distributional shift, and post-hoc temperature scaling can degrade shifted calibration.This pattern appears in both the MNIST and image-model experiments and is also reflected in the Criteo Brier-score results.
  • Image models: On CIFAR-10 and ImageNet, accuracy and ECE worsen with increasing corruption, while ensembles achieve the highest shifted accuracy and outperform other methods at larger shifts.Temperature scaling has low ECE at small shifts but its ECE increases significantly as shift increases; increasing model capacity alone does not reproduce ensemble gains.
  • Image and text models: Across CIFAR-10, ImageNet, and text experiments, ensembles provide the strongest overall confidence-versus-accuracy and entropy behavior under shifted or OOD inputs.Only ensembles consistently show higher entropy on shifted text data, while LL-Dropout and LL-SVI can produce higher-confidence incorrect predictions.
  • Categorical data: On Criteo, ensembles are superior in AUC and Brier score for most shift levels, while dropout improves shifted AUC and surpasses ensembles in Brier score above 60% randomization.SVI performed poorly, and temperature scaling had a worse Brier score than Vanilla under dataset shift.

5 Takeaways and Recommendations

Across datasets and modalities, uncertainty quality generally worsens under shift, while deep ensembles are the most consistently robust method; computational cost remains an important trade-off.

  • Uncertainty quality consistently degrades as dataset shift increases, regardless of method.
  • Better i.i.d. calibration and accuracy usually do not translate to better calibration on shifted or completely OOD data.
  • Temperature scaling performs well on i.i.d. data and small shifts but is outperformed by methods modeling epistemic uncertainty as shift increases.
  • Last layer Dropout produces less uncertainty on shifted and OOD datasets than standard Dropout.
  • SVI is promising on MNIST and CIFAR but is difficult to deploy on larger datasets such as ImageNet and on LSTMs.
  • Deep ensembles perform best across most metrics and appear more robust to dataset shift, with ensemble size M = 5 potentially sufficient.
  • On a challenging genomics problem, observations matched the broader experiments: ensembles performed best, but substantial room for improvement remained.
  • The strongest-performing methods tend to be more expensive in computational and memory terms.

Uncertainty Under Dataset Shift: Appendix

The appendix describes dataset-specific architectures, training procedures, uncertainty-method implementations, and validation-based hyperparameter tuning used in the benchmark.

  • MNIST experiments used LeNet and MLP models, reporting LeNet because both showed similar metric trends.Stochastic methods averaged 300 predictions, while ensembles used 10 independently initialized instances.
  • CIFAR experiments used ResNet-20 V1 with data augmentation, scheduled Adam training, and method-specific dropout where applicable.
  • ImageNet experiments used ResNet-50 V1 trained with scheduled SGD and stochastic prediction samples.
  • The 20 Newsgroups pipeline used a 30,000-word vocabulary and fixed 250-word padded or truncated inputs.
  • For 20 Newsgroups, vanilla, Dropout, LL-SVI, ensembles, and stochastic methods differed in their final layers, replication, and sampling schemes.The vanilla model achieved in-distribution accuracy 0.955.
  • Criteo models hashed categorical features into one-hot or embedded representations before combining them with numerical features in a three-hidden-layer MLP.
  • Criteo architectural parameters were tuned for a vanilla model and then applied across methods.
  • Stochastic methods used 128 prediction samples per example, with learning rate, batch size, and dropout tuned by method.

A.9 Computational and Memory Complexity of Different methods

The appendix frames method selection as a computational and memory trade-off while documenting the image shifts used in the experiments.

  • Applications must consider computational and memory costs in addition to predictive performance.Table S1 discusses these costs for each evaluated method.
  • Table S1 compares computational and memory costs using full-model, last-layer, replication, inducing-point, evaluation-point, and validation-set quantities.Sampling introduces an implicit memory/compute trade-off.
  • B Shifted Images: MNIST shifts were generated with rotations and cyclic translations.
  • B Shifted Images: ImageNet-C supplied ImageNet corruptions across 16 types and five intensity levels.
  • B Shifted Images: Figure S1 illustrates rotated and cyclically translated MNIST digits, with corresponding accuracy and calibration results reported in Figure 1.
  • B Shifted Images: Figure S2 illustrates Gaussian blur on ImageNet-C from intensity 0, uncorrupted, through intensity 5, the maximum included.
  • B Shifted Images: Figure S3 illustrates all 16 ImageNet-C corruption types at intensity 3, which were also applied to CIFAR-10.

C Evaluating uncertainty under distributional shift: Additional Results

Additional results compare uncertainty methods across CIFAR-10, ImageNet, and Criteo, showing that ensembles and Dropout remain strongest under corruption while capacity alone does not improve calibration.

  • Ensembles and Dropout outperform other methods across corruptions, whereas LL-SVI shows no improvement over the baseline.
  • The comparisons use Brier score, reliability, resolution, and other metrics across CIFAR-10, ImageNet, and Criteo.Lower reliability indicates better calibration, while higher resolution indicates better predictive quality.
  • Higher-capacity Vanilla and Dropout models are not better calibrated than lower-capacity counterparts, suggesting ensemble calibration is not explained simply by capacity.
  • CIFAR-10 boxplots summarize metric distributions across corruption types and shift levels using quartiles and minimum-to-maximum error bars.
  • ImageNet boxplots compare methods across corruption types and shift levels using quartiles and minimum-to-maximum error bars.
  • Criteo results show that worsening reliability accounts for most Brier degradation and largely determines method rankings.

D Effect of the number of samples on the quality of uncertainty

Increasing sample sizes improves predictive uncertainty for SVI and Dropout, with diminishing returns beyond five samples, while last-layer variants change little. Ensemble performance similarly improves with diminishing returns beyond size five, evaluated using Brier score and its calibration components.

  • Increasing sample sizes improves SVI and Dropout performance, with diminishing returns beyond size 5.Last-layer variants show little effect from increasing sample sizes.
  • Increasing ensemble size improves Brier scores on CIFAR-10 and ImageNet, with diminishing returns beyond size 5.
  • Brier score is decomposed into reliability, uncertainty, and resolution components.Reliability measures calibration violations, while higher resolution indicates greater deviation from marginal uncertainty.

E Variational Gaussian Process Results

The section reports uncertainty metrics across increasing shift levels on CIFAR-10 using a last-layer Variational Gaussian Process, with level 0 denoting the test set.

  • Uncertainty metrics are evaluated across shift levels on CIFAR-10.
  • Shift level 0 corresponds to the CIFAR-10 test set.
  • The model uses a last-layer Variational Gaussian Process.Experiment details are provided in Appendix A.7.

F OOD detection for genomic sequences

The genomic-sequence experiment trains classifiers on 10 in-distribution bacteria classes and tests OOD detection on 60 OOD classes. Ensembles provide the best confidence–accuracy trade-off, although even high-confidence predictions remain challenging.

  • Classifiers train on 10 in-distribution bacteria classes and test OOD detection on 60 OOD bacteria classes.The architecture uses convolutional layers, global max pooling, dense layers, and classifier-specific uncertainty mechanisms.
  • In-distribution classification accuracy is around 0.8 across the classifier variants.
  • Ensembles significantly outperform other methods and achieve the best accuracy-versus-confidence trade-off.Dropout outperforms temperature scaling, and both outperform LL-Dropout, LL-SVI, and Vanilla.
  • Accuracy remains below 65% even for the best method on examples with p(y|x) ≥0.9.The dataset is described as challenging for benchmarking future methods.

G Tables of Metrics

The tables report quartiles of Brier score, negative log-likelihood, and ECE for each model and dataset, computed across corrupted dataset variants.

  • Quartiles are reported for Brier score, negative log-likelihood, and ECE.
  • Quartiles are computed over all corrupted variants of each dataset.
  • The tables compare Vanilla, Temperature Scaling, Ensembles, Dropout, LL-Dropout, SVI, and LL-SVI.Some table layouts omit SVI or LL-SVI from the listed methods.
Loading 1906.02530v2…