PaperlayerView exact revision on arXiv

Source-linked AI summary

Predicting with Confidence on Unseen Distributions

Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, Ludwig Schmidt

arXiv:2107.03315v2cs.LGcs.CVstat.ML

TL;DR

The paper addresses how to estimate model accuracy on unlabeled, unseen distributions when distribution shift can substantially alter performance. It evaluates distributional distances and confidence-based signals, finding that DoC reliably predicts performance changes across natural and synthetic shifts, reducing predictive error by nearly half in challenging natural settings.

  • Problem

    Models can vary substantially in accuracy on data from distributions different from training, while acquiring labels for each new shift may be prohibitively costly.

  • Method

    The paper compares distributional distances and confidence-based signals, including DoC, as features or predictors of accuracy changes between base and unseen target distributions.

  • Results

    DoC accurately predicts performance change across natural and synthetic shifts and reduces predictive error by 46% across challenging natural distribution shifts.

  • Takeaways & Limitations

    Simple confidence differences can outperform more elaborate distributional-distance measures for estimating model performance under distribution shift.

  • Takeaways & Limitations

    The problem remains far from solved, and reducing the performance drop itself remains open.

Abstract

from arXiv · show

Recent work has shown that the performance of machine learning models can vary substantially when models are evaluated on data drawn from a distribution that is close to but different from the training distribution. As a result, predicting model performance on unseen distributions is an important challenge. Our work connects techniques from domain adaptation and predictive uncertainty literature, and allows us to predict model accuracy on challenging unseen distributions without access to labeled data. In the context of distribution shift, distributional distances are often used to adapt models and improve their performance on new domains, however accuracy estimation, or other forms of predictive uncertainty, are often neglected in these investigations. Through investigating a wide range of established distributional distances, such as Frechet distance or Maximum Mean Discrepancy, we determine that they fail to induce reliable estimates of performance under distribution shift. On the other hand, we find that the difference of confidences (DoC) of a classifier's predictions successfully estimates the classifier's performance change over a variety of shifts. We specifically investigate the distinction between synthetic and natural distribution shifts and observe that despite its simplicity DoC consistently outperforms other quantifications of distributional difference. $DoC$ reduces predictive error by almost half ($46\%$) on several realistic and challenging distribution shifts, e.g., on the ImageNet-Vid-Robust and ImageNet-Rendition datasets.

1. Introduction

Models can lose substantial accuracy under distribution shift, making unlabeled performance estimation important. The paper evaluates distributional distances and confidence-based signals, finding that DoC estimates accuracy changes more reliably across challenging shifts.

  • 1. Introduction: Distribution shift can substantially change model accuracy even when new test data is designed to resemble the training distribution.Models deployed in real-world environments encounter different data distributions, motivating reliable estimates on unseen test sets.
  • 1. Introduction: The study compares domain-adaptation distances and predictive-confidence signals for automatic model evaluation across natural and synthetic shifts.The evaluated distances include Fréchet distance, MMD, and discriminative discrepancy.
  • 1. Introduction: Common distributional distances fail to reliably predict accuracy changes on natural shifts, while average confidence outperforms them without regression.These distances can encode held-out synthetic shifts but not natural distribution shifts reliably.
  • 1. Introduction: DoC reduces predictive error by 46% across challenging natural distribution shifts, including ImageNet-VidRobust and ImageNet-Rendition.The method uses confidence differences between base and target distributions to estimate the accuracy gap.
  • 1. Introduction: The proposed pipeline feeds base and target examples into a featurizer, learns a regressor from distributional differences to accuracy gaps, and adds the estimate to base accuracy.The target distribution is unlabeled when the estimate is produced.

3. Baseline Distance Measures

The baseline framework predicts performance on an unseen target distribution from differences between base and target data. It compares distance measures computed from model-derived features and trains a regression model on calibration shifts.

  • 3. Baseline Distance Measures: Fair comparison evaluates all approaches as target-accuracy predictors after applying the base-accuracy-plus-predicted-gap transformation.Some methods directly predict accuracy, whereas others predict the accuracy gap.
  • 3. Baseline Distance Measures: Distance measures are computed between base and target subsets using features extracted from the classifier, including penultimate activations, logits, probabilities, or convolutional activations.The default featurizer uses penultimate-layer activations.
  • 3. Baseline Distance Measures: The evaluated baselines include discriminative distances, Fréchet distance, and related domain-adaptation measures.Domain discriminators are assessed using final AUC and A-proxy distances.
  • 3. Baseline Distance Measures: The framework estimates target accuracy by predicting the accuracy gap between base and target distributions from a learned distance measure.Predicted gaps can be combined with known base-distribution accuracy to recover target accuracy.
  • 3. Baseline Distance Measures: The regression model is trained on calibration accuracy gaps and distance measures, minimizing mean squared error, then evaluated on validation shifts.Primary experiments use linear regression, while supplementary nonlinear models follow the same trends.

4. Predicting Performance with Difference of Confidences

The paper uses differences in model confidence between base and target distributions to quantify shifts and predict accuracy changes without labeled target data. DoC outperforms alternative approaches, while DoE also improves prediction on natural shifts but remains weaker than DoC.

  • A perfectly calibrated model has expected confidence equal to accuracy, but modern neural networks are often miscalibrated.This motivates using confidence changes rather than assuming average confidence directly equals accuracy.
  • Average confidence alone is an unreliable accuracy estimate, but its difference across distributions is informative under distribution shift.The baseline directly estimates unseen-distribution accuracy, whereas DoC instead captures the change between base and target confidence.
  • DoC quantifies distribution shift using the difference between classifier confidences on the base and target distributions.The paper proposes this confidence difference as a simple summary of model-output probabilities.
  • 46%: DoC reduces predictive error across challenging natural and synthetic distribution shifts, outperforming other evaluated methods.The method treats DoC as a feature for regression models that estimate the accuracy gap between base and target distributions.
  • DoE also outperforms other baselines on natural shifts, although it reduces prediction error less effectively than DoC.DoE is defined as a difference of average entropy between distributions and is evaluated alongside DoC as a shift feature.

5. Experiments

Experiments evaluate accuracy prediction across ImageNet models and synthetic, natural, and adversarial distribution shifts. DoC generally provides the strongest natural-shift predictions, while performance varies by shift type and model training scheme.

  • Synthetic to Natural: DoC reduces natural-shift prediction error by more than 45%, achieving an average accuracy-prediction error of 5.0 ± 0.010 across the evaluated models and shifts.DoC, DoC-Feat, and DoE are the only approaches reported to outperform the AC baseline overall.
  • Model Specific Performance: DoC improves natural-shift prediction for every evaluated model variant except AugMix-DeepAugment, where synthetic-corruption training may reduce the benefit.The reported decline is associated with models incorporating robustness interventions for synthetic corruptions.
  • Synthetic to Synthetic: On held-out synthetic shifts, Frechet distance and DoC tie for best performance at MAE 0.039 ± 0.007, reducing relative error 49% against AC.Most evaluated approaches improve over the AC baseline in this setting, except Rotation Prediction.
  • Adversarial Distribution Shift: ImageNet-A is the most difficult setting: DoC alone improves over AC, but its MAE remains high at 0.389 ± 0.027.The adversarially designed dataset yields the highest prediction error across the studied shifts.
  • Natural Calibration: Calibrating and evaluating on natural shifts improves DoC and DoE, but the reported errors remain substantially worse than the AC baseline.DoC and DoE are the only encodings reported to outperform AC in that natural-calibration setting, with errors of 0.293 ± 0.024 and 0.371 ± 0.033.

6. Conclusion

The paper finds that difference of classifier confidences (DoC) predicts performance changes across natural and synthetic distribution shifts. It presents DoC as a promising step for detecting performance drops, while emphasizing that reducing those drops remains unresolved.

  • DoC accurately predicts performance change across a wide range of natural and synthetic distribution shifts.
  • The problem of reducing performance drops under distribution shift remains completely open.
  • DoC could help focus expensive data collection on difficult sub-distributions as a potential filtering mechanism.

7. Non-Linear Regression

The non-linear regression experiments use a three-layer neural network to predict model accuracy from distributional-difference features. Under synthetic-to-natural evaluation, DoC is the best-performing approach and the only approach described as outperforming AC in the supplied figure.

  • The non-linear regressor predicts model accuracy from distributional-difference features using a three-layer fully connected neural network.The network has layers of 512, 256, and 128 units.
  • DoC is the best-performing approach when calibrated on synthetic distributions and evaluated on natural shifts.
  • DoC, DoC-Feat, and DoE are the only approaches reported to outperform the AC baseline in the supplied synthetic-to-natural comparison.

8. Distribution Shifts and Grouping

The study separates natural and synthetic distribution shifts and evaluates calibration across their combinations. DoC and DoE outperform the other approaches across the reported synthetic-to-natural and natural-to-adversarial settings, while synthetic-shift prediction is less discriminative among methods.

  • Natural shifts include ImageNet-V2, ImageNet-Vid-Robust, ImageNet-Sketch, and ImageNet-Rendition, while synthetic shifts are generated programmatically from existing data.
  • When predicting accuracy under synthetic shifts with non-linear regression, all approaches outperform AC, while DoC, DoE, Frechet, and MMD perform comparably.
  • For ImageNet-A under non-linear regression, DoC is the only approach reported to outperform the AC baseline.
  • ImageNet-A is evaluated separately because it is adversarially collected and may differ from model-agnostic shifts.
  • DoC and DoE outperform all other approaches across the reported synthetic-to-natural and natural-to-adversarial calibration settings.The reported settings are Syn1 → Natural, Syn1 → Adv., Syn2 → Natural, Syn2 → Adv., and Natural → Adv.

9. Visualizing Features and Predictions

Feature visualizations show that confidence-based measures correlate most strongly with accuracy gaps on natural shifts, whereas Frechet distance and MMD appear non-informative for global prediction. Across held-out shifts, confidence-based approaches provide the best estimates, although synthetic and natural shifts produce markedly different patterns.

  • Confidence-based measures show the highest correlations with accuracy gaps on natural distribution shifts.
  • Frechet distance and MMD appear non-informative for predicting accuracy under distribution shifts in global feature visualizations.
  • Confidence-based approaches have the lowest MAE and highest R2 when predicting accuracy gaps over natural shifts.
  • Predictions differ sharply between natural and held-out synthetic shifts, highlighting the importance of evaluating both types of shift.
  • Confidence-based approaches provide the best estimates across held-out synthetic, natural, and adversarial shifts.
  • DoC aligns predicted and actual accuracy gaps across architectures even without exposure to natural shifts during calibration.

10. Baseline Algorithms

The paper evaluates distributional distances and predictive signals as features for estimating accuracy under unseen shifts. Baselines include MMD, rotation prediction, Frechet distance, discriminative distances, and temperature-scaled confidence.

  • Baseline Algorithms: MMD, rotation prediction, Frechet distance, and discriminative distances are evaluated as features for predicting accuracy on unseen distributions.The approaches draw on domain adaptation and self-supervised generalization methods.
  • Baseline Algorithms: Rotation-prediction models are trained on base data and evaluated on target datasets using accuracy and AUC.The task predicts which of four rotations—0, 90, 180, or 270 degrees—was applied to an image from the model featurization.
  • Baseline Algorithms: Frechet distance is computed from feature summary statistics across the complete base and target datasets, ignoring train, validation, and test splits.The computation uses the average featurization and covariance matrix for each dataset.
  • Baseline Algorithms: Discriminative distances estimate how well linear classifiers distinguish base from target data using train, validation, and test splits.The classifiers operate on featurizations, with multilayer perceptrons explored separately in the supplement.
  • Baseline Algorithms: Temperature scaling calibrates the average-confidence baseline on the base dataset using one temperature optimized by negative log likelihood.Average confidence is used as the uncalibrated baseline.

11. Experimental Details

The experiments account for label-subset differences when predicting accuracy changes and compare methods across synthetic and natural shifts. DoC performs best across the reported synthetic and natural evaluations, while its performance varies across individual corruptions.

  • Experimental Details: Accuracy changes are predicted relative to ImageNet-Val accuracy restricted to the classes present in each shifted distribution.Distributional differences are computed only over instances containing the target distribution’s classes.
  • Experimental Details: DoC improves performance on 6 of 8 corruptions but degrades performance on Defocus Blur and Gaussian Blur.The result shows that synthetic shifts differ in difficulty for the method.
  • Experimental Details: All approaches outperform the AC baseline on synthetic shifts calibrated with Syn2 grouping, while DoC produces the best estimates.DoE is the second highlighted approach among the tested encodings of distributional difference.

12. Res-Ensemble

The study evaluates a ResNet ensemble as a calibration baseline and summarizes accuracy-prediction methods across calibration settings. DoC remains strongest on natural and adversarial evaluations in the supplied results.

  • Res-Ensemble: The Res-Ensemble calibration baseline combines pretrained ResNet-18, ResNet-34, ResNet-50, ResNet-101, and ResNet-152 architectures.Deep ensembles were selected based on prior evidence of strong calibration under distribution shift.
  • Res-Ensemble: DoC is the best-performing approach when calibrated on Syn2 and evaluated on natural distributions.DoE performs near parity with AC, while prior approaches perform worse than the baseline.
  • Res-Ensemble: DoC is the best-performing approach on ImageNet-A after Syn2 calibration, whereas DoE performs worse than the baseline.ImageNet-A remains a distinct challenge among the evaluated shifts.
  • Res-Ensemble: DoC produces the best overall prediction for each target distribution and remains best for natural shifts across calibration settings.Table 2 reports mean absolute error and standard deviation for the compared approaches.
  • Res-Ensemble: The method summary specifies the data used to compute distributional differences and the accuracy value predicted by each approach.The comparison covers all accuracy-prediction methods used in the work.
Loading 2107.03315v2…