PaperlayerView exact revision on arXiv

Source-linked AI summary

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Saurabh Garg, Sivaraman Balakrishnan, Zachary C. Lipton, Behnam Neyshabur, Hanie Sedghi

arXiv:2201.04234v3cs.LGstat.ML

TL;DR

The paper studies how to predict target-domain accuracy when only labeled source data and unlabeled target data are available. It proposes Average Thresholded Confidence, which transfers a source-calibrated confidence threshold to the target, and reports 2–4× lower estimation error than prior methods across varied shifts and datasets. The theory also shows that reliable estimation requires assumptions about the shift, with large errors remaining for some novel-population datasets.

  • Problem

    Target accuracy is difficult to assess under distribution shift because deployment data are unlabeled and collecting labeled target examples is costly.

  • Method

    ATC learns a threshold on source validation confidence so the above-threshold fraction matches source accuracy, then uses the above-threshold fraction on unlabeled target data.

  • Results

    ATC outperforms prior methods by 2–4× in predicting target accuracy across model architectures, datasets, and diverse distribution shifts.

  • Takeaways & Limitations

    ATC provides a practical accuracy-estimation strategy across several synthetic and natural shift benchmarks using labeled source and unlabeled target data.

  • Takeaways & Limitations

    Accuracy is not generally identifiable from labeled source and unlabeled target data alone, and ATC still has large estimation error on novel populations such as BREEDS.

Abstract

from arXiv · show

Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold. ATC outperforms previous methods across several model architectures, types of distribution shifts (e.g., due to synthetic corruptions, dataset reproduction, or novel subpopulations), and datasets (Wilds, ImageNet, Breeds, CIFAR, and MNIST). In our experiments, ATC estimates target performance $2$-$4\times$ more accurately than prior methods. We also explore the theoretical foundations of the problem, proving that, in general, identifying the accuracy is just as hard as identifying the optimal predictor and thus, the efficacy of any method rests upon (perhaps unstated) assumptions on the nature of the shift. Finally, analyzing our method on some toy distributions, we provide insights concerning when it works. Code is available at https://github.com/saurabhgarg1996/ATC_code/.

1 INTRODUCTION

The paper asks whether target-domain accuracy can be predicted from labeled source data and unlabeled target data, avoiding costly target labels. It introduces ATC, while showing that accuracy estimation requires assumptions about the distribution shift and performs strongly across diverse shifts and datasets.

  • Motivation: Target accuracy estimates from held-out source data become unreliable under distribution shift, while deployment-time target data are typically unlabeled.Target accuracy can degrade when source and target distributions differ.
  • Problem: The paper targets unlabeled-target accuracy prediction because collecting fresh labeled target examples is expensive and time-consuming, especially for non-stationary targets.
  • Theory: Without assumptions on the source classifier or shift, no method can generally estimate target accuracy because the target conditional may not be uniquely identified.The paper characterizes accuracy identification as being as hard as identifying the optimal predictor.
  • Method: ATC learns a confidence-score threshold on source validation data so the fraction above it matches source accuracy, then applies that fraction to unlabeled target data.The score can be maximum confidence or negative entropy.
  • Results: 2–4×: ATC outperforms prior methods in target-accuracy prediction across architectures, datasets, and corruption, reproduction, subpopulation, and wild distribution shifts.The evaluation includes ImageNet-C, ImageNet-v2, ImageNet-R, BREEDS, and WILDS.
  • Limitations: ATC can still return inconsistent estimates on some shifts, despite its empirical superiority, as illustrated by a toy model involving varying proportions of spurious-feature populations.

2 PRIOR WORK

Prior work estimates OOD performance either pointwise or directly from unlabeled target data, but existing approaches often depend on target calibration or labeled target domains. These dependencies can weaken across different kinds of distribution shift.

  • Out-of-distribution detection: OOD detection methods use model confidence, uncertainty, distances, density models, or unreliability estimates to identify or characterize unseen examples.
  • Predicting model generalization: Methods for predicting OOD classifier error either predict correctness for individual unlabeled points or directly estimate aggregate error.
  • Existing limitations: Target-calibration methods seek consistent accuracy estimates, but source-trained and calibrated deep models are seldom calibrated on unseen domains.
  • Existing limitations: Regression approaches learn from labeled target domains and distribution statistics, but statistic–accuracy correlations can vary substantially across shift types.
  • Existing limitations: Importance-reweighting methods can require additional information about the axis along which the distribution shifts, represented as slicing functions.

3 PROBLEM SETUP

The paper studies predicting a fixed classifier’s target accuracy using labeled source data and unlabeled target data. Without assumptions on the distribution shift, target accuracy is unidentifiable; suitable shift assumptions can make it identifiable.

  • Problem setup: The goal is to predict a fixed classifier’s target accuracy without labeled target data, using a model, validation source data, and unlabeled target data.The setup explicitly distinguishes performance prediction from adapting the model to target data.
  • Possibility and impossibility results: Without assumptions on the shift, target accuracy cannot generally be estimated even when the full source distribution and target input marginal are known.The unknown target conditional p_T(y|x) can change arbitrarily.
  • Possibility and impossibility results: Target accuracy is identifiable if and only if the available distributions and shift assumptions uniquely determine the target conditional p_T(y|x).This is the paper’s central identifiability criterion for arbitrary classifiers.
  • Possibility and impossibility results: Covariate shift and label shift provide examples where assumptions can uniquely determine the target conditional and permit accuracy estimation.Under covariate shift, p_S(y|x)=p_T(y|x); under label shift, p_S(x|y)=p_T(x|y).
  • Possibility and impossibility results: No single accuracy-estimation method works across all distribution-shift families when no classifier assumptions are imposed.The same observed source joint distribution and target input marginal can correspond to different target errors under covariate and label shift.

4 PREDICTING ACCURACY WITH AVERAGE THRESHOLDED CONFIDENCE

ATC predicts target performance by calibrating a confidence threshold on labeled source data and applying it to unlabeled target examples. Its source-derived threshold performs effectively across the paper’s broad evaluation settings.

  • Method: ATC uses a black-box classifier and labeled validation source data to predict target accuracy from unlabeled target data.The classifier is treated as fixed throughout the method.
  • Method: ATC scores each softmax prediction using maximum confidence or negative entropy, with higher scores intended to indicate likely correctness.These scores reduce the model output to a scalar confidence measure.
  • Threshold selection: ATC learns a source threshold t so the fraction of source examples scoring below t matches the classifier’s source error.The threshold is obtained by matching the two quantities using finite samples.
  • Target estimation: ATC estimates target error as the fraction of unlabeled target examples whose score falls below the learned threshold.Target accuracy can be obtained by complementing this error estimate.
  • Empirical behavior: The same source-derived threshold performs remarkably well in experiments across a wide range of settings, despite a potentially different threshold existing on the target distribution.The paper identifies this empirical transfer of the threshold as its main contribution.

5 EXPERIMENTS

Experiments evaluate accuracy estimation across diverse datasets, architectures, and distribution shifts using unlabeled target data. ATC generally achieves the lowest estimation error, including under novel-subpopulation shifts where regression-based alternatives perform poorly.

  • Experimental setup: Methods estimate target accuracy from unlabeled target data, while labeled target examples are used only to obtain true error estimates.The main comparisons keep the underlying model fixed while varying target datasets.
  • Experimental setup: The evaluation spans synthetic corruptions, natural dataset-reproduction shifts, artistic renditions, BREEDS subpopulation shifts, CIFAR shifts, MNIST shifts, and language and vision datasets.ImageNet, BREEDS, CIFAR, MNIST, and WILDS settings are evaluated with DenseNet121, ResNet50, DistilBERT, or a multilayer perceptron.
  • Results: On BREEDS novel subpopulations, ATC without regression achieves MAE 10.22, outperforming DOC with a fitted regression model at MAE 13.26.DOC improves from MAE 24.41 without fitting to 13.26 with fitting, but remains worse than ATC.
  • Results: ATC-NE with temperature scaling typically achieves the smallest MAE, improving by more than 2× on CIFAR and 3–4× on ImageNet over GDE.Across CIFAR, ImageNet, MNIST, and WILDS, ATC MAE ranges from 0.4–5.8.
  • Results: ATC remains competitive on BREEDS novel-subpopulation shifts, where MAE is substantially higher for all methods than on same-subpopulation shifts.The broader evaluation reports superior ATC performance across datasets, with lower MAE indicating better estimation.

6 INVESTIGATING ATC ON TOY MODEL

The toy model studies ATC under shifts in spurious-feature correlation using a binary linear-classification setting. ATC can consistently estimate target accuracy when the classifier retains a predictive invariant-feature component, but its estimates become biased under certain support shifts.

  • Toy model: The toy task is binary classification with a predictive invariant feature and a correlated but non-predictive spurious feature.Target shifts vary the degree of spurious correlation without initially shifting the label distribution.
  • Toy model: Finite-sample max-margin classifiers can depend on the spurious feature despite the invariant feature being predictive.The analysis considers linear sigmoid classifiers with weights on both invariant and spurious features.
  • ATC behavior: For classifiers with winv > 0, ATC with maximum confidence consistently estimates target accuracy as spurious correlation changes.The ATC threshold remains invariant under this distribution shift, allowing the target accuracy to be recovered.
  • ATC behavior: In empirical validation, ATC accurately estimates target performance while AC, DOC, and GDE fail under varying spurious correlation; IM is sensitive to histogram binning choices.The setup tunes the threshold on held-out source examples before shifting the target distribution.
  • Limitations: Shrinking the target support of the invariant feature can make ATC overestimate target performance.The paper identifies this as a biased-estimation case outside the conditions where ATC is guaranteed to be consistent.

7 CONCLUSION AND FUTURE WORK

The paper proposes ATC for estimating target accuracy from labeled source and unlabeled target data, and reports low estimation error across synthetic and natural shifts. It also establishes fundamental identifiability limits, identifies remaining failure cases, and outlines extensions to broader settings.

  • ATC estimates target-domain accuracy using unlabeled target data and labeled source data, achieving low estimation error across synthetic and natural shift benchmarks.The method uses softmax confidence thresholds and is motivated by methods for OOD detection.
  • The authors plan to extend the theoretical model beyond binary classification and the simple toy setting to study calibration and broader consistency conditions.In binary classification, post hoc temperature scaling does not change the effective threshold.
  • The theory shows that target accuracy is generally not identifiable from labeled source and unlabeled target data without substantial constraints on the target conditional distribution.The paper plans to characterize broader conditions under which ATC can obtain consistent estimates.
  • ATC still has large estimation error on datasets containing novel populations, including BREEDS.This limitation motivates future work on accuracy estimation for such datasets.
  • The released code, stored models, hyperparameters, and seeds support reproduction of the reported results without hyperparameter tuning.The reproducibility statement describes the use of benchmarked hyperparameters and training procedures.

A PROOFS FROM SEC. 3 “

The proofs establish when target accuracy can be identified from labeled source and unlabeled target distributions. Without restrictions on the shift, different target conditionals can yield different errors; covariate-shift and label-shift assumptions provide identifiable special cases.

  • Target accuracy is identifiable when the possible target conditional distribution is uniquely determined by the labeled source and unlabeled target marginals.If multiple target conditionals remain possible, they can produce different errors for the same classifier.
  • Different target conditional distributions can produce different classifier errors despite identical labeled source and unlabeled target information.Therefore, no method can consistently estimate both quantities from that information alone.
  • Restrictions on the nature of the distribution shift are necessary to identify target accuracy.The proof explicitly concludes that the admissible target-conditionals set must be a singleton.
  • Covariate shift: Under covariate shift, target accuracy can be estimated when target labels conditionally follow the source labels given the covariates.The corresponding expression uses target covariate information and source conditional labels.
  • Label shift: Under label shift, target accuracy can be estimated when class-conditional covariate distributions remain unchanged between source and target.Importance ratios can be estimated with BBSE or MLLS; MLLS is consistent when target class-conditionals are strictly linearly independent.
  • Covariate shift versus label shift: The same observed source and target covariate information can correspond to different errors under covariate shift and label shift.Without further assumptions, neither error is discernible from the shared information.

D DETAILS ON THE TOY MODEL

The toy model studies ATC under shifts in spurious-feature prevalence and explains both a setting where ATC is consistent and settings where it becomes biased. Its analysis links finite-sample classifier behavior to geometric and statistical skew.

  • Finite-sample linear classifiers can depend on spurious features because minority-group margins are large under geometric skew.The dependence persists through statistical skew when training uses a finite number of epochs.
  • The toy model varies the fraction of examples with aligned or misaligned spurious features while preserving the within-set distributions.This isolates shifts in spurious-feature prevalence rather than changes within the two feature-defined groups.
  • Under this spurious-correlation shift, maximum-confidence ATC consistently estimates target accuracy for linear classifiers with a nonzero invariant-feature weight.The threshold matches the source error on the relevant error subset, and the unchanged within-group distributions preserve that match across target mixtures.
  • Shifting the target class-conditional support can bias ATC; shrinking the support while preserving uniformity causes overestimation of target performance.The same threshold then identifies more false correct than false incorrect examples.
  • The toy model assumes uniform class-conditional invariant-feature distributions on source and target, while varying spurious-feature correlation in the target.The target spurious-correlation parameter ranges across its allowed interval, and the setup does not initially vary label proportions.
  • ATC's threshold classifies low-score examples as predicted errors and estimates target error from their fraction among unlabeled target points.The method assumes the distributions of falsely identified correct and incorrect examples remain suitably aligned under the shift.

E BASLINE METHODS

The baseline methods estimate target accuracy through importance weighting, confidence calibration, confidence differences, or model disagreement. Their applicability depends on assumptions such as covariate shift, target calibration, or access to additional shift information.

  • Importance re-weighting: Importance re-weighting estimates target accuracy by weighting source errors with target-to-source covariate density ratios.This approach is particularly applicable to within-support covariate shift.
  • Importance re-weighting: Mandolin extends importance weighting by using additional supervision about the axis along which the distribution shifts.The paper compares against an importance-weighting baseline without assuming such axis information.
  • Confidence-based baselines: Average Confidence equates confidence with accuracy when the classifier is argmax calibrated on the target distribution.This baseline therefore relies on target-domain calibration.
  • Confidence-based baselines: Difference Of Confidence estimates target error by adjusting source error with the source-target difference in average maximum confidence.DOC-Feat uses this confidence difference as a distributional statistic.
  • Model disagreement: Generalized Disagreement Equality uses disagreement between independently trained models as an approximate measure of accuracy.The models share a training set but differ in initialization or data ordering.

F DETAILS ON THE DATASET SETUP

The evaluation spans synthetic and natural distribution shifts across vision, language, medical-imaging, satellite, and benchmark datasets. It includes corruptions, changed data-collection processes, novel subpopulations, geographic and temporal variation, reviewer differences, demographic identities, and experimental batch effects.

  • The testbed covers ImageNet, CIFAR, FMoW-WILDS, RxRx1-WILDS, Amazon-WILDS, CivilComments-WILDS, and MNIST.
  • ImageNet evaluation includes 95 ImageNet-C corruption datasets, three ImageNet-v2 datasets, ImageNet-R, and ImageNet-Sketch.The ImageNet-R and ImageNet-Sketch evaluations use ImageNet-200 because those datasets cover 200 classes.
  • BREEDS evaluates novel-subpopulation shifts by making training and test subpopulations disjoint, with 99 target datasets across same and different subpopulations.
  • CIFAR evaluation uses common corruptions for CIFAR-10 and CIFAR-100, plus CIFARv2 as a natural CIFAR-10 shift.
  • WILDS evaluations vary geography and time in FMoW, experimental batches in RxRx1, reviewers in Amazon, and demographic identities in CivilComments.FMoW produces 12 target sets; CivilComments produces 18 target datasets based on eight demographic identities.
  • MNIST evaluation uses USPS, SVHN, and QMNIST as three real shifted datasets.

G DETAILS ON THE EXPERIMENTAL SETUP

Experiments compare accuracy-estimation methods across standard neural architectures, datasets, and training procedures. Evaluation uses mean absolute error between estimated and true target accuracy, with additional checks for binary classification, calibration, and pretraining.

  • CIFAR models are trained from scratch with SGD for 300 epochs, while ImageNet experiments use Adam and evaluate both ImageNet and ImageNet-200.CIFAR-10 also includes ImageNet-pretrained models; ImageNet-200 permits longer training and use of ImageNet-R.
  • WILDS experiments use pretrained DenseNet121 and ResNet50 for FMoW and RxRx1, and fine-tuned DistilBERT models for Amazon and CivilComments.
  • BREEDS models are trained without pretraining using SGD, with DenseNet121 and ResNet50 evaluated on Living17, Nonliving26, Entity13, and Entity30.
  • For binary CivilComments classification, entropy and maximum-confidence scoring produce the same ordering, so temperature scaling does not change the estimates.
  • Pretraining does not change ATC’s efficacy on CIFAR, and pretrained DenseNet121 results show similar behavior to models trained from scratch.

H.2 BREEDS RESULTS WITH REGRESSION MODEL

The BREEDS regression analysis tests whether fitting DOC on labeled same-subpopulation target data transfers to novel subpopulations. ATC without regression remains competitive or superior in this setting.

  • BREEDS results compare DOC, GDE, ATC-NE, IM, and AC using predicted-versus-true OOD accuracy scatter plots and MAE summaries.
  • Fine-tuning DOC with a robust linear regression model on same-subpopulation shifts substantially improves DOC on novel subpopulations.
  • ATC without regression continues to outperform or match fine-tuned DOC on novel-subpopulation BREEDS shifts.
  • The ResNet50 evaluation reports MAE aggregated over four random seeds and states that lower MAE is better.
Loading 2201.04234v3…