PaperlayerView exact revision on arXiv

Source-linked AI summary

Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization

John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, Ludwig Schmidt

arXiv:2107.04649v2cs.LGstat.ML

TL;DR

Reliable machine learning requires understanding performance beyond the training distribution, but the relationship between in-distribution and out-of-distribution performance is not fully characterized. This paper investigates that relationship across models and distribution shifts, finding striking linear correlations in many settings while identifying exceptions and proposing a Gaussian-data theory involving covariance structure.

  • Problem

    The paper addresses how out-of-distribution performance relates to in-distribution performance, which standard same-distribution testing does not establish.

  • Method

    The paper empirically compares paired in-distribution and out-of-distribution performance across diverse models, shifts, training conditions, and probit-scaled accuracy plots, then develops a Gaussian data model.

  • Results

    Many models and distribution shifts exhibit striking linear correlations, with trends holding across architectures, hyperparameters, training-set sizes, and training durations.

  • Takeaways & Limitations

    Improving in-distribution performance reliably improves out-of-distribution performance in the studied settings, although robustness interventions may not always follow the same trend.

Abstract

from arXiv · show

For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distribution performance is strongly correlated with in-distribution performance for a wide range of models and distribution shifts. Specifically, we demonstrate strong correlations between in-distribution and out-of-distribution performance on variants of CIFAR-10 & ImageNet, a synthetic pose estimation task derived from YCB objects, satellite imagery classification in FMoW-WILDS, and wildlife classification in iWildCam-WILDS. The strong correlations hold across model architectures, hyperparameters, training set size, and training duration, and are more precise than what is expected from existing domain adaptation theory. To complete the picture, we also investigate cases where the correlation is weaker, for instance some synthetic distribution shifts from CIFAR-10-C and the tissue classification dataset Camelyon17-WILDS. Finally, we provide a candidate theory based on a Gaussian data model that shows how changes in the data covariance arising from distribution shift can affect the observed correlations.

1 Introduction

The paper asks how out-of-distribution performance relates to in-distribution performance and investigates this relationship across diverse models, datasets, and distribution shifts. It finds precise linear correlations in many settings, while identifying weaker cases and proposing a Gaussian-data explanation involving covariance structure.

  • Motivation: In-distribution evaluation provides only a narrow guarantee for samples from the training distribution, motivating the question of how performance transfers to new environments.The introduction frames out-of-distribution reliability as essential for systems deployed across homes, cities, hospitals, or time periods.
  • Motivation: Existing domain-adaptation bounds constrain possible accuracy differences using distributional distances but do not prescribe a specific trend within broad bounds.The paper contrasts these loose bounds with the precise empirical relationships it observes.
  • Contributions: Precise linear trends occur across dataset reproductions, benchmark changes, synthetic tasks, satellite imagery, and wildlife classification.Examples include CIFAR-10.2, ImageNet-V2, CINIC-10, YCB-Objects, FMoW-WILDS, and iWildCam-WILDS.
  • Contributions: The trends hold across convolutional networks, visual transformers, self-supervised models, classical methods, architectures, hyperparameters, training-set sizes, and training durations.The consistency extends beyond modern deep-learning architectures to logistic regression, nearest neighbors, and kernel machines.
  • Limitations and exceptions: The correlation is weaker or less regular for some CIFAR-10-C corruptions, Camelyon17-WILDS, and an alternative iWildCam-WILDS split.These cases are analyzed through additional experiments to investigate possible causes of the differing behavior.
  • Further analysis: Pretraining sometimes changes the observed trend, while a Gaussian data model identifies distribution-shift covariance structure as affecting performance correlation.ImageNet-pretrained models can follow the same trends in some settings but a different trend in iWildCam-WILDS.
  • Discussion: Improving in-distribution performance reliably improves out-of-distribution performance in the studied settings, but the best robustness interventions remain unresolved.The authors call for more research on the extent of the trends and whether interventions can improve beyond empirical risk minimization.

2 Experimental setup

The experimental framework compares in-distribution and out-of-distribution performance across independently trained models, using transformed scatter plots to inspect their relationship. The testbed spans varied model families and training choices, with large-scale evaluation and confidence intervals.

  • Evaluation framework: Each experiment studies losses on a training distribution D and an out-of-distribution distribution D1, usually training models only on D.In-distribution performance is measured on a held-out test set sampled from D, while OOD performance is evaluated on D1.
  • Experimental procedure: For each distribution pair, the procedure trains independently varied models, evaluates them on both test sets, and plots their paired accuracies.Variation includes training-set size, architecture, random seed, and optimization algorithm.
  • Axis scaling: The scatter plots apply a probit transform to each bounded loss axis, mapping [0,1] to an unbounded scale where linear trends become more precise.The probit transform is the inverse standard-Gaussian CDF; closely related logit scaling is also considered.
  • Visualization: Each plotted point represents one model rather than an average over random seeds, and accuracies include 95% Clopper-Pearson confidence intervals.This preserves model-level variation while displaying uncertainty in the test accuracies.
  • Testbed: The unified testbed includes classical methods and high-performance convolutional networks, covering more than 3,000 trained models and 100,000 test-set evaluations.The scale reflects evaluations of models and their training checkpoints across the experiments.

3 The linear trend phenomenon

Precise linear relationships between in-distribution and out-of-distribution performance appear across diverse models, datasets, domains, and distribution shifts. These relationships persist across changes in model and training conditions, with additional examples spanning benchmark, synthetic, and real-world shifts.

  • Precise ID-OOD linear trends occur across models, data domains, and distribution shifts.The experiments cover five datasets: CIFAR-10, ImageNet, FMoW-WILDS, iWildCam-WILDS, and YCB-Objects.
  • Dataset reproduction shifts: Dataset reproductions such as CIFAR-10.1 and ImageNet-V2 exhibit linear relationships between original ID and new OOD accuracies.The paper extends earlier findings to both deep and classical models trained on CIFAR-10.
  • Distribution shifts between machine learning benchmarks: CIFAR-10 models also follow linear trends when evaluated on compatible benchmarks including CINIC-10 and STL-10.
  • Synthetic perturbations: Many CIFAR-10-C perturbations, including Fog, preserve linear trends, while isotropic Gaussian noise is a notable exception.YCB-Objects pose-estimation models also follow a linear trend under perturbed lighting and texture conditions.
  • Distribution shifts in the wild: FMoW-WILDS models exhibit linear trends under temporal and subpopulation shifts, while iWildCam-WILDS is included among the studied real-world shifts.FMoW-WILDS evaluates satellite imagery across geographic regions and later time periods, including worst-region accuracy.
  • Variations in model hyperparameters, training duration, and training set size: On CIFAR-10 and CIFAR-10.2, hyperparameter changes, training duration, and training-set size move models along the same trend rather than changing the linear fit.Each variation produced R2 “ 0.99, and the same invariance was also found for CINIC-10, CIFAR-10-C Fog, YCB-Objects, and FMoW-WILDS.

4 Distribution shifts with weaker correlations

Some distribution shifts produce weaker or less precise ID-OOD relationships than the linear-trend cases. Camelyon17-WILDS shows substantial OOD variability, while CIFAR-10-C results depend strongly on the corruption and on covariance alignment.

  • Camelyon17-WILDS: At 95% ID accuracy, Camelyon17-WILDS models span approximately 50% to 95% OOD accuracy.The variation remains significant even after averaging each model over ten independent training runs.
  • Camelyon17-WILDS: Strong data augmentation significantly reduces, but does not eliminate, OOD variation, whereas ImageNet pretraining does not increase the ID-OOD correlation.
  • Camelyon17-WILDS: Camelyon17-WILDS has no precise linear trend, with highly variable OOD accuracy across ID accuracies.The dataset shifts across hospitals because of differences in staining and imaging protocols.
  • CIFAR-10-Corrupted: CIFAR-10-C corruption choice substantially affects ID-OOD correlation, with Gaussian noise among the least precise cases.Fog corruption is presented as a more precise fit, while Gaussian noise is mathematically easy to describe but performs poorly in correlation.
  • CIFAR-10-Corrupted: Matching OOD and ID data covariance improves the linear fit to R2 “ 0.93 versus R2 “ 0.44 for isotropic Gaussian noise.This comparison tests the Gaussian data model’s prediction that linear fits occur when covariances match up to constant scaling.
  • CIFAR-10-Corrupted: Every corruption examined is more linear on ImageNet-C than on CIFAR-10-C, a discrepancy left for future investigation.

5 The effect of pretrained models

Pretraining and fine-tuning improve both ID and OOD performance, but their effects on the ID-OOD relationship depend on the task and training procedure. Fine-tuning preserves the from-scratch trend on some datasets, changes it on iWildCam-WILDS, and zero-shot models follow a distinct trend.

  • The effect of pretrained models is nuanced because auxiliary data changes the meaning of ID while task and use of the pretrained model determine the observed relationship.
  • Fine-tuned pretrained models improve both ID and OOD performance across the evaluated tasks.
  • On CIFAR-10 reproductions and FMoW-WILDS, fine-tuning keeps models on the same ID-OOD trend as models trained from scratch.
  • On iWildCam-WILDS, fine-tuning changes the ID-OOD relationship, and fine-tuning weight decay also affects the resulting trend.
  • Zero-shot ImageNet models on CIFAR-10 deviate from the basic trend and form a different trend closer, but not identical, to x = y.

6 Summary of empirical phenomena

Across the empirical studies, ID-OOD linear trends are broad and precise but not universal. Their stability spans model and training variations, while scaling and distribution-shift choice affect their precision and presence.

  • Linear ID-OOD trends hold across model families and variation in architecture, hyperparameters, and training duration.
  • Probit or logit scaling on both axes makes the observed linear trends more precise.
  • Changing the amount of ID training data does not affect the linear trend, whereas pretraining on another distribution can change it.
  • Some distribution shifts produce precise linear trends only for subsets of models or not at all.

7 Theoretical models for linear fits

The paper develops a simple Gaussian shift model to explain linear ID-OOD relationships and to identify mechanisms that can break them. Its predictions match linear and nonlinear classifier behavior, while covariance and model-dependent shifts produce departures.

  • 7.1 A simple Gaussian distribution shift setting: The theoretical model uses binary labels and isotropic Gaussian features, with shifts represented by changes in the mean and variance parameters.
  • 7.1 A simple Gaussian distribution shift setting: Theorem 1 predicts a near-linear relationship between probit-transformed ID and OOD accuracies when the classifier is independent of the random shift direction.
  • 7.1 A simple Gaussian distribution shift setting: The deviation from linearity is of order d^-1/2 and vanishes in high dimension.
  • 7.1 A simple Gaussian distribution shift setting: Simulations agree with the theoretical trend for logistic and ridge regression, while nearest neighbors and random forests exhibit the same probit trend.
  • 7.2 Modeling departures from the linear trend: Adversarial shifts, auxiliary pretraining data, and non-isotropic covariance can break the linear trend, whereas scalar covariance scaling preserves it.
  • 7.2 Modeling departures from the linear trend: The simplified theory sheds light on departures but does not precisely distinguish several real-world shifts, which the paper leaves for future work.

8 Related work

Prior domain-generalization theory gives bounds that do not explain highly predictable yet different OOD performance. This paper broadens empirical evidence for linear trends beyond dataset reproductions and relates them to transfer-learning findings.

  • Domain generalization theory: Classical domain-generalization bounds relate source and target errors through distribution or classifier-induced divergence measures.
  • Domain generalization theory: These bounds do not capture the paper’s linear trends, where OOD performance is predictable from ID performance despite being significantly different.
  • Domain generalization theory: The paper’s Gaussian model explains linear trends in particular distributions and for linear classifiers, but is limited in scope compared with general bounds.
  • Empirical observations of linear trends: Earlier empirical studies focused mainly on dataset reproductions, whereas this work includes additional shifts, real-world applications, and negative examples.
  • Transfer learning: Transfer-learning results showing linear correlations between ImageNet accuracy and post-fine-tuning accuracy suggest a broader phenomenon of predictable generalization.
  • Transfer learning: Concurrent work studies fine-tuning and effective robustness on CIFAR-10 and ImageNet, while this paper examines a complementary set of datasets.

9 Discussion

The paper formalizes when out-of-distribution performance tracks in-distribution performance and discusses implications for model selection and robustness research. It finds broad evidence for linear trends, while emphasizing that the property is conditional on model families and distribution shifts.

  • Correlation property: Linear in-distribution/out-of-distribution trends occur across neural and classical models, including convolutional networks, vision transformers, self-supervised models, logistic regression, nearest neighbors, and kernel machines.The trends also persist when varying architectures, hyperparameters, training-set size, or training duration.
  • Correlation property: The correlation property is defined by an approximate relationship between losses on two distributions under a monotone transform.The paper uses this definition to formalize when out-of-distribution performance is predictable from in-distribution performance.
  • Correlation property: The paper conjectures that CIFAR-10 to CIFAR-10.1 and related shifts have a 1%-approximate correlation property for models trained by ERM on the corresponding training distribution.The conjecture is posed for shifts including ImageNet to ImageNet-V2 and FMoW-WILDS.
  • Possible implications: When the property holds, selecting the best in-distribution model can reduce to selecting the best model for shifted distributions.The implication is explicitly conditional, and the paper notes that more research is needed to establish its extent.
  • Possible implications: A robustness method should improve out-of-distribution performance beyond the ERM trend or reduce the in-distribution/out-of-distribution gap relative to existing in-distribution methods.The paper recommends reporting both performance measures and visualizing relevant models with scatter plots.
  • Possible implications: Pre-training can preserve the same relationship in some settings but produce different in-distribution/out-of-distribution relationships in others.The paper gives ImageNet pre-training on CIFAR-10 and FMoW-WILDS as examples of preserved trends, and iWildCam-WILDS as an exception.
  • Methodological qualification: The linearity may depend on the performance transformation: random interpolation is linear in raw accuracy but not necessarily after a nonlinear transform such as the probit transform.Characterizing the models for which approximate linearity holds remains future work.

A Experimental testbeds

The experimental program uses multiple testbeds and dataset universes to evaluate models across in-distribution and shifted distributions. Its datasets cover benchmark reproductions, synthetic perturbations, satellite imagery, wildlife camera traps, and related WILDS tasks.

  • Testbeds: The study uses three testbeds: a custom CIFAR-10/WILDS testbed, an ImageNet testbed, and a YCB-Objects pose-estimation testbed.Each testbed contains compatible datasets and models that can be trained and evaluated across distributions.
  • Dataset universes: The main custom testbed contains four dataset universes: CIFAR-10, FMoW-WILDS, Camelyon17-WILDS, and iWildCam-WILDS.Each universe contains at least three datasets for in-distribution and out-of-distribution evaluation.
  • CIFAR-10 universe: The CIFAR-10 universe includes CIFAR-10, two dataset reproductions, CINIC-10, STL-10, and CIFAR-10-C.These datasets represent reproduction, source-image, class-structure, and synthetic-perturbation shifts.
  • WILDS datasets: FMoW-WILDS classifies 62 land- or building-use categories from satellite images spanning geographical regions and years.Its in-distribution data cover 2002–2013, while out-of-distribution evaluation includes later periods and, for worst-region accuracy, regional shifts.
  • WILDS datasets: iWildCam-WILDS evaluates animal-species classification from camera-trap images whose shifts arise from differences in camera locations, angles, lighting, and backgrounds.Performance is measured with macro F1 to account for rare species.
  • Evaluation: The iWildCam macro-F1 confidence interval uses a heuristic built from per-class Clopper-Pearson intervals and combines them across classes.The intervals are relatively large because many test classes are rare, including classes with 10 or fewer examples.
  • Models: The CIFAR-10 and WILDS testbed includes models trained with additional unlabeled data from 80 Million Tiny Images.This baseline tests whether extra data from a potentially more diverse source move models away from the trend observed for CIFAR-10-only training.

B.1.1 CIFAR-10

The CIFAR-10 experiments show highly regular relationships between in-distribution accuracy and out-of-distribution accuracy across dataset reproductions, benchmark shifts, and selected synthetic perturbations. These relationships hold across broad model collections and training variations.

  • Dataset reproduction shifts: R2 > 0.99 for each CIFAR-10 dataset-reproduction shift, with linear fits capturing both classical and neural models on CIFAR-10.1 and CIFAR-10.2.The evaluations plot out-of-distribution test accuracy against in-distribution CIFAR-10 test accuracy.
  • Benchmark shifts: CINIC-10 and STL-10 also show linear relationships with in-distribution CIFAR-10 accuracy across the CIFAR-10 testbed models.The STL-10 comparison uses nine-class variants because its class structure differs from CIFAR-10.
  • Synthetic perturbations: Eight selected CIFAR-10-C perturbations exhibit clean linear trends between in-distribution and out-of-distribution accuracy.The perturbations include fog, brightness, snow, defocus blur, spatter, elastic transform, frost, and saturate.
  • Figure 7: Figure 7 compares out-of-distribution accuracies with in-distribution CIFAR-10 test accuracies across four distribution shifts and reports 95% Clopper-Pearson confidence intervals.The shifts are CIFAR-10.1, CIFAR-10.2, CINIC-10, and STL-10.

B.1.2 FMoW-WILDS

FMoW-WILDS experiments evaluate ID/OOD accuracy across classical and neural models, metrics, hyperparameters, training duration, and training-set size. Across these variations, the supplied passages report precise linear trends between ID and OOD performance.

  • FMoW-WILDS compares classical methods and ImageNet networks using average and worst-region OOD accuracy on validation and test sets.
  • On the ImageNet natural shifts, CLIP models are robust across shifts, while the standard-network fit is less predictive for ObjectNet, YTBB-Anchors, and ImageNet-A in the low-accuracy regime.
  • The synthetic YCB pose-estimation task also follows a linear ID/OOD relationship, with R2 = 0.99 under a background-texture shift.
  • The experiments vary model hyperparameters, training duration, and training-set size while comparing evaluations to a common linear trend.
  • The YCB-Objects pose-estimation relationship remains linear when training-set size changes, with R2 = 0.98 and no change to the fitted line.

C.1 Camelyon17-WILDS

Camelyon17-WILDS exhibits much weaker ID/OOD predictability than the paper’s stronger examples. Averaging seeds, early stopping, and targeted augmentation clarify how training randomness and data augmentation affect the observed variation.

  • Averaging ten training runs leaves weak ID/OOD correlation, with R2 = 0.39 versus R2 = 0.40 without averaging.
  • Early stopping on the ID validation set increases correlation to R2 = 0.77, whereas stopping on a separate OOD validation set gives R2 = 0.46.
  • ImageNet pretraining produces extremely high performance variability, with R2 = 0.05 and seed averaging improving it only to R2 = 0.14.
  • Targeted color-jitter augmentation increases average OOD performance and yields R2 = 0.77, while averaging augmented models raises correlation to R2 = 0.95.
  • Correlated image patches from the same slides and hospitals may contribute to high OOD variation in this dataset.
  • The iWildCam-WILDS v1.0 split shows a far less pronounced ID/OOD correlation, with fine-tuned models near-vertical across OOD values.

D.1 Detailed findings for CIFAR-10

Detailed CIFAR-10 experiments test whether ID/OOD linear trends persist across auxiliary data, pretrained models, metrics, and iWildCam comparisons. The results show that training source and model treatment can shift or preserve the relationship.

  • Models trained on ImageNet-derived CINIC-10 data lie above the CIFAR-10/CIFAR-10.2 linear fit and outperform ImageNet zero-shot models.
  • Zero-shot and linear-probe CLIP models also lie above the CIFAR-10/CIFAR-10.2 trend, especially the Vision Transformer.
  • Auxiliary unlabeled data experiments include self-training and out-distribution-aware self-training using 80 Million Tiny Images.
  • Fine-tuning improves FMoW-WILDS performance without deviating from the ID/OOD line across four combinations of average- and worst-region metrics.
  • For iWildCam-WILDS fine-tuned models, nonzero weight decay produces a different ID/OOD trend above the zero-weight-decay line, while scratch-trained effects remain uncertain.

E.1 Proof of Theorem 1

The theoretical analysis uses a Gaussian data model to explain when ID/OOD probit accuracies follow a linear trend and how specific covariance, adversarial, or auxiliary-data changes can break it.

  • Theorem 1 assumes the distribution-shift parameter ∆ is independent of the classifier and yields a high-probability relationship between ID and OOD accuracies.
  • Applying Theorem 1 with a union bound extends the guarantee to multiple linear classifiers and implies a linear trend in scatter plots.
  • Adding auxiliary data related to the shifted distribution can improve OOD performance while worsening ID performance, although the theory does not guarantee the resulting fit.
  • An adversarial shift aligned with a target classifier can substantially reduce that model’s OOD performance and break the linear trend.
  • Non-isotropic covariance shifts break linearity because ID and OOD performance favor different classifier norms; stronger ℓ2 regularization can improve OOD performance.

F Additional related work

Related work frames distribution-shift behavior as nuanced: some shifts show precise linear relationships between in-distribution and out-of-distribution performance, while others do not. Existing theory, robustness methods, and benchmark studies provide partial explanations but do not generally predict where linear trends will occur.

  • Theoretical models: Earlier hardness-mixture and Gaussian-hardness models reproduce linear fits, but neither those models nor this paper’s model predicts which real distributions will exhibit linear trends.The paper’s theoretical analysis instead uses a concrete generative model that permits studying covariance modifications from distribution shift.
  • Underspecification: Evidence on underspecification is nuanced: ImageNet-C and CIFAR-10-C contain both substantial accuracy variation and comparatively clean linear trends, while ObjectNet remains broadly predictable from ImageNet accuracy.Related medical-imaging results likewise differ across datasets and statistical tests.
  • Counterexamples: Adversarial shifts are a prominent counterexample, with standard CIFAR-10 accuracy around 91% but only 0–10% robust accuracy without robustness intervention against the common ℓ8 setting.Robust training can reach about 65% robust accuracy while retaining 91% standard accuracy, showing that no precise universal linear trend holds in this setting.
  • Robustness methods: Robustness interventions include transfer-learning and domain-adaptation methods that use target-distribution samples, alongside methods that do not use such samples.The related literature also connects linear trends in transfer learning to similarities with the shifts studied here.
Loading 2107.04649v2…