PaperlayerView exact revision on arXiv

Source-linked AI summary

OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization

Nanyang Ye, Kaican Li, Haoyue Bai, Runpeng Yu, Lanqing Hong, Fengwei Zhou, Zhenguo Li, Jun Zhu

arXiv:2106.03721v3cs.LG

TL;DR

OoD generalization research spans disconnected areas, while evidence about the datasets used to assess algorithms remains limited. The paper quantifies diversity and correlation shifts and evaluates algorithms across both; it finds that most surpass ERM in at most one shift category.

  • Problem

    OoD generalization research spans several areas, and the data used to evaluate its algorithms remain insufficiently understood.

    Source: §1 · p. 1

  • Method

    The paper quantifies diversity and correlation shifts in OoD datasets and evaluates algorithms across datasets dominated by each type.

    Source: §1 · pp. 1–2

  • Results

    Most OoD generalization algorithms surpass ERM in at most one of the two shift categories.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors recommend evaluating algorithms on datasets dominated by both shift types and investigating the shift before designing algorithms.

    Source: §1 · p. 2

    Each heatmap grid cell was run once, so the heatmaps also reflect estimation variance that averaging over multiple runs could compensate for.

    Source: §3.2 · p. 7

Abstract

from arXiv · show

Deep learning has achieved tremendous success with independent and identically distributed (i.i.d.) data. However, the performance of neural networks often degenerates drastically when encountering out-of-distribution (OoD) data, i.e., when training and test data are sampled from different distributions. While a plethora of algorithms have been proposed for OoD generalization, our understanding of the data used to train and evaluate these algorithms remains stagnant. In this work, we first identify and measure two distinct kinds of distribution shifts that are ubiquitous in various datasets. Next, through extensive experiments, we compare OoD generalization algorithms across two groups of benchmarks, each dominated by one of the distribution shifts, revealing their strengths on one shift as well as limitations on the other shift. Overall, we position existing datasets and algorithms from different research areas seemingly unconnected into the same coherent picture. It may serve as a foothold that can be resorted to by future OoD generalization research. Our code is available at https://github.com/ynysjtu/ood_bench.

1. Introduction

OoD generalization is difficult because training data can misrepresent real-world distributions, undermining reliability, while research and evaluation span fragmented areas. The paper quantifies two kinds of shift and compares algorithms across them, finding that most surpass ERM in at most one category.

1. Introduction

  • Distribution mismatches are common in the real world and have led to performance drops, undermining reliability in applications such as medical imaging, autonomous driving, and security.Source: §1 · p. 1
  • OoD generalization research spans areas including domain generalization, causal inference, and stable learning, while prior comparisons suggest advanced algorithms may remain on par with ERM.Source: §1 · p. 1
  • The paper quantifies shifts in OoD datasets and evaluates algorithms across both shift types, recommending comprehensive evaluation and shift analysis before algorithm design.Source: §1 · pp. 1–2
  • Most OoD algorithms surpass ERM in at most one of two distinct shift categories, which the authors hypothesize helps explain their limited gains over ERM.The paper frames the categories as diversity shift and correlation shift.Source: §1 · p. 2
  • The authors recommend designing large-scale datasets that capture subtle real-world shifts, which can still be conspicuous to neural networks.Source: §1 · p. 2

2. Diversity Shift and Correlation Shift

The paper characterizes distribution shift through diversity and correlation dimensions, then estimates both from labeled training and test data. Its estimates decompose shifts in varied datasets, while the authors report stable estimation across network architectures.

2. Diversity Shift and Correlation Shift

  • For ImageNet and ImageNet-V2, the estimate identifies non-trivial correlation shift despite an almost imperceptible discrepancy, and other OoD datasets show varying degrees of both shifts.Source: §2 · p. 2
  • Diversity shift concerns differences in latent-environment distributions across support sets, whereas correlation shift concerns density differences on the same support set.Source: Figure 2 · p. 3
  • The formal setup assumes a shared labeling rule and no label shift, with invariant causal features Z1 and other features Z2 that may vary across environments.The paper notes label shift can be addressed in practice with techniques such as sample reweighting.Source: §2 · pp. 2–3
  • The authors partition Z2 into features absent from one environment, associated with diversity shift, and shared features whose label relationship changes, associated with correlation shift.Source: §2 · p. 3
  • The proposed measures are bounded from 0 to 1, and the correlation-shift formula downweights features that hardly appear in either environment.The authors describe these formulations as intuitively reasonable and numerically stable, while noting they are not the only viable formulations.Source: §2 · p. 3
  • The estimator trains a network to distinguish equally sized training and test samples, uses KDE on its learned features, and approximates the shift integrals by importance-sampled Monte Carlo integration.The environment-classification objective encourages features whose joint distribution with labels varies across environments.Source: §2 · pp. 3–4
  • The authors report that feature extraction converges to a unique solution as network width grows and that experiments across architectures show stable estimates.Source: §2 · p. 4
  • Most existing OoD datasets lie over or near one shift axis, while ImageNet-A, ImageNet-R, and ImageNet-V2 are decomposed across diversity and correlation shift.The Figure 3 estimates for ImageNet variants use original ImageNet as the reference.Source: §2 · p. 4, Figure 3 · p. 4

3. Experiment

The experiments benchmark OoD algorithms across datasets dominated by diversity or correlation shift, and test whether the shift estimates behave as expected. Results show that no algorithm consistently outperforms ERM across both shift directions.

3.1. Benchmark

  • The benchmark covers PACS, OfficeHome, Terra Incognita, and Camelyon17-WILDS for diversity shift, and Colored MNIST, NICO, and modified CelebA for correlation shift.For PACS, OfficeHome, and Terra Incognita, each domain is held out in turn; other datasets use fixed training and test environments.Source: §3.1 · p. 4
  • No OoD generalization algorithm consistently outperforms ERM across both shift directions: RSC, MMD, and SagNet score higher on diversity-shift datasets but lower on correlation-shift datasets, while VREx and GroupDRO show the reverse.The benchmark uses separate dataset groups dominated by each type of shift.Source: Table 1 · p. 5, Table 2 · p. 6, §3.1 · p. 6
  • The study compares ERM with representative OoD algorithms and uses dataset-specific model-selection procedures because no consensus exists on selection methods.Validation uses training domains for PACS, OfficeHome, and Terra Incognita; OoD validation for Camelyon17-WILDS and NICO; and test-domain validation for Colored MNIST and CelebA.Source: §3.1 · p. 5
  • ResNet-18 is used for all datasets except Colored MNIST, which uses a two-layer perceptron; NICO is not ImageNet-pretrained because its classes overlap substantially with ImageNet.The authors chose smaller models partly because larger models may be more robust to OoD data and saturate performance on small datasets.Source: §3.1 · p. 5
  • The magnitude of diversity or correlation shift does not measure absolute generalization difficulty, and noise and changing factors leave cross-dataset patterns unclear.The authors frame shift magnitude as an indicator of the likelihood that some algorithms outperform others under the same type of shift.Source: §3.1 · pp. 6–7

3.2. Further Study

  • When training and test color correlations are similar, estimated correlation shift is negligible; varying the blue-color distribution changes estimated diversity shift, and the estimates show a trade-off between shift types.The Colored MNIST sanity checks manipulate red/green correlation and blue intensity; each grid cell is run once, so heatmaps also reflect estimation variance that multiple runs could reduce.Source: §3.2 · p. 7
  • EMD and MMD are insensitive to correlation shift, EMD is also insensitive to diversity shift, and NI measures only one dimension; OoD-Bench produces more stable and interpretable estimates.As training and test color correlations approach one another, the estimated correlation shift decreases to zero, while estimated diversity shift remains zero until the final scenario.Source: §3.2 · p. 7

4. Related Work

Prior distribution-shift metrics generally do not distinguish shift types, while OoD algorithms span multiple research traditions. OoD-Bench differs from DomainBed by benchmarking datasets and interpreting algorithm performance across diversity and correlation shift.

4. Related Work

  • Non-i.i.d. Index, KL divergence, EMD, MMD, and A-distance measure distribution discrepancy but do not distinguish different kinds of distribution shift.The paper identifies diversity and correlation shift as two dimensions and provides quantitative results on OoD datasets.Source: §4 · p. 7
  • Domain-generalization approaches include invariant representations, domain adversarial learning, meta-learning, mixup, feature or gradient alignment, and invariant risk minimization.Some methods assume multiple training domains; others address single-source settings or assume style and texture variation drives shift.Source: §4 · p. 8
  • DomainBed reports fourteen algorithms on seven datasets and concludes ERM beats most DG algorithms; OoD-Bench additionally benchmarks datasets and compares algorithms by diversity and correlation shift.OoD-Bench also includes new algorithms and datasets, especially datasets dominated by correlation shift.Source: §4 · p. 8

5. Conclusion

The conclusion presents diversity and correlation shift as a unified characterization of OoD datasets and reports strengths and weaknesses among existing algorithms. It calls for evaluation on both types of datasets and leaves open whether one algorithm can handle both shifts.

5. Conclusion

  • The paper identifies diversity and correlation shift as two main forms in OoD datasets, uses them to position datasets in a unified picture, and reports strengths and weaknesses of existing algorithms.The authors recommend evaluating algorithms on datasets dominated by each shift and leave open whether any algorithm performs well under both.Source: §5 · p. 8

B. Proofs

The proofs bound the two shift measures and characterize optimal environment classification. The appendix also describes practical estimation using learned features and empirical distributions.

B. Proofs

  • Diversity shift and correlation shift are each bounded between 0 and 1 for any two environment probability functions.The proof establishes non-negativity and upper bounds for both measures.Source: B. Proofs · p. 13
  • With equal numbers of examples from two environments, the best environment-classification accuracy is determined by the environments as one half the integral of max{p(x), q(x)}.The maximizing classifier predicts the environment from which an example is more likely sampled.Source: B. Proofs · pp. 13–14
  • A network reaches optimal environment-classification performance only if every non-i.i.d. input has some label whose conditional distribution differs across environments in its learned representation.The theorem states this condition for representations z = g(x).Source: B. Proofs · p. 14

C.1. Pseudo codes and supporting theoretical justifications

  • The feature extractor and environment classifier are trained with balanced sampling across environments and labels, under the assumption that label distributions match and there is no label shift.The estimation algorithm fits kernel density estimates and uses importance sampling with thresholds to estimate the two shifts.Source: §C.1 · pp. 14–15

C. Practical Estimation of Diversity and Correlation Shift

  • The practical estimator replaces environment distributions with empirical estimates and integrates diversity and correlation differences over feature-space subsets.The estimator uses subsets defined by whether the estimated environment distributions have shared support; the feature extractor processes training and test inputs before estimation.Source: §C.1 · p. 15

C.2. Implementation details

The implementation uses fixed neural-network and optimization settings to estimate distribution shifts. The convergence analysis shows that hidden features approach a kernel-governed differential-equation solution as network width grows.

C.2. Implementation details

  • The implementation uses MLPs for Colored MNIST and ImageNet-pretrained ResNet-18s for other datasets, with Adam, 2,000 iterations, and feature dimension 8.The batch size is 32 per environment; 90% of each random split is used for training and 10% for validation.Source: §C.2 · p. 16

D. Discussion on the Convergence of the Hidden Feature

  • The feature dynamics use a continuous-time index, with the hidden-layer output evolving according to its kernel and loss-gradient terms.The passage defines G as the loss gradient for an intermediate-layer output and u as that output across training examples.Source: D. Discussion on the Convergence of… · pp. 16–17
  • As hidden-layer widths grow sequentially, the extracted feature converges to a system of differential equations governed by fixed kernels.The analysis uses the neural tangent kernel for hidden-layer outputs.Source: D. Discussion on the Convergence of… · pp. 16–17

E. Effects of the Neural Network Architecture

The authors test how network architecture affects estimates of diversity and correlation shift. Across the tested settings, estimates show no significant architectural difference and tend to converge as capacity increases.

E. Effects of the Neural Network Architecture

F. Estimation Results with Error Bars

The section presents estimation results with standard error bars. The reported statistics average five runs with different initializations and training/validation splits.

F. Estimation Results with Error Bars

G. Datasets

The benchmark groups datasets by whether diversity shift or correlation shift dominates. Its diversity-shift group spans domain-generalization datasets and hospital-collected pathology images, while its correlation-shift group includes settings with spurious attribute or context correlations.

Diversity-shift datasets

  • The diversity-shift group includes PACS, OfficeHome and Terra Incognita, whose domains or locations vary across visual styles, image types or camera-trap sites.PACS has four styles; OfficeHome has four domains; Terra Incognita uses four camera locations.Source: G. Datasets · p. 19

Correlation-shift datasets

  • Colored MNIST makes digit color strongly class-correlated, with that correlation differing between training and test; NICO uses animal and vehicle contexts to construct similar spurious correlations.The NICO split uses four overlapping contexts: “on snow”, “in forest”, “on beach” and “on grass”.Source: G. Datasets · p. 19

Diversity-shift datasets

  • CelebA uses hair color as the target and gender as the spurious attribute across three environments, while Camelyon17-WILDS comprises pathology slides from five hospitals.The CelebA subset has 27,040 images; Camelyon17-WILDS has 455,954 examples.Source: G. Datasets · p. 19

H. Model Selection Methods

The section compares training-domain, test-domain and OoD validation for model selection. OoD validation substitutes for leave-one-domain-out validation because it avoids substantial computational cost and preserves training data when few domains are available.

Training-domain validation

  • Training-domain validation selects the model with the best accuracy on pooled held-out training-domain data, assuming training and test examples have similar distributions.Each training domain is split into training and validation subsets.Source: H. Model Selection Methods · p. 20

Test-domain validation

  • Test-domain validation selects the model with the best accuracy on validation data matching the test distribution, allowing one query per hyperparameter choice at the last checkpoint.Early stopping is disallowed.Source: H. Model Selection Methods · p. 20

OoD validation

  • Leave-one-domain-out validation is replaced by OoD validation because it multiplies experiments by k −1 and, with few domains, can sharply reduce training examples.The authors also cite the existing OoD validation set for Camelyon17 as a reason for this choice.Source: H. Model Selection Methods · p. 20
  • OoD validation selects the model with the best accuracy on data matching neither training nor test distributions, assuming performance there predicts test generalization.Source: H. Model Selection Methods · p. 20

I. ImageNet-V2 Experiment

On ImageNet-V2, ERM outperforms the tested alternatives, reversing their ranking relative to diversity-shift results. The appendix also describes the hyperparameter search space used for comparisons.

I. ImageNet-V2 Experiment

  • ImageNet comparisons are limited to CORAL, SagNet and RSC because the variants are small validation sets and there is no standard multi-domain ImageNet construction.ImageNet is also seldom considered in domain-generalization literature, and most OoD algorithms assume multiple training domains.Source: §I · p. 20
  • ERM outperforms CORAL, SagNet and RSC on ImageNet-V2, reversing the algorithms’ relative ranking from diversity-shift results.ImageNet and ImageNet-V2 are described as having dominant correlation shift; Table 11 reports the performance comparison.Source: Table 11 · p. 20, §I · p. 20

J. Hyperparameter Search Space

Related papers

Loading 2106.03721v3…