PaperlayerView exact revision on arXiv

Source-linked AI summary

BREEDS: Benchmarks for Subpopulation Shift

Shibani Santurkar, Dimitris Tsipras, Aleksander Madry

arXiv:2008.04859v1cs.CVcs.LGstat.ML

TL;DR

Models often encounter data subpopulations absent from training, motivating benchmarks for robustness to subpopulation shift. BREEDS uses existing class structure to construct controlled, natural shifts and applies them to ImageNet, where models remain substantially challenged despite robustness interventions.

  • Problem

    Models need to generalize to unseen data subpopulations because training datasets do not fully capture deployment diversity.

  • Method

    BREEDS groups semantically similar existing classes into superclasses and partitions their subclasses into disjoint source and target subpopulations to create controlled benchmarks.

  • Results

    Existing models still have a long way to go before fully tackling BREEDS subpopulation shifts, even with robustness interventions.

  • Takeaways & Limitations

    BREEDS provides a framework for studying model robustness to distribution shift using realistic, controlled shifts within existing large-scale datasets.

  • Takeaways & Limitations

    Directly using WordNet produces poorly calibrated tasks because hierarchy depth does not consistently reflect category specificity and can induce skewed shifts across classes.

Abstract

from arXiv · show

We develop a methodology for assessing the robustness of models to subpopulation shift---specifically, their ability to generalize to novel data subpopulations that were not observed during training. Our approach leverages the class structure underlying existing datasets to control the data subpopulations that comprise the training and test distributions. This enables us to synthesize realistic distribution shifts whose sources can be precisely controlled and characterized, within existing large-scale datasets. Applying this methodology to the ImageNet dataset, we create a suite of subpopulation shift benchmarks of varying granularity. We then validate that the corresponding shifts are tractable by obtaining human baselines for them. Finally, we utilize these benchmarks to measure the sensitivity of standard model architectures as well as the effectiveness of off-the-shelf train-time robustness interventions. Code and data available at https://github.com/MadryLab/BREEDS-Benchmarks .

1 Introduction

BREEDS addresses whether models can generalize to unseen data subpopulations, a challenge because real-world deployment encounters diversity absent from training data. It constructs controlled benchmarks from existing dataset structure and uses them to study model robustness and train-time interventions.

  • Models must generalize to data subpopulations that were not seen during training.
  • Unseen subpopulations arise because training datasets cannot perfectly capture real-world diversity, including unexpected weather or diagnostic setups.
  • BREEDS repurposes existing class labels and semantically similar superclass groupings to create classification tasks whose training and test subpopulations are disjoint.
  • Standard models lose substantial accuracy under subpopulation shifts, although models with higher source accuracy tend to preserve a larger fraction of it in the target domain.
  • Train-time robustness interventions provide small, non-trivial improvements, sometimes at the expense of original-domain performance, but do not significantly resolve sensitivity to subpopulation shift.

2 Designing Benchmarks for Distribution Shift

Distribution-shift benchmarks must define realistic target distributions while isolating the intended source of change. BREEDS addresses this by simulating shifts within existing datasets, avoiding input modifications and reducing external influences.

  • Benchmark design centers on specifying a realistic target distribution that varies from the training source distribution.
  • Data-corruption benchmarks modify source inputs with transformations intended to mimic real-world corruptions.
  • Data-source benchmarks evaluate models on independently collected datasets to test reliance on training-dataset idiosyncrasies.
  • Subpopulation-shift benchmarks vary how well represented different subpopulations are between source and target distributions.
  • External factors in synthetic transformations or new data collection can obscure the intended shift and distort robustness assessments.
  • BREEDS limits such influences by simulating distribution shifts within existing datasets without modifying inputs.

3 The BREEDS Methodology

BREEDS constructs controlled subpopulation shifts by deriving hierarchical superclasses from existing labels and assigning disjoint subclass groups to source and target domains. The methodology supports benchmarks with tunable granularity, known shifts, and reuse across datasets.

  • The BREEDS Methodology: BREEDS targets shifts where test subpopulations are entirely absent from the training distribution.
  • Devising subpopulation structure: The method bootstraps subpopulation annotations by grouping semantically similar dataset classes into broader, hierarchical superclasses.
  • Simulating subpopulation shifts: For each superclass, subclasses are randomly split into disjoint source and target sets, creating the simulated shift.
  • Extensions: The methodology can also represent milder shifts in which source and target distributions overlap but differ in relative subpopulation frequencies.
  • Benefits: Different semantic groupings produce BREEDS tasks with varying granularity and shift severity, from similar classes to broad superclasses with many subclasses.
  • Benefits: Because both domains come from one dataset, the exact shift is characterized while external data-collection effects are minimized.
  • Benefits: Randomly assigning subpopulations to source and test domains is expected to yield tasks with comparable difficulty.
  • Benefits: The approach requires no additional data collection or annotation beyond selecting class groupings and can be reused across large-scale datasets.

4 Simulating Subpopulation Shifts Within ImageNet

BREEDS adapts ImageNet’s hierarchy into calibrated, controllable subpopulation-shift benchmarks, then validates their perceptual tractability with human studies. The resulting tasks vary in granularity and separate source and target subclasses while preserving the same high-level classification task.

  • 4.1 Utilizing the ImageNet class hierarchy: WordNet is unsuitable without modification because its groupings can be abstract, unevenly granular, or overlapping.These issues can produce visually dissimilar classes, mismatched specificity, and ambiguous classification tasks.
  • 4.1 Utilizing the ImageNet class hierarchy: BREEDS manually recalibrates the hierarchy around visual characteristics, comparable specificity, and a single-parent tree structure.The process removes classes that do not fit naturally and places categories such as dog and cat at the same level.
  • 4.2 Creating BREEDS tasks: Tasks are generated by selecting superclasses at a chosen hierarchy distance and partitioning their ImageNet leaf classes into source and target domains.The source and target subclasses are disjoint, while the number of subpopulations per superclass is fixed to maintain balance.
  • 4.2 Creating BREEDS tasks: The methodology produces ImageNet benchmarks with adjustable granularity, including specialized living and non-living tasks, and can generate additional variants automatically.The benchmark suite includes ENTITY-13 and LIVING-17 examples, while Table 1 defines task granularity and dataset naming.
  • 4.3 Calibrating BREEDS benchmarks via human studies: Human studies test whether source and target domains preserve the same visual classification task by training annotators on source images and evaluating them on target images.Annotators classify target samples into two undisclosed superclass groups, with source-domain performance serving as a baseline.
  • 4.3 Calibrating BREEDS benchmarks via human studies: Annotators remain fairly robust to subpopulation shift, with especially small source-to-target drops for ENTITY-30 and LIVING-17, and outperform WordNet-based tasks.This supports the perceptual similarity of the source and target domains and the calibration of the modified hierarchy.

5 Evaluating Model Performance under Subpopulation Shift

BREEDS benchmarks expose substantial model brittleness to subpopulation shift and enable evaluation of standard training, target-domain adaptation, and robustness interventions. Standard models lose accuracy across domains, while interventions yield limited direct gains but often help after final-layer retraining.

  • 5.1 Standard training: Standard ERM models suffer accuracy drops of more than 30% in most subpopulation-shift cases.Source-domain accuracy is typically high, but target-domain performance falls considerably.
  • 5.1 Standard training: Higher source accuracy typically corresponds to preserving a larger fraction of accuracy on the target domain.The relationship exceeds a constant-drop baseline across models.
  • 5.1 Standard training: Models outperform annotators on unseen source-domain samples but perform worse than humans on target-domain BREEDS tasks.This contrast indicates greater model brittleness despite higher source accuracy.
  • 5.1 Standard training: Retraining only the final layer on target-domain data significantly improves target accuracy but does not match end-to-end target-domain training.A non-trivial gap remains relative to both original source accuracy and direct target-domain training.
  • 5.2 Robustness interventions: Relative accuracy, defined as target accuracy divided by source accuracy, compares interventions while accounting for source-domain performance changes.This metric supports an accuracy–robustness trade-off analysis and Pareto-frontier visualization.
  • 5.2 Robustness interventions: Adversarial training and erase noise produce small robustness improvements, sometimes at the cost of source accuracy, with gains of no more than a few percentage points.After final-layer target retraining, intervention effects become more pronounced; adversarially trained models can approach initial source accuracy on LIVING-17.
  • 5.2 Robustness interventions: Stylized training performs consistently worse, suggesting texture remains important for these subpopulation-shift tasks.The authors also note that intervention hyperparameters were not exhaustively tuned.

6 Additional Related Work

Related work studies robustness through distributional perturbations, data-source differences, domain adaptation, domain generalization, and zero-shot learning. BREEDS instead focuses on generalizing to unseen subpopulations while keeping the class set fixed.

  • Distributional robustness: Distributional robustness studies shifts bounded by f-divergence, but this notion can be too pessimistic for realistic data variations.Related work also connects distributional robustness to causality through injected spurious correlations.
  • Domain adaptation and transfer learning: Domain adaptation uses relatively few target-domain samples for the same task, whereas transfer learning may also change the task.Domain generalization trains across explicitly annotated domains to generalize to a different domain.
  • Zero-shot learning: Zero-shot learning recognizes previously unseen classes, unlike BREEDS, which studies unseen subpopulations for the same classes.The distinction is between class novelty and subpopulation novelty.

7 Conclusion

BREEDS constructs controlled, natural subpopulation-shift benchmarks from existing dataset class structure, applies them to ImageNet, and evaluates models alongside human baselines. The results indicate that current models remain far from fully handling these shifts, even with robustness interventions.

  • Method: BREEDS uses semantic superclasses and disjoint source-target subclasses to create controlled subpopulation shifts without synthetic input changes or new data collection.The approach repurposes existing ImageNet classes as subpopulations within superclass classification tasks.
  • Evaluation: Human baselines provide context for model performance and test whether the induced subpopulation shifts significantly affect human classification.The analysis uses ILSVRC2012 classes organized within the WordNet hierarchy as the starting dataset structure.
  • Results: Existing models remain far from fully tackling BREEDS subpopulation shifts, even when robustness interventions are applied.The benchmarks are presented as a framework for studying robustness to distribution shift in real-world machine-learning deployments.
  • Method: The methodology modifies WordNet through node collapse, node insertion, node deletion, and edge addition to produce superclasses calibrated for visual classification.These operations address differences in granularity, abstract groupings, and unsuitable hierarchy structure.
  • Benchmarks: The ImageNet benchmarks vary in granularity and use sampled disjoint classes for source and target domains while approximately balancing subclasses across superclasses.The benchmark tasks are identified as ENTITY-13, ENTITY-30, LIVING-17, and NON-LIVING-26.

A.5 Annotator task

The annotator study tests whether humans can classify target-domain images into unfamiliar superclass groupings using source-domain examples. Tasks use paired superclasses, fixed image counts, random permutations, multiple comparisons, and three annotators per task.

  • Annotator task: Annotators first viewed two source-domain image groups, then assigned target-domain images to one of the two groups.Control tasks used images from the source domain instead of the target domain.
  • Purpose: The study’s goal was to determine whether humans could classify images into superclasses without knowing the semantic grouping.A screenshot of the annotation task is provided as Figure 7.
  • Task design: Each task contained 20 source-domain images per superclass and 12 images to classify, with images resized and center-cropped to 224 × 224.The preprocessing matched the input size used for model predictions.
  • Task design: The two superclasses were randomly permuted at load time, and each superclass was compared against three randomly chosen other superclasses.Three annotators completed each task to improve the concentration of accuracy estimates.

A.6.1 Model architectures and training

The model analysis uses standard PyTorch architectures trained with fixed optimization settings, then evaluates accuracy on randomly paired superclasses to compare models with human annotators.

  • Training: Models use standard implementations from the PyTorch library and are trained until convergence.Training uses batch size 128, weight decay 10^-4, and architecture-specific learning rates listed in Table 7.
  • Training: ENTITY-13 and ENTITY-30 training runs for 300 epochs, with learning rates reduced tenfold every 100 epochs.The passage also describes training schedules for LIVING-17 and NON-LIVING-26, though the supplied text is truncated.
  • Evaluation: Model accuracy is evaluated on pairs of superclasses by selecting the superclass with the higher predicted probability for each image.Repeating the procedure across random superclass pairs estimates average-case binary accuracy.

A.6.3 Robustness interventions

Robustness interventions are evaluated with intervention-specific hyperparameters, but computational constraints restrict the architectures used for these experiments.

  • Architectures: Adversarial training uses ResNet-18 and ResNet-50, while other robustness interventions use ResNet-18 and ResNet-34.The restricted architecture set reflects computational constraints.
  • Implementation: Adversarial training is implemented with the robustness library, and random erasing uses PyTorch transforms.Additional intervention-specific hyperparameters are listed in Appendix Table 8.

B.1 Human Baselines for BREEDS Tasks

BREEDS tasks are evaluated with human baselines and model accuracies across source and target subpopulations. The benchmarks are constructed by partitioning superclass subpopulations, allowing robustness to vary with split structure.

  • Human baselines: Human annotators perform better on binary tasks built from the modified BREEDS hierarchy than on tasks obtained directly from WordNet.Annotator accuracy distributions are reported for superclass pairs with test data sampled from both source and target domains.
  • Model evaluation: Models perform fairly uniformly across superclasses on source-domain test data, indicating that the BREEDS tasks are comparably difficult across classes.This pattern is reported for AlexNet, ResNet-50, and DenseNet-121 visualizations.
  • Model evaluation: Robustness to subpopulation shift varies widely across superclasses, measured by the accuracy drop from source to target domains.The per-class source and target accuracies expose heterogeneous target-domain degradation across BREEDS superclasses.
  • Task construction: BREEDS tasks identify balanced superclasses and randomly partition their subpopulations between source and target domains.The desired number of subpopulations is sampled at random before the source-target split is made.
  • Task construction: Source-target splits can be made more or less adversarial by controlling how closely related the subpopulations in the two domains are.Less adversarial splits can use subpopulations sharing a common parent, whereas more adversarial splits separate them more strongly.
  • Split sensitivity: Model robustness is lowest for the most adversarial split, while retraining the linear layer on target-domain data recovers a considerable fraction of the accuracy drop.The recovery occurs across split types and indicates that learned features transfer to unknown subpopulations.

B.2.2 Robustness Interventions

The robustness-intervention evaluation reports raw model accuracies under standard and adversarial training. Models are compared on source accuracy, target accuracy, and target accuracy after target-domain linear-layer retraining.

  • Evaluation: The reported intervention results include raw accuracies for models trained with various train-time robustness interventions.These accuracies are presented in Tables 9 and 10.
  • Adversarial training: Adversarial-training comparisons use models trained on source-domain samples with either standard training (ε = 0.0) or adversarial training.The table evaluates source accuracy, target accuracy, and target accuracy after retraining the linear layer on target-domain data.
  • Evaluation: The broader intervention table evaluates source accuracy, target accuracy, and target accuracy after retraining the model’s linear layer with target-domain data.It reports 95% bootstrap confidence intervals and highlights the maximum task accuracy over ε while accounting for the interval.
Loading 2008.04859v1…