PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring Robustness to Natural Distribution Shifts in Image Classification

Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, Ludwig Schmidt

arXiv:2007.00644v2cs.LGcs.CVstat.ML

TL;DR

The paper asks whether robustness to synthetic image shifts transfers to natural shifts in real data. It evaluates ImageNet models across a large testbed using accuracy-controlled robustness measures, finding little to no transfer and limited benefits from current interventions, with larger and more diverse training data as the main exception. These results identify robustness to natural distribution shifts as an open research problem.

  • Problem

    Synthetic robustness evaluations leave limited evidence about whether robustness transfers to distribution shifts arising naturally in real image data.

  • Method

    The paper conducts a comprehensive ImageNet study evaluating 204 pre-trained models in 213 settings and compares synthetic robustness metrics with effective robustness on natural shifts.

  • Results

    Current robustness interventions offer little to no improvement on available natural distribution shifts, while larger and more diverse datasets provide only small gains in some cases.

  • Takeaways & Limitations

    Robust models should be evaluated on natural distribution shifts, and robustness metrics should control for baseline accuracy.

  • Takeaways & Limitations

    The dataset-growth experiments use only i.i.d. subsets smaller than ILSVRC, so they may miss robustness improvements that emerge at larger scales or with different data composition.

Abstract

from arXiv · show

We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets. Most research on robustness focuses on synthetic image perturbations (noise, simulated weather artifacts, adversarial examples, etc.), which leaves open how robustness on synthetic distribution shift relates to distribution shift arising in real data. Informed by an evaluation of 204 ImageNet models in 213 different test conditions, we find that there is often little to no transfer of robustness from current synthetic to natural distribution shift. Moreover, most current techniques provide no robustness to the natural distribution shifts in our testbed. The main exception is training on larger and more diverse datasets, which in multiple cases increases robustness, but is still far from closing the performance gaps. Our results indicate that distribution shifts arising in real data are currently an open research problem. We provide our testbed and data as a resource for future work at https://modestyachts.github.io/imagenet-testbed/ .

1 Introduction

The paper examines whether robustness to synthetic shifts transfers to natural shifts in real image data. Across a large ImageNet testbed, it finds that accuracy-controlled synthetic robustness often transfers weakly, while diverse-data training is the main intervention that improves robustness across multiple natural shifts.

  • Motivation: Synthetic robustness research largely evaluates precisely defined pixel-level modifications, whereas real-world shifts include subtle changes in scenes, objects, and lighting.Natural shifts are not artificially created and therefore are less precisely specified.
  • Study design: 204 pre-trained ImageNet models were evaluated in 213 settings, creating a testbed substantially larger than prior work.The testbed contains 109 model predictions and is described as 100 times larger than prior work.
  • Findings: Many apparent robustness gains can be explained by higher standard accuracy, so robustness measurements should control for baseline accuracy.Different architectures often do not improve robustness beyond what their higher standard accuracy predicts.
  • Findings: Current synthetic robustness measures are at most weakly predictive of robustness on the natural distribution shifts available in the testbed.Synthetic shifts remain useful for purposes such as security, but alone do not comprehensively measure robustness.
  • Findings: Current robustness interventions are often ineffective on natural distribution shifts, although the result is not universal.Figure 1 illustrates that even models with the highest effective robustness remain far from closing the shift-induced accuracy gap.
  • Findings: Training with more diverse data is the only intervention found to improve robustness across multiple natural distribution shifts.Some models trained on 100 times more data than standard ImageNet nevertheless provide no robustness.
  • Implications: Robustness on real data remains a clear challenge, likely requiring new algorithmic ideas and more evaluation on natural shifts as larger datasets yield diminishing returns.The paper is a meta-study rather than a new classification method or image dataset.

2 Measuring robustness

The paper separates robustness from standard accuracy by comparing shifted performance with an accuracy-based baseline. It defines effective robustness as performance beyond that baseline and complements it with relative robustness measuring intervention effects directly.

  • Setup: The method evaluates each model in a standard setting and a distribution-shifted setting, denoting their accuracies acc1(f) and acc2(f).This two-setting design provides the inputs for distinguishing accuracy from robustness.
  • Setup: Ranking models by shifted accuracy alone confounds robustness with standard accuracy: fa drops 5% from 0.8 to 0.75, while fb drops 14% from 0.9 to 0.76.Although fb has higher shifted accuracy, fa is treated as more robust because its relative drop is smaller.
  • Effective robustness: Effective robustness measures shifted accuracy beyond what higher standard accuracy predicts.It corresponds graphically to performance above the linear trend fitted from standard models.
  • Effective robustness: The baseline β(x) gives expected shifted accuracy for standard accuracy x, fitted with a log-linear model on models without robustness interventions.Other shifts may require different baseline trends or additional properties such as architecture.
  • Effective robustness: Effective robustness is defined as ρ(f) = acc2(f) − β(acc1(f)); a model on the baseline has ρ(f) = 0.Positive values indicate shifted performance above the current standard-model trend.
  • Relative robustness: Relative robustness complements effective robustness by quantifying the intervention’s direct effect on shifted accuracy.This avoids treating increased effective robustness as useful when both standard and shifted accuracies decline.
  • Interpretation: Only a few classification models achieve both positive effective and relative robustness, and none achieve both at large levels.The two measures capture distinct aspects of useful robustness.

3 Experimental setup

The study evaluates ImageNet models across natural and synthetic distribution shifts, using standardized test-set comparisons to assess robustness. Its testbed spans multiple shift types and model categories, including robustness-focused and larger-data training approaches.

  • Experimental setup: The evaluation measures models on a standard test set S1 and a shifted test set S2.Human-reviewed labels help ensure that accuracies across the two test sets are comparable.
  • Types of distribution shifts: Natural distribution shifts use unmodified images, whereas synthetic shifts modify existing images to test robustness.
  • Natural distribution shifts: Seven natural distribution shifts are organized into consistency, dataset, and adversarially filtered categories.
  • Natural distribution shifts: Consistency shifts select the worst frame from each perceptually similar video-frame set using the pm-k metric.Anchor frames define S1, while the worst selected frame determines shifted accuracy.
  • Natural distribution shifts: Dataset shifts compare compatible classification tasks collected differently, including ImageNetV2, ObjectNet, ImageNetVid-Robust-anchor, and YTBB-Robust-anchor.
  • Synthetic distribution shifts: Synthetic evaluation includes image corruptions, style transfer, and untargeted adversarial examples.The corruption suite includes 38 corruptions, while adversarial perturbations use projected gradient descent under ℓ∞ or ℓ2 bounds.
  • Model testbed: The 204-model testbed covers standard models, robustness-intervention models, and models trained on substantially more data.It includes 78 standard models, 86 robust models, and 30 models trained with more data.

4 Main results

The study evaluates effective and relative robustness across natural distribution shifts and tests whether synthetic robustness predicts natural robustness. Standard accuracy often predicts shifted accuracy, but synthetic robustness generally transfers weakly; larger, more diverse training data is the main exception, with limited gains.

  • 4 Main results: The study measures effective and relative robustness across three types of natural distribution shift before comparing synthetic and natural robustness.The testbed covers dataset shifts, consistency shifts, and adversarially filtered shifts.
  • 4.1 Results on natural distribution shifts: Standard test accuracy reliably predicts shifted accuracy for ImageNetV2, ObjectNet, and ImageNet-Vid-Robust, with r2 values of 1.00, 0.95, and 0.95.The relationship is noisier for YTBB-Robust, where r2 = 0.83.
  • 4.1 Results on natural distribution shifts: Models trained on substantially more data are the main positive effective-robustness outliers, although not every larger-data model improves robustness.Examples include models trained on 11,000 classes, 1 billion Instagram images, or JFT-300M; a ResNet101 trained on JFT-300M has ρ = −0.23%.
  • 4.1 Results on natural distribution shifts: ℓp-adversarially robust models show average effective robustness of 6.7% on ImageNet-Vid-Robust and 4.9% on YTBB-Robust.These gains do not necessarily produce relative robustness: average τ is −8.5% and −0.7% for ResNet50 models on the two datasets.
  • 4.1 Results on natural distribution shifts: On ImageNet-A, models below ResNet-50 accuracy perform near chance, while higher-accuracy models rapidly improve; each ImageNet percentage point yields at least an 8% ImageNet-A improvement.The adversarial filtering creates a knee-shaped relationship rather than uniformly increasing difficulty.
  • 4.2 Results on synthetic distribution shifts: Synthetic robustness is at most weakly predictive of natural robustness, with ImageNetV2 correlations of r = 0.24 for corruptions and r = −0.05 for ℓp-attacks.Averaging many synthetic corruptions does not provide a comprehensive measure that predicts natural-shift robustness.
  • 4.3 Takeaways and discussion: Current robustness interventions offer little to no improvement on the natural shifts studied, while larger-data training yields only small dataset-shift gains.For EfficientNet-L2 NoisyStudent, the best reported ImageNetV2 accuracy-drop improvement is from 8.6% to 7.5%.

5 How does the amount of training data impact robustness?

Larger and more diverse training datasets can improve robustness to natural distribution shifts, but the effect is not uniform and simple i.i.d. dataset growth does not reliably improve effective robustness.

  • More data and robustness: Models trained on more data can achieve positive effective robustness on dataset shifts, but the effect is not uniform.ResNet101 trained on JFT-300M has negligible effective robustness (ρ = −0.23%) despite using 300× more data than standard ImageNet models.
  • Varying the number of images: Larger training subsets increase accuracy but do not improve effective robustness on ImageNetV2.The experiment subsampled the ILSVRC-2012 training set by factors of 2, 4, 8, 16, and 32.
  • Varying the number of classes: Changing the number of training classes affects accuracy but does not impact effective robustness.Models trained on subsets with 500, 250, and 125 classes were evaluated on the 125-class subset.
  • Interpretation: Neither growing images nor classes i.i.d. is effective, although larger datasets can provide meaningful robustness improvements in other experiments.The discrepancy may reflect limitations of subsampling, dataset sizes below an accuracy-saturation threshold, or differences in label diversity or quality.

6 Related work

The paper unifies research on synthetic and natural distribution shifts through a substantially larger ImageNet testbed, while situating its findings alongside related robustness, transfer, domain generalization, and fairness work.

  • Natural distribution shifts: Earlier work consistently found a linear accuracy drop from ImageNet to other natural test sets, a pattern this study confirms.This passage places the paper within prior research on natural distribution shifts.
  • Synthetic and natural shifts: The work unifies previously independent research on synthetic and natural distribution shifts.It incorporates many prior synthetic robustness measures and compares them with natural distribution shifts.
  • Core findings: A testbed 100× larger than prior work reveals that synthetic and natural robustness often behave differently.The study also argues that robustness evaluations should control for accuracy and reports that most interventions lack effective robustness.
  • Concurrent work: Extra training data provides the most robustness on ImageNet-R and ImageNet-Sketch, reaching ρ = 29.1% on ImageNet-R, though the effect is not uniform.DeepAugment combined with AugMix is discussed as another effective intervention after models trained on more data.
  • Synthetic robustness connections: Adversarial robustness can help on ImageNet-R in some settings, including effective robustness ρ = 22.7% and relative robustness τ = 5.7% for a feature-denoising PGD-style model.An AdvProp model reaches 57.8% absolute accuracy on ImageNet-R without extra data.
  • Related paradigms: The testbed evaluates robustness without target-domain fine-tuning, complementing domain adaptation and relating closely to domain generalization.It does not currently include algorithms explicitly following the domain-generalization paradigm.
  • Scope and neighboring work: The study also relates to distributionally robust optimization, adversarial filtering, and fairness research, but remains focused on ImageNet image classification.Other work addresses worst-case nearby distributions, adversarially filtered tests, or demographic-group shifts.

7 Conclusion

The paper evaluates whether robustness methods transfer from synthetic perturbations to natural image shifts and finds substantial remaining accuracy gaps. Most interventions yield little consistent benefit on real data, while larger datasets help only modestly and inconsistently.

  • Conclusion: The paper complements synthetic robustness tests with comprehensive experiments on distribution shifts arising from real data.Its goal is to assess how current image classification models behave under natural variation.
  • Conclusion: Current image classification models still suffer substantial accuracy drops on natural distribution shifts.The conclusion presents this as a central unresolved robustness problem.
  • Conclusion: Robustness interventions effective against synthetic perturbations yield little to no consistent improvement on real data.Training on larger datasets is the only approach with broad benefits, but its gains are small and inconsistent.
  • Future research: Even training on 1,000 times more data remains far from closing the accuracy gaps.The paper identifies robustness metrics that control for baseline accuracy and recommends evaluating robust models on natural shifts.

Broader Impact

The paper frames robustness as important for understanding real-world deployment, while recognizing that its testbed covers limited natural shifts and omits demographic dataset bias. It also cautions that robust systems can enable harmful uses.

  • Potential benefits: Robustness evaluation may help researchers better understand deployed-system behavior before deployment, potentially reducing negative consequences for users.The paper connects this motivation to real-world applications such as medical diagnosis.
  • Potential harms and interpretation: The study does not intend its findings to discourage synthetic robustness research, including adversarial examples that address genuine security problems.Synthetic corruption types remain valuable independently of their correlation with currently available natural shifts.
  • Scope limitation: The testbed captures only a small number of possible natural distribution shifts, so solving the evaluated shifts would not resolve all robustness concerns.The selected shifts were chosen because they are established in the literature and concrete examples of the target phenomenon.
  • Scope limitation: The study does not evaluate dataset bias affecting demographic-group representation.The paper notes that neglecting implicit demographic biases can harm underrepresented groups and that application-domain analysis remains necessary.
  • Potential harms: More reliable machine learning can also enable harmful uses such as widespread surveillance or autonomous weapon systems.The paper calls for regulation and awareness of unintended consequences from technological advances.
  • Testbed overview: Figure 7 organizes models by rows and evaluation settings by columns, with corruption severities and storage formats represented across cells.Some cells are empty because of resource constraints.
  • Relative and effective robustness: Figure 8 illustrates relative robustness for an intervention applied to a baseline model, including the ideal quadrant where both effective and relative robustness are positive.The figure provides a graphical view of how an intervention can improve effective robustness while reducing shifted-set accuracy.

B.2 Relative and effective robustness for ResNet50 models

Across ResNet50 variants, training on more data is the only intervention consistently associated with positive relative and effective robustness on natural dataset shifts, though the effect is small. Adversarial robustness can yield effective robustness on consistency shifts, while robustness interventions have little impact on ImageNet-A.

  • On YTBB-Robust, some ImageNet-C augmentations provide significant relative and effective robustness, but the gains do not consistently transfer across natural shifts.Reported examples include greyscale (ρ = 6.9%, τ = 1.8%) and jpeg compression (ρ = 5.4%, τ = 6.3%).
  • Adversarially robust models provide effective robustness averaging ρ = 4.3% on ImageNet-Vid-Robust and ρ = 3.9% on YTBB-Robust.They provide relative robustness only sometimes on YTBB-Robust.
  • Robustness interventions have little impact on ImageNet-A accuracy, whose knee-like response curve is largely an artifact of adversarial filtering.The knee occurs at the ResNet50 model accuracy point.
  • More-data training is the only model class with consistently positive relative and effective robustness on natural dataset shifts.The effect is small, and not every model trained on more data is more robust.

C Effective robustness scatterplots

Synthetic robustness measures generally show weak or absent relationships with effective robustness on natural shifts. The main exceptions are limited correlations for some consistency-shift and corruption comparisons.

  • Synthetic robustness measures are generally weakly correlated with effective robustness on natural dataset shifts.Figure 12 reports Pearson coefficients of r = 0.24, −0.05, −0.01, −0.26, 0.61, 0.30, 0.52, and 0.36 in reading order.
  • The strongest dataset-shift correlations are between ImageNet-Vid-Robust and image corruptions (r = 0.61) and between YTBB-Robust and image corruptions (r = 0.52).These correlations are described as significant but not very strong.
  • Consistency shifts are largely uncorrelated with synthetic robustness, except that adversarial-attack accuracy correlates with effective robustness for ℓp-adversarially trained models.Effective robustness on consistency shifts does not always imply relative robustness.
  • ImageNet-A shows no observed correlation between synthetic and natural robustness after effective robustness is computed piecewise around the ResNet50 accuracy.This piecewise definition accounts for the adversarially filtered shift’s response curve.

D Corruption robustness

Robustness to one synthetic corruption generally transfers poorly to other corruptions, and corruption robustness does not promote effective robustness to real distribution shifts. PGD training can also reduce robustness to low-frequency corruptions.

  • Corruption robustness does not promote effective robustness to real distribution shift.The section studies this relationship using ResNet50 models trained on selected ImageNet-C corruptions.
  • Training against one synthetic corruption or corruption set generally does not transfer robustness to other corruptions.Models are overall robust mainly to the corruption on which they were trained, although some transfer cases occur.
  • PGD models show reduced robustness to low-frequency corruptions such as contrast.The same phenomenon was observed in prior work cited by the paper.
  • The corruption evaluation includes 38 corruption types, averaged over five severities.Corruptions are implemented either in memory with custom GPU code or on disk using ImageNet-C.

H Synthetic robustness correlation with natural robustness

The paper compares effective robustness across synthetic and natural shifts using Pearson correlations after fitting baselines on standard models. No synthetic shift is highly correlated with every natural shift, and observed correlations are not causal evidence of transfer.

  • No synthetic shift has high correlation with all natural distribution shifts.ℓp-robustness correlates most with consistency shifts, whereas brightness, gaussian blur, defocus blur, and saturate correlate more with dataset shifts.
  • Correlations between synthetic and natural robustness measures are not causal: training on correlated synthetic shifts does not significantly improve dataset-shift effective robustness.This is stated specifically for brightness, gaussian blur, defocus blur, and saturate.
  • Figures 16–22 visualize the two highest-correlated synthetic shifts for each natural shift.Table 2 provides the full list of correlation values.
  • Synthetic and natural effective robustness correlations are computed after filtering out standard models and using linear fits on those standard models.The analysis computes Pearson correlations between effective robustness values across shifts.

I.2 Ablations on our main figures

The ablations clarify how the main-figure analyses behave under alternative plotting choices and extend the comparison to additional natural shifts, including renditions and sketches. They also show that robustness to synthetic shifts often transfers weakly, while extra training data can help inconsistently.

  • Figure construction: Logit scaling emphasizes that gains at higher accuracies represent greater progress, with 95% confidence intervals computed from 1,000 bootstrapped linear fits.The fits resample models with replacement.
  • Natural dataset shifts: ImageNet-R and ImageNet-Sketch follow the broader natural-shift trends, with models trained on extra data providing the most robustness, though not uniformly.ImageNet-R samples renditions, whereas ImageNet-Sketch samples sketches.
  • Cross-shift comparisons: Synthetic robustness measures are weakly correlated with ImageNet-R effective robustness, with Pearson correlations r = 0.35 and 0.30.The comparison indicates that stronger corruption or adversarial robustness generally does not improve effective robustness on ImageNet-R.
  • Natural dataset shifts: DeepAugment+AugMix reaches effective robustness of 11.2% for ResNeXt101 and 10.2% for ResNet50 on ImageNet-R.These models outperform most other models in the ImageNet-R comparison.
  • Cross-shift comparisons: DeepAugment provides effective robustness on ImageNet-Vid-Robust and YTBB-Robust comparable to multiple synthetic-perturbation training, but not on ImageNetV2 or ObjectNet.The synthetic combination includes Gaussian noise, contrast, motion blur, and JPEG compression.

K.2 Video robustness

Prior work linked video robustness strongly to color-corruption accuracy, but this analysis identifies standard accuracy as a confounder and finds that correlation alone does not establish causation. A saturation-trained model illustrates the distinction between corruption robustness and video robustness.

  • Prior measurements: Earlier video-robustness measurements reported strong correlations with brightness, hue, and saturation robustness, with r near 0.95.Those measurements evaluated neighboring video frames conditional on correct anchor-frame classification.
  • Confounding factors: Standard accuracy remains a confounder because models with higher standard accuracy are more likely to classify neighboring frames correctly.Conditioning on correct anchor-frame classification only partly accounts for this effect.
  • Correlation versus causation: The effective-robustness measure explicitly corrects for the confounding effect of standard accuracy.The authors use this correction to distinguish video robustness from raw accuracy relationships.
  • Data preparation: The testbed datasets differ from earlier YT-BB evaluations because ImageNet-Vid and YouTube-BB sequences were rigorously cleaned and human-annotated.Cleaning checked subject visibility, class correctness, and blur.
Loading 2007.00644v2…