Measuring Robustness to Natural Distribution Shifts in Image Classification

Study scope

• 204 pre-trained ImageNet models evaluated in 213 settings provide a testbed 100 times larger than prior work for studying robustness interventions.The study covers popular models and distribution shifts using 109 model predictions.Source: §1 · p. 2

Main findings

• Synthetic robustness measures are at most weakly predictive of robustness to currently available natural distribution shifts, and interventions are often ineffective on them.Synthetic shifts remain useful for precisely defined experiments and security-related questions, but they do not comprehensively measure robustness.Source: §1 · pp. 1–2
• Training on more diverse data is the only intervention that improves robustness across multiple natural distribution shifts, although some models trained on 100 times more data show no robustness.The paper identifies this overarching trend through its large testbed and leaves open when and why additional data helps.Source: §1 · p. 2

Motivation and contribution

• The paper is a meta-study rather than a new method or dataset, highlighting robustness on real data as an open challenge likely requiring new algorithms and more natural-shift evaluations.The authors note diminishing returns from larger training datasets.Source: §1 · pp. 1–2

Evaluation principles

• Robustness measurements should control for standard accuracy because many apparent gains reflect better unperturbed-test performance rather than robustness beyond that accuracy.Different architectures, for example, do not substantially improve robustness beyond what higher standard accuracy predicts.Source: §1 · p. 2, Figure 1 · p. 3

Measuring robustness

• Robustness is evaluated using standard accuracy acc1(f) and shifted-test accuracy acc2(f), treated as distinct quantities.The experiments always use a standard test set and a test set with distribution shift.Source: §2 · p. 3
• Shifted-test accuracy alone can favor a higher-accuracy model that suffers a larger relative drop, so robustness comparisons must disentangle baseline accuracy from degradation.For example, fa drops 5% from 0.8 to 0.75, whereas fb drops 14% from 0.9 to 0.76; fa is therefore more robust despite lower shifted accuracy.Source: §2 · p. 3

Effective robustness

• Effective robustness measures shifted accuracy beyond the baseline expected from standard accuracy, with models above the fitted trend considered more robust.The baseline β(x) is fit using a log-linear trend for models without a robustness intervention; other shifts may require different trends or additional factors such as architecture.Source: §2 · p. 4

Relative robustness

• Relative robustness directly measures an intervention’s change in shifted-test accuracy as τ(f′) = acc2(f′) − acc2(f).This complements effective robustness because an intervention can increase effective robustness while decreasing both standard and shifted-test accuracy.Source: §2 · p. 4

Overall assessment

• Few classification models achieve both positive effective and relative robustness, and none achieve both large effective and relative robustness.A useful intervention must improve performance beyond the standard-accuracy baseline while also increasing shifted-test accuracy.Source: §2 · p. 4

3 Experimental setup

• Models are evaluated on a standard test set S1 and a shifted test set S2, with human-reviewed natural shifts intended to make their accuracies comparable.An ideal robust classifier should not have a substantial accuracy gap between the two sets; humans achieve similar accuracy on ImageNet and ImageNetV2 [39] [76].Source: §3 · p. 4

3.1 Types of distribution shifts

• Natural shifts use unmodified images, whereas synthetic shifts modify existing images specifically to test robustness.The evaluation distinguishes these two shift types and provides further details and visual overviews in the appendices.Source: §3.1 · p. 5

3.1.1 Natural distribution shifts

• The testbed includes seven natural shifts covering consistency, dataset, and adversarially filtered variation.Consistency shifts use worst-frame accuracy (the pm-k metric) [35] [76]; dataset shifts include ImageNetV2, ObjectNet, ImageNetVid-Robust-anchor, and YTBB-Robust-anchor [76]; ImageNet-A [39] represents adversarial filtering.Source: §3.1.1 · p. 5

3.1.2 Synthetic distribution shifts

• Synthetic evaluation covers 38 image corruptions, stylized ImageNet, and untargeted projected-gradient adversarial examples bounded in ℓ∞ or ℓ2 norms [38, 33, 34, 44, 55].Corruptions include noise, blur, simulated weather, and JPEG compression; adversarial examples use specified ε values for both norms.Source: §3.1.2 · p. 6

3.2 Classification models

• The model testbed contains 204 ImageNet models: 78 standard models, 86 robustness-focused models, and 30 models trained on substantially more data.The larger-data models use sources including Instagram, YFCC, JFT, OpenImages, and a 21,841-class ImageNet subset [56] [104] [82] [102] [98].Source: §3.2 · p. 6

4.1 Results on natural distribution shifts

• On natural shifts, standard accuracy predicts dataset-shift accuracy, adversarial training helps consistency effective robustness, and substantially more data yields only small gains.For ImageNetV2, EfficientNet-L2 NoisyStudent improves the accuracy drop from 8.6% to 7.5%; adversarially trained models average ρ = 6.7% on ImageNet-Vid-Robust and ρ = 4.9% on YTBB-Robust, but relative robustness remains negative.Source: §4.1 · pp. 6, 8, Figure 2 · p. 7, §4.3 · p. 11

4.3 Takeaways and discussion

• More training data improves effective robustness, but the required scale is orders of magnitude larger than standard ImageNet training and does not guarantee positive robustness.High-accuracy outliers include models trained on 11,000 classes, 1 billion Instagram images, or JFT-300M, while a JFT-300M ResNet101 has ρ = −0.23%.Source: §4.1 · p. 8, §4.3 · p. 11, Figure 6 · p. 12

4.2 Results on synthetic distribution shifts

• Synthetic robustness is weakly predictive of ImageNetV2 effective robustness, with Pearson r = 0.24 for corruptions and r = −0.05 for ℓp-attacks.Averaging many synthetic corruptions does not provide a comprehensive measure of robustness to natural distribution shifts, although individual measures vary in predictiveness.Source: §4.2 · pp. 10–11, Figure 5 · p. 10

4.3 Takeaways and discussion

• Current robustness interventions offer little to no improvement on natural shifts, while synthetic robustness metrics do not reliably predict natural robustness.Effective robustness under non-adversarial corruptions or ℓp-attacks does not imply effective robustness to natural distribution shifts.Source: §4.3 · p. 11

Motivation

• More training data does not guarantee robustness: ResNet101 trained on JFT-300M has ρ = −0.23% despite using 300× more data.The passage suggests label diversity or quality may help explain why data scale alone has inconsistent effects.Source: §5 · p. 12

Varying the number of images

• Larger training subsets increase accuracy but do not improve effective robustness on ImageNetV2.The experiment subsampled ILSVRC-2012 by factors of 2, 4, 8, 16, and 32.Source: §5 · p. 12

Varying the number of classes

• Varying the training set from 125 to 500 classes changes accuracy but does not affect effective robustness on the 125-class evaluation subset.Models were trained on subsets containing 500, 250, or 125 classes.Source: §5 · p. 12

Overall findings

• Neither increasing images nor classes via i.i.d. subsampling improves effective robustness, despite larger datasets sometimes providing meaningful robustness gains.The discrepancy may arise because subsampling ILSVRC lacks diversity, uses datasets below the accuracy-saturation threshold, or does not capture how sufficiently large datasets affect robustness.Source: §5 · p. 12

Core contribution

• The study unifies synthetic and natural distribution-shift research, finding that synthetic robustness often differs from natural robustness and that most interventions provide little effective robustness after controlling for accuracy.Training with more data is the main exception, improving robustness across natural shifts; ℓp-adversarial robustness helps in some consistency shifts.Source: §6 · p. 13
• The testbed is 100× larger than prior work and confirms earlier patterns, including linear performance drops from ImageNet to individual natural-shift datasets.It incorporates many prior synthetic robustness measures and is consistent with earlier synthetic-shift results, including the reported adversarial-robustness/Gaussian-noise relationship [31].Source: §6 · p. 13

Concurrent and subsequent work

• On ImageNet-R, extra-data models reach effective robustness up to ρ = 29.1%, while DeepAugment with AugMix achieves ρ = 11.2%.Adversarial robustness also helps: an AdvProp model [100] reaches 57.8% accuracy and ρ = 7.5%, while feature denoising with PGD-style robust optimization [55] [101] reaches ρ = 22.7% and τ = 5.7%.Source: §6 · pp. 13–14

Domain adaptation / transfer learning

• Domain adaptation uses target-domain data, whereas this work studies generalization without target fine-tuning; domain generalization instead trains across multiple source domains before testing on a new domain.The testbed does not yet evaluate explicit domain-generalization algorithms, and prior meta-study results paralleling this work found standard ERM matched or outperformed eight such algorithms [36].Source: §6 · p. 14

Adversarial filtering

• Adversarial filtering can produce test sets that are difficult mainly for a specific existing model class, as later architectures rapidly closed ImageNet-A and Swag gaps.The paper also notes that DRO models are absent from the current ImageNet testbed because suitable pretrained models were unavailable, while fairness work addresses related brittleness across demographic shifts [59].Source: §6 · pp. 14–15

Further domains

• Related work beyond image classification reports similar limited transfer between synthetic and natural robustness, including one-word substitutions in natural language processing.The paper focuses on image classification, while noting broader robustness research across other domains [52] [58] [106].Source: §6 · p. 15

• Current image-classification models suffer substantial accuracy drops under natural distribution shifts, and synthetic-robustness interventions yield little to no consistent improvement on real data.The paper complements synthetic robustness tests with comprehensive experiments on distribution shifts arising from real data.Source: §7 · pp. 15–16
• Training on larger datasets is the only approach providing broad benefits on natural shifts, but its gains remain small and inconsistent.Source: §7 · p. 16
• Even training on 1,000 times more data remains far from closing accuracy gaps, indicating that robustness on real data requires new algorithmic ideas and better understanding of training data.Source: §7 · p. 16
• Future evaluations should control robustness metrics for baseline accuracy and assess models on natural distribution shifts alongside synthetic perturbations.The proposed testbed combines nuanced robustness metrics with multiple types of distribution shift to track progress toward reliable machine learning on real data.Source: §7 · pp. 15–16

Broader Impact

• Robustness remains a substantial barrier to deploying machine learning safely in real-world settings, including medical diagnosis.Laboratory accuracy may not predict performance in chaotic environments, and current chest X-ray classifiers can be brittle without recognized confounders.Source: Broader Impact · p. 16
• The testbed aims to improve evaluation of robustness to natural distribution shifts and help researchers anticipate deployed-system behavior before deployment.The authors hope this supports more accurate assessment of whether proposed techniques improve robustness to particular shifts.Source: Broader Impact · p. 16
• The study should not be interpreted as a reason to abandon synthetic robustness research, including adversarial examples and common corruptions.The authors note that these topics remain independently valuable, with adversarial examples posing genuine security risks.Source: Broader Impact · p. 16
• The testbed evaluates only a small, literature-grounded subset of natural distribution shifts, so solving its selected shifts would not eliminate other robustness concerns.The selected shifts are concrete examples, but many possible shifts remain uncaptured.Source: Broader Impact · p. 16
• Fixed-dataset evaluation does not fully address demographic dataset bias, fairness, or safety, making application-domain validation essential.The authors highlight geographical representation bias and its potential harm to underrepresented demographic groups [75] [51].Source: Broader Impact · pp. 16–17
• More reliable machine learning may also enable harmful uses such as widespread surveillance or autonomous weapon systems, requiring regulation and awareness of unintended consequences.These risks are presented as broader consequences of technological advances.Source: Broader Impact · p. 17

A Testbed overview

• The testbed organizes models across evaluation settings, including five corruption severities and in-memory versus on-disk variants, with some cells omitted for resource constraints.The accompanying code and data are publicly provided.Source: Figure 7 · p. 27

B.1 Relative and effective robustness graphical sketch

• Relative and effective robustness are both needed to separate robustness changes from original model accuracy: an intervention can improve effective robustness while reducing shifted accuracy, whereas the ideal improves both.Figure 8 illustrates the ideal intervention as occupying the quadrant with positive effective and relative robustness.Source: §B.1 · p. 28, Figure 8 · p. 28

Natural dataset shifts

• On YTBB-Robust, grayscale, pixelate, JPEG compression, combined augmentations, and Gaussian-noise training achieve positive relative and effective robustness, but these gains do not generalize consistently across natural shifts.Reported pairs are grayscale (ρ = 6.9%, τ = 1.8%), pixelate (ρ = 5.4%, τ = 2.0%), JPEG compression (ρ = 5.4%, τ = 6.3%), combined augmentations (ρ = 4.8%, τ = 5.0%), and Gaussian noise (ρ = 3.6%, τ = 4.0%).Source: §B.2 · p. 28
• On YTBB-Robust, some ℓp-adversarially robust models show effective robustness, including ℓ2 and ℓinf robust ResNet50 models at ρ = 6.4%, but most fail to provide positive relative robustness.ResNet50 smoothed with 0.25 Gaussian noise and adversarially 1-step PGD training reaches ρ = 5.0%.Source: §B.2 · p. 28

Natural consistency shifts

• Adversarially robust models provide effective robustness on consistency shifts, averaging ρ = 4.3% on ImageNet-Vid-Robust and ρ = 3.9% on YTBB-Robust, but only sometimes provide relative robustness on YTBB-Robust.The distinction between effective and relative robustness is shown across the two consistency shifts.Source: §B.2 · pp. 28–29, Figure 10 · p. 30

Adversarially filtered shift

• Robustness interventions have little impact on ImageNet-A accuracy, and its knee-like response curve is largely an artifact of adversarial filtering around the ResNet50 accuracy.Effective robustness is defined piecewise around the ResNet50 model-accuracy point because ImageNet-A contains images incorrectly classified by that model.Source: §B.2 · p. 29, Figure 11 · p. 30

Natural dataset shifts

• Training on more data is the only intervention with consistently positive relative and effective robustness on natural dataset shifts, but the effect is small and inconsistent.This comparison covers ImageNetV2, ObjectNet, ImageNet-Vid-Robust-anchor, and YTBB-Robust-anchor.Source: §B.2 · p. 28, Figure 9 · p. 29

Natural dataset shifts

• Synthetic-shift robustness measures are largely uncorrelated with natural dataset shifts, with correlations ranging from r = −0.26 to r = 0.61.The strongest correlations are between ImageNet-Vid-Robust and image corruptions (r = 0.61) and between YTBB-Robust and image corruptions (r = 0.52).Source: C Effective robustness scatterplots · p. 31, Figure 12 · p. 32

Natural consistency shifts

• Effective robustness on natural consistency shifts is largely uncorrelated with synthetic robustness, except for adversarial accuracy in ℓp-adversarially trained models.This correlation does not always imply relative robustness.Source: C Effective robustness scatterplots · p. 31, Figure 13 · p. 33

Adversarially filtered shift

• After piecewise computation around ResNet50 accuracy, synthetic and natural robustness show no observed correlation on ImageNet-A.ImageNet-A is the adversarially filtered shift.Source: C Effective robustness scatterplots · p. 31, Figure 14 · p. 33

D Corruption robustness

• The corruption analysis evaluates ResNet50s trained on selected ImageNet-C corruptions alongside models trained with more data or standard training.Figure 15 presents this comparison using sampled cells from the main grid in Figure 7.Source: Figure 15 · p. 34, D Corruption robustness · p. 34
• Robustness trained against one synthetic corruption or corruption set generally transfers poorly to other corruptions, although some transfer occurs.Overall, models are mainly robust to the corruption on which they were trained.Source: D Corruption robustness · p. 34
• PGD models show reduced robustness to low-frequency corruptions such as contrast.This phenomenon was also observed in [107].Source: D Corruption robustness · p. 34

E.1 Natural distribution shifts

• The testbed evaluates natural shifts using multiple ImageNetV2 variants, ObjectNet, ImageNet-Vid-Robust, YTBB-Robust, and ImageNet-A with dataset-specific prediction procedures.ImageNetV2 results use imagenetv2-matched-frequency-format-val unless otherwise stated; other datasets apply the class and frame-selection rules described in the protocol.Source: §E.1 · p. 35

E.2 Corruptions

• The corruption suite contains 38 corruption types, averages results over five severities, and distinguishes custom in-memory implementations from on-disk ImageNet-C corruptions.The protocol includes common noise, blur, weather, color, transformation, and compression corruptions; jpeg compression can significantly affect model accuracies [31].Source: §E.2 · p. 35

E.3 Adversarial attacks

• Models are evaluated with four 100-step PGD attacks spanning ℓ∞ norms of 0.5/255 and 2/255 and ℓ2 norms of 0.1 and 0.5.Most models use a class-balanced 10% dataset subset because of computational constraints, producing larger error bars.Source: §E.3 · pp. 35–36

F Models in the testbed

• The testbed catalogs evaluated ImageNet models with references, source-code links, and model-type labels distinguishing standard training, more-data training, and robustness interventions.The catalog includes 61 listed models across these categories.Source: F Models in the testbed · p. 37
• Robustness-intervention models cover adversarial training, smoothing, augmentation, anti-aliased networks, and texture-shape training.Examples include facebook_adv_trained_resnet152_denoise [101], randomized smoothing [13], AugMix [41], CutMix [108], and models trained on SIN [34].Source: F Models in the testbed · pp. 39, 41, 42
• The testbed includes standard-training models such as fbresnet152 [37], inceptionv3 [86], nasnetalarge [117], polynet [116], and mobilenet_v2 [73].Source: F Models in the testbed · pp. 39–40
• The testbed also includes models trained with more data, including google_resnet101_jft-300M [82], Instagram ResNeXt variants [56], and semi-supervised ResNet models [104].These entries are explicitly labeled as trained with more data models.Source: F Models in the testbed · pp. 39–40
• The testbed includes standard-training, robustness-intervention, and more-data models spanning diverse architectures, with entries numbered through 204.Examples include ResNeXt, VGG, SqueezeNet, ShuffleNet, SE-ResNet, and Xception models, alongside models trained with more data or robustness interventions.Source: F Models in the testbed · pp. 43–44

G Model accuracies

• Table 1 reports Top-1 ImageNet validation accuracy, effective robustness relative to ImageNetV2, mean corruption accuracy, and mean PGD-attack accuracy.Because PGD averages many attacks, its column is not a worst-case measure; specific attacks are deferred to E.3.Source: Table 1 · p. 45

H Synthetic robustness correlation with natural robustness

• The correlation analysis computes Pearson coefficients between each synthetic shift’s effective robustness and effective robustness under each natural shift.This procedure is applied separately for every synthetic shift in the testbed.Source: H Synthetic robustness correlation… · p. 50
• No synthetic robustness measure has high correlation with all natural distribution shifts, although some measures predict particular shifts better than others.Table 2 reports Pearson correlations after computing effective robustness and filtering out standard models; Figures 16–22 visualize the two strongest synthetic predictors for each natural shift.Source: H Synthetic robustness correlation… · p. 50, Table 2 · p. 50, Figure 16 · p. 52
• ℓp-robustness correlates most strongly with consistency shifts but only weakly with dataset shifts, whereas brightness, gaussian blur, defocus blur, and saturate correlate more strongly with dataset shifts.These relationships are summarized in Table 2 and illustrated through the corresponding scatter plots.Source: H Synthetic robustness correlation… · p. 50, Table 2 · p. 50, Figure 20 · p. 56
• The testbed does not support a causal interpretation of these correlations: training on brightness, gaussian blur, defocus blur, or saturate does not significantly improve effective robustness on dataset shifts.The authors identify finer-grained connections between synthetic and natural shifts as an important direction for future work.Source: H Synthetic robustness correlation… · p. 50

I.1 Constructing beta β

• For each distribution shift, β is constructed from the linear relationship between standard-model performance on the original and shifted distributions.Standard models exclude systems designed with robustness properties or trained on data beyond the standard 1,000-class ImageNet training set.Source: §I.1 · p. 59
• Model accuracies are transformed using the logit distribution before fitting β because this transform assigns greater mass to the tails and produced the best linear fits experimentally.β is the linear predictor of shifted-distribution performance from original-distribution performance in the transformed space.Source: §I.1 · p. 59

I.2 Ablations on our main figures

• The ablation figures preserve the main trends while varying logit scaling, model inclusion, axis subsampling, and display of the full y=x line.Confidence intervals use 1,000 bootstrap linear fits, and the alternative plots clarify fit quality, full-validation axes, and distribution-shift gaps.Source: §I.2 · p. 59, Figure 23 · p. 60, Figure 24 · p. 61

J.1 Natural distribution shift images

• The natural-shift examples cover dataset shifts from ImageNetV2, ObjectNet, ImageNetVid-Robust, and YTBB-Robust, plus consistency shifts in video frames and adversarial shifts from ImageNet-A.These examples distinguish dataset, temporal-consistency, and adversarial forms of natural distribution shift.Source: Figure 26 · p. 63, Figure 27 · p. 63, Figure 28 · p. 64

J.2 Synthetic distribution shift images

• The synthetic-shift examples illustrate the testbed’s generated perturbations, with the complete list deferred to Appendix E.The figure explicitly notes that its displayed list is not complete.Source: Figure 29 · p. 65

K.1 ImageNet-R & ImageNet-Sketch

• Models trained on 1 billion Instagram images or JFT-300M substantially outperform most others on ImageNet-R and ImageNet-Sketch, although the effect of extra data is not uniform.Figure 30 shows these models in the top-right green clusters on both datasets.Source: Figure 30 · p. 66, §K.1 · p. 66
• Several models trained on YFCC 100 Million or full ImageNet-21k achieve close to zero or negative effective robustness, qualifying the benefits of larger training sets.The caveat is that extra datasets may contain renditions absent from ImageNet, and the Instagram data’s rendition frequency is uncertain.Source: §K.1 · pp. 66–67
• Synthetic robustness is weakly correlated with ImageNet-R robustness overall (r = 0.35, 0.30), while other-robustness-intervention models show a stronger correlation (r = 0.76).The comparison concerns effective robustness on synthetic shifts versus ImageNet-R; Figure 31 reports the corresponding scatter plots.Source: §K.1 · p. 67, Figure 31 · p. 67
• DeepAugment+AugMix reaches ρ = 11.2% for ResNeXt101 and ρ = 10.2% for ResNet50 on ImageNet-R, while ℓp-robust models also show substantial robustness on both datasets.Feature-denoising models form a high-performing cluster, and AdvProp attains the highest ImageNet-R accuracy among the compared ImageNet-trained models.Source: §K.1 · pp. 67–68
• On ImageNet-C, DeepAugment+AugMix achieves ρ = 14.2% for ResNet50 and ρ = 13.2% for ResNeXt101, comparable to ρ = 13.3% from direct training on four corruptions.On ImageNet-Vid-Robust and YTBB-Robust anchors, DeepAugment is comparable to multi-perturbation training, but it does not provide effective robustness on ImageNetV2 or ObjectNet.Source: §K.1 · p. 68

Video robustness

• Although prior work [35] reported correlations near 0.95 between video robustness and color-corruption accuracy, this testbed finds weaker correlations of 0.1–0.5 on YTBB-Robust and ImageNet-Vid-Robust.The prior analysis defines video robustness conditional on correct anchor-frame classification, leaving standard accuracy as a confounder that also correlates with corruption accuracy.Source: §K.2 · p. 68
• A saturation-trained model drops only 1% under saturation but 18% on video, versus 12% and 19% for a baseline, showing little transfer between these robustness types.This example motivates effective robustness, which corrects for standard-accuracy confounding.Source: §K.2 · pp. 68–69
• Unlike Gu et al. [35]’s split-based evaluation, the derived robust datasets use expert cleaning to ensure subjects remain visible, match the class, and are not very blurry.This preparation is intended to better calibrate classification-performance measurements.Source: §K.2 · p. 69