Source-linked AI summary
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer
TL;DR
Robustness research lacks consensus because evaluations and findings across distribution shifts are inconsistent. This paper introduces four real-world datasets, evaluates robustness interventions, and finds that some methods help particular shifts but no evaluated method consistently improves robustness across all settings.
Problem
Robustness research lacks consensus on evaluation benchmarks and contains conflicting findings about how models generalize to distribution shifts.
Method
The paper introduces four real-world distribution-shift datasets and reevaluates larger models, self-attention, data augmentation, pretraining, and DeepAugment across new and prior benchmarks.
Results
No single method consistently improved robustness across all distribution shifts, while larger models and diverse data augmentation helped on several shifts and DeepAugment outperformed a model pretrained on 1000× more labeled data.
Takeaways & Limitations
Robustness evaluations should study multiple distribution shifts simultaneously because interventions that help texture and local-statistics shifts may help little with geographic shifts.
Takeaways & Limitations
The study finds that no evaluated robustness method consistently helps in all settings, reflecting the complexity of the robustness landscape.
Abstract
from arXiv · showhide
We introduce four new real-world distribution shift datasets consisting of changes in image style, image blurriness, geographic location, camera operation, and more. With our new datasets, we take stock of previously proposed methods for improving out-of-distribution robustness and put them to the test. We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work. We find improvements in artificial robustness benchmarks can transfer to real-world distribution shifts, contrary to claims in prior work. Motivated by our observation that data augmentations can help with real-world distribution shifts, we also introduce a new data augmentation method which advances the state-of-the-art and outperforms models pretrained with 1000 times more labeled data. Overall we find that some methods consistently help with distribution shifts in texture and local image statistics, but these methods do not help with some other distribution shifts like geographic changes. Our results show that future research must study multiple distribution shifts simultaneously, as we demonstrate that no evaluated method consistently improves robustness.
1. Introduction
The paper addresses conflicting accounts of robustness by introducing diverse real-world distribution-shift datasets and evaluating proposed interventions across them. It finds that augmentation and larger models can improve some real-world shifts, while no method consistently helps across all settings.
- Existing robustness research lacks consensus because studies use inconsistent benchmarks and propose conflicting hypotheses about model fragility and effective interventions.
- The authors introduce four datasets covering naturally occurring shifts in rendition style, image capture, geography, and blur.
- ImageNet-R contains 30,000 naturally occurring renditions whose textures and local image statistics differ from ImageNet images.
- SVSF and DFR isolate changes in location, year, camera, occlusion, orientation, zoom, and scale while controlling texture-related confounds.
- DeepAugment improves robustness on ImageNet-R and, combined with other augmentations, outperforms a model pretrained on 1000× more labeled data.
- The evaluation strengthens some robustness hypotheses but shows that future work must test multiple distribution shifts because methods do not help uniformly.
2. Related Work
Prior work introduced varied OOD benchmarks and interventions but produced conflicting conclusions about natural robustness, texture bias, and the transferability of synthetic robustness gains. The paper situates its datasets within this unsettled literature.
- Robustness benchmarks have produced dissonant findings about whether modern models generalize to naturally occurring distribution shifts.
- ImageNetV2 evaluations reported an 11-14% accuracy drop across architectures, while later work found data augmentation did not significantly improve robustness on that benchmark.
- Data augmentation studies report robustness gains from noise, unnatural transformations, and compositions of simple image operations.
- ImageNet-R visualizes diverse rendition styles for ImageNet objects, with rendition styles shown for clarity rather than used as classes.
3. New Datasets
The paper constructs four benchmarks that separate naturally occurring distribution shifts across visual rendition, image capture, object presentation, and blur. Their metadata-controlled designs target confounds in prior evaluations.
- The four benchmarks comprise ImageNet-R and Real Blurry Images for ImageNet classifiers, plus SVSF and DFR with their own training and multiple test sets.
- 3.1. ImageNet-Renditions: ImageNet-R contains artistic renditions of ImageNet objects, reversing the original dataset’s emphasis on photographs rather than paintings or drawings.
- 3.1. ImageNet-Renditions: ImageNet-R contains 30,000 renditions for 200 ImageNet classes, filtered through annotator and graduate-student quality control.
- 3.2. StreetView StoreFronts and 3.3. DeepFashion Remixed: SVSF uses storefront imagery to vary country, year, and camera, while DFR varies object size, occlusion, viewpoint, and zoom using metadata.
- 3.2. StreetView StoreFronts and 3.3. DeepFashion Remixed: SVSF and DFR alter one metadata attribute at a time, providing controlled test distributions for image-capture shifts.
- 3.4. Real Blurry Images: Real Blurry Images contains 1,000 Flickr images from 100 ImageNet classes and serves as a real-world analog for ImageNet-C blur corruptions.
4. DeepAugment
DeepAugment generates training images by perturbing internal representations of image-to-image networks rather than applying simple operations directly to raw images. The resulting distortions remain semantically consistent and diverse, improving robustness on ImageNet-C and ImageNet-R.
- DeepAugment augments images by perturbing internal representations of deep networks instead of applying simple primitives directly to raw images.
- The method perturbs network weights and feedforward signals at random layers using operations such as zeroing, negating, convolving, and transposing.
- DeepAugment produces semantically consistent images with unique and visually diverse distortions.
- DeepAugment outperforms other methods on ImageNet-C and ImageNet-R despite using randomly designed perturbation operations.
- The experiments use CAE and EDSR image-to-image architectures and train on clean and distorted versions simultaneously.
5. Experiments
Experiments across ImageNet-R, SVSF, DFR, and ImageNet-C show that robustness interventions help for some shifts but not others. Augmentation and larger models are especially effective for texture, local-statistics, and blur shifts, while geographic and other shifts remain difficult.
- ImageNet-R: DeepAugment+AugMix improves ImageNet-R top-1 error over the baseline by over 10 percentage points.The combined method demonstrates that progress on ImageNet-R is tractable.
- ImageNet-R: DeepAugment is the best standalone ImageNet-R method, achieving a 57.8% error rate.Style Transfer, AugMix, and DeepAugment improve generalization relative to simpler augmentation techniques.
- StreetView StoreFronts: On SVSF, data augmentation hardly helps, and country-shift error rates roughly double across methods.The evaluation is limited to augmentations because of a 30-day retention window for each dataset instantiation.
- DeepFashion Remixed: On DFR, evaluated methods have average OOD mAP close to baseline, with most gains tracking IID mAP rather than substantially closing the generalization gap.Size and occlusion shifts hurt performance most; Random Erasure helps occlusion, while Style Transfer helps more.
- Cross-dataset findings: Across the evaluated shifts, no method substantially improves robustness everywhere.Larger models, self-attention, augmentation, and pretraining improve some robustness measures, but their benefits depend on the shift type.
- Real Blurry Images and ImageNet-C: All interventions that help on ImageNet-C also help on Real Blurry Images, supporting ImageNet-C as a controlled proxy for real-world corruptions.DeepAugment+AugMix achieves the lowest error rate on Real Blurry Images.
6. Conclusion
The paper evaluates robustness interventions across multiple real-world and synthetic distribution shifts, finding that benefits are method- and shift-specific rather than universal. Its results support synthetic benchmarks for some real-world shifts while showing that geographic changes remain difficult.
- The study introduces four real-world robustness datasets and uses them to re-evaluate prior robustness interventions and hypotheses.
- Approximately 19% accuracy increase from DeepAugment+AugMix on ResNet-50 shows that IID accuracy and ImageNet-C accuracy are not coupled.
- Larger models, self-attention, diverse augmentation, and pretraining each improve some benchmarks but fail to help consistently across all distribution shifts.
- Diverse augmentation improves ImageNet-R, ImageNet-C, and Real Blurry Images, while methods that help texture-related shifts help little with geographic shifts.
- Gains from data augmentation on ImageNet-C generalize to ImageNet-R and Real Blurry Images, supporting synthetic benchmarks as robustness measures for these shifts.
- The authors argue that robustness research should use multifaceted evaluations because no evaluated method consistently helps in all settings.
A. Additional Results
Additional results show that model scale and pretraining have sharply different effects across robustness benchmarks. ImageNet-R benefits strongly from some pretraining setups, but controlled pretraining provides little improvement.
- ImageNet-R: 37.5% improvement on ImageNet-R reduces ResNeXt-101 32×8d top-1 error from 57.5% to 24.2% with WSL pretraining on Instagram images.
- ImageNet-R: ImageNet-R error falls from 63.9% with ResNet-50 to 58.7% with ResNet-152, indicating that larger models improve performance.
- Implications for the Four Methods: Larger models help ImageNet-C, ImageNet-A, and ImageNet-R, but do not markedly improve DeepFashion Remixed performance.
- Implications for the Four Methods: Self-attention helps ImageNet-C and ImageNet-A but not ImageNet-R or DeepFashion Remixed.
- Implications for the Four Methods: Diverse augmentation helps ImageNet-C and ImageNet-R but not ImageNet-A, DeepFashion Remixed, or StreetView StoreFronts.
- Implications for the Four Methods: Pretraining helps ImageNet-C and ImageNet-A but does not markedly improve DeepFashion Remixed or ImageNet-R.
B. DeepAugment Details
DeepAugment creates diverse training images by perturbing image-to-image networks, while Noise2Net extends this idea with randomly sampled, untrained networks and real-time per-example augmentation. These augmentation strategies improve robustness across several evaluated benchmarks, with DeepAugment combining particularly strongly with AugMix.
- DeepAugment: DeepAugment perturbs network weights and feedforward activations while processing clean images through image-to-image networks, producing distorted training examples.Perturbations include zeroing, negating, convolving, transposing, and applying activation functions at randomly selected layers.
- Noise2Net: Noise2Net samples its architecture and weights randomly, then applies four residual blocks with ε ∼U(0.375, 0.75) to augment images.Weights are resampled every minibatch, while dilation and kernel sizes are resampled every epoch.
- Noise2Net: Noise2Net processes batch examples differently in parallel using grouped convolutions, increasing real-time augmentation diversity compared with applying one transformation to an entire minibatch.Grouped convolutions use one independent convolutional group per batch example.
- Results: Noise2Net outperforms the EDSR and CAE DeepAugment variants on ImageNet-R despite using randomly sampled, untrained networks.The result supports the utility of diverse deep augmentations without training the augmentation network.
- Results: DeepAugment+AugMix improves robustness by over 23 mCE on ImageNet-C.The reported mCE averages the 15 corruption-error values.
- Results: Larger models and diverse data augmentation help ImageNet-Renditions robustness, whereas self-attention hurts and pretraining provides mixed benefits.The comparison includes ImageNet-21K and WSL pretraining, CBAM and SE self-attention, larger ResNets or ResNeXts, and augmentation methods.
C. Further Dataset Descriptions
The paper’s benchmarks cover object renditions, clothing attributes, storefront conditions, and multiple image transformations. Their test sets vary factors including occlusion, viewpoint, zoom, capture year, camera system, and country.
- StreetView StoreFronts: StreetView StoreFronts contains 20 business-front classes assigned from Google Maps listings using machine learning models and human annotators.The images are cropped from StreetView images by an object detection model.
- DeepFashion Remixed: DeepFashion Remixed covers clothing classes and tests shifts in occlusion, size, viewpoint, and zoom.The supplied class description includes short sleeve outerwear, while the benchmark defines three levels for each shift factor.
- ImageNet-Renditions: ImageNet-Renditions evaluates ImageNet-trained models on varied object renditions across 200 ImageNet classes.The listed classes span animals, objects, foods, vehicles, and people.
- StreetView StoreFronts: StreetView StoreFronts test sets vary capture year, camera system, and country alongside an in-distribution set.The listed shifts include 2018 versus 2017 capture, new versus other camera systems, and France.
- Noise2Net: Noise2Net produces different augmented outputs as ε changes, with ε = 0 corresponding to the original image.The method’s parallel implementation uses grouped convolutions so different transformations can be applied to each image in a batch.