PaperlayerView exact revision on arXiv

Source-linked AI summary

RobustBench: a standardized adversarial robustness benchmark

Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, Matthias Hein

arXiv:2010.09670v3cs.LGcs.CRcs.CVstat.ML

TL;DR

Adversarial-robustness progress is difficult to assess because evaluations can overestimate robustness and the field lacks a systematic benchmark. RobustBench standardizes model restrictions and AutoAttack-based evaluation while supporting adaptive attacks, then reports broad benchmark and model-analysis results. It provides ongoing leaderboards, unified access to robust models, and analyses spanning robustness-related properties.

  • Problem

    The field lacks a systematic understanding of adversarial-robustness progress, while error-prone evaluations can overestimate robustness and obscure which defenses genuinely improve robustness.

  • Method

    RobustBench restricts evaluated classifiers, uses AutoAttack’s white- and black-box attack ensemble, supports external adaptive evaluations, and provides leaderboards plus an open Model Zoo.

  • Results

    Robustness is associated with significantly greater underconfidence and worse calibration, while not all robust models have higher privacy leakage than standard models.

  • Takeaways & Limitations

    The benchmark and Model Zoo support standardized progress tracking, downstream use of robust models, and broader evaluation of attacks and robustness-related properties.

  • Takeaways & Limitations

    The current benchmark focuses on selected image-classification threat models and excludes randomized classifiers and inference-time optimization-loop defenses.

Abstract

from arXiv · show

As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models. A key challenge in benchmarking robustness is that its evaluation is often error-prone leading to robustness overestimation. Our goal is to establish a standardized benchmark of adversarial robustness, which as accurately as possible reflects the robustness of the considered models within a reasonable computational budget. To this end, we start by considering the image classification task and introduce restrictions (possibly loosened in the future) on the allowed models. We evaluate adversarial robustness with AutoAttack, an ensemble of white- and black-box attacks, which was recently shown in a large-scale study to improve almost all robustness evaluations compared to the original publications. To prevent overadaptation of new defenses to AutoAttack, we welcome external evaluations based on adaptive attacks, especially where AutoAttack flags a potential overestimation of robustness. Our leaderboard, hosted at https://robustbench.github.io/, contains evaluations of 120+ models and aims at reflecting the current state of the art in image classification on a set of well-defined tasks in $\ell_\infty$- and $\ell_2$-threat models and on common corruptions, with possible extensions in the future. Additionally, we open-source the library https://github.com/RobustBench/robustbench that provides unified access to 80+ robust models to facilitate their downstream applications. Finally, based on the collected models, we analyze the impact of robustness on the performance on distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.

1 Introduction

RobustBench addresses unreliable adversarial-robustness evaluation by standardizing model restrictions and attacks. It provides leaderboards, a model collection, and analyses of robustness across several properties.

  • Motivation: More than 3000 papers exist on adversarial robustness, yet it remains unclear which defenses genuinely improve robustness rather than exploit weak attacks.The evaluation problem is computationally hard, and unsuitable attacks can overestimate robustness.
  • Benchmark design: RobustBench restricts evaluated defenses by excluding zero-gradient, randomized, and inference-time optimization-loop classifiers.The authors state that some non-certified defenses violating these restrictions mainly hinder gradient-based attacks without substantially improving robustness.
  • Benchmark outputs: The benchmark uses standardized AutoAttack evaluations, complemented by external adaptive evaluations, to track progress and identify successful robust-training ideas.The leaderboard covers more than 120 evaluations and is intended to reflect the current state of the art.
  • Benchmark outputs: The Model Zoo offers more than 80 robust models for downstream applications and facilitates evaluations of new adversarial attacks.The collection is designed to make robust models easy to use and broaden attack testing.
  • Analysis: RobustBench analyzes how robustness relates to distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.The analysis finds robust models are significantly underconfident, leading to worse calibration, while not all robust models have higher privacy leakage than standard models.

2 Background and related work

The paper frames robustness evaluation around computationally difficult perturbation search and standardized threat models. It distinguishes RobustBench from earlier attack libraries and benchmarks through broader, ongoing, and more standardized evaluation.

  • Background: Robust accuracy is the fraction of datapoints classified correctly for all perturbations in the allowed set, but exact computation is generally intractable.For ℓp-balls, exact robust-accuracy computation is NP-hard even for single-layer neural networks.
  • Background: Attack-based evaluations can overestimate robustness when attacks or parameters are unsuitable, especially under gradient obfuscation.Certified methods instead provide lower bounds that often underestimate robustness.
  • Threat models: The fully white-box threat model assumes the attacker knows the model, with ℓp perturbations constrained by Δ_p = {δ ∈ R^d, ∥δ∥_p ≤ ε}.The paper focuses on ℓ∞ and ℓ2 settings while noting that small ℓp robustness is necessary but not sufficient for broader robustness.
  • Threat models: Common-corruption robustness is measured by average classification accuracy across corruption types and severity levels, without an attacker changing the classifier’s decision.These corruptions aim to mimic naturally occurring image modifications and are not imperceptible like ℓp perturbations.
  • Related work: Earlier attack libraries lacked up-to-date state-of-the-art models, while prior benchmarks were limited by fixed deadlines, incomplete evaluations, or small testbeds.The paper notes that RobustML did not provide a comprehensive overview because community submissions were infrequent.
  • RobustBench: RobustBench combines standardized worst-case evaluations with adaptive-evaluation support, explicit threat models, broader defense coverage, an updating leaderboard, and an open Model Zoo.Its standardized attack ensemble includes both white-box and black-box attacks, unlike RobustML’s reliance solely on adaptive evaluations.

3 Description of RobustBench

RobustBench standardizes robustness evaluation through restricted model classes, fixed threat-model setups, AutoAttack, adaptive checks, and unified model access. It also supports attack comparisons and analyses across robustness-related properties.

  • Restrictions: The benchmark restricts models to avoid defenses that make standard attacks fail without genuinely improving robustness.The stated restrictions include non-zero input gradients, deterministic forward passes, and no optimization loop in the forward pass.
  • Maintenance: The benchmark is intended to remain reliable and sustainable, while new defenses and adaptive evaluations are added as the leaderboard evolves.The authors welcome community contributions and require new entries to satisfy the benchmark restrictions and provide a publicly available paper.
  • Leaderboards: RobustBench organizes leaderboards for ℓ∞, ℓ2, and common-corruption threat models across CIFAR-10, CIFAR-100, and ImageNet.CIFAR-10 and CIFAR-100 use ε∞=8/255 and ε2=0.5; ImageNet uses ε∞=4/255.
  • Evaluation: AutoAttack evaluates ℓ∞ and ℓ2 defenses with four sequential white- and black-box attacks, without hyperparameter tuning.The ensemble uses two PGD variants, targeted FAB, and Square Attack, with later attacks applied to previously unsolved points.
  • Evaluation: AutoAttack can trigger additional adaptive evaluations when black-box attacks outperform white-box attacks, indicating possible robustness overestimation.For the Xiao et al. model, Square Attack improved robust accuracy by more than 10%.
  • Model Zoo: The Model Zoo centralizes checkpoints and provides unified access to robust models for downstream applications and attack comparisons.The repository aims to simplify model integration and enable evaluations across more than 80 models.

4 Analysis

The Model Zoo analysis examines how adversarial robustness relates to distribution shifts, calibration, OOD detection, fairness, privacy leakage, smoothness, and transferability. Robustness often improves some properties but can also worsen calibration and OOD detection, while effects on privacy leakage are mixed.

  • Performance across various distribution shifts: Improved robust accuracy on CIFAR-10 correlates with improved standard accuracy across several distribution shifts.The tested shifts include CIFAR-10-C, CIFAR-10.1, and CINIC-10.
  • Calibration: Most ℓ∞-robust models are significantly underconfident and have worse calibration than standard models before temperature rescaling.ECE reaches 70% for Pang et al.’s model and averages 21% for robust models, compared with 4% for the standard model.
  • Out-of-distribution detection: ℓ∞-robust training generally degrades OOD detection quality compared with standard training, whereas ℓ2-robust models are generally comparable to standard models.The ℓ∞ analysis uses AUROC on CIFAR-100, SVHN, and Describable Textures; Augustin et al.’s ℓ2 model performs better.
  • Fairness in robustness: Higher robust accuracy generally corresponds to lower classwise RSD, indicating reduced disparity, although MART can increase disparity at similar robustness.The class “deer” shows notable degradation under the ℓ∞ threat model.
  • Privacy leakage: Privacy leakage varies widely across robust models, with no strong correlation to robust accuracy and a closer relationship to the generalization gap.Some robust models have lower privacy leakage than the standard model.
  • Transferability and smoothness: Adversarial examples transfer strongly among robust models and from non-robust to robust models, but transfer poorly from robust to non-robust models.For a fixed architecture, standard training produces significantly less smooth classifiers than robust training.

5 Outlook

RobustBench aims to provide a standardized, resource-bounded assessment of adversarial robustness while supporting broader analysis and future expansion to new settings.

  • The benchmark uses defined threat models, model restrictions, AutoAttack, and adaptive attacks to improve robustness assessment when exact evaluation is infeasible.AutoAttack supports standardized comparison, while potentially unreliable evaluations are flagged for additional adaptive attacks.
  • The leaderboard tracks more than 120 evaluations to identify successful ideas in robust-model training and reflect the current state of the art.
  • The Model Zoo provides unified access to many robust models for testing attacks, defenses, perceptual distances, and other robustness-related properties.
  • Future leaderboards may cover sparse, multi-norm, optimized-corruption, and unseen perturbations, as well as test-time adaptation defenses.
  • The benchmark may extend beyond image classification to tasks such as segmentation and retrieval, and domains including natural language processing and malware detection.

A Broader impact

The authors acknowledge that benchmark restrictions may bias research directions and that robustness improvements can have broader effects beyond adversarial accuracy. They therefore examine multiple properties and caution that benchmark performance does not ensure real-world safety.

  • Benchmark restrictions may discourage research directions that violate the allowed-defense criteria, although the authors consider those restrictions necessary for reliable evaluation.
  • Improving adversarial robustness may have unwanted side effects on tasks where adversarial attacks can serve beneficial purposes.
  • The analysis covers calibration, privacy leakage, fairness, distribution shifts, out-of-distribution detection, smoothness, and transferability to assess broader effects of robustness.
  • Good benchmark performance does not guarantee deployment safety because real-world applications may require domain-specific threat models beyond digital ℓp-bounded attacks and artificial common corruptions.

B Licenses

The benchmark code and model collection use open-source or author-selected licenses, with most models under MIT and some under more restrictive terms.

  • The benchmarking code and AutoAttack code are released under the MIT license.
  • Model Zoo classifiers use licenses chosen by their authors, including MIT, Attribution-NonCommercial-ShareAlike 4.0 International, Apache License 2.0, and BSD 3-Clause License.

C Maintenance plan

RobustBench is maintained through free hosting services, cloud storage, and research-group GPU resources, with modest storage costs and community submission processes supporting its operation.

  • The leaderboard is hosted on GitHub Pages, which the authors use as a free service.
  • The library is hosted on GitHub, whose basic maintenance features are available without cost.
  • The Model Zoo occupies around 24 GB in Google Drive, fitting within a 100 GB plan costing 2 USD per month.
  • Evaluations run on GPU servers available to the authors’ research groups and incur no extra costs.
  • Expanding datasets and threat models may increase maintenance costs, while submission instructions are provided to encourage community-driven growth.

D Details of the ImageNet leaderboards

The ImageNet leaderboard uses a fixed 5,000-image validation subset and accommodates differing preprocessing and input resolutions as model design choices.

  • ImageNet evaluation uses 5,000 randomly sampled validation images whose IDs are released in the Model Zoo.The same fixed images support ℓp-robustness and common-corruption evaluation.
  • Models may use different preprocessing steps and input resolutions, which the benchmark treats as additional design choices.
  • The ImageNet ℓ∞ leaderboard applies ε∞ = 4/255 to every classifier regardless of post-preprocessing input dimensionality.The threshold is shared because ℓ∞ constraints are componentwise independent.

E Reproducibility and runtime

The benchmark provides code and fixed dependencies for reproducibility, while assessing seed variability and runtime as part of its evaluation procedure.

  • Benchmark code, example commands, installation instructions, and automatically installed requirements are provided in the repository README.
  • Reproducibility analysis considers robust-accuracy variability across random seeds and average benchmark runtime.
  • ImageNet uses a fixed 5,000-image subset with released IDs for both robustness and common-corruption evaluations.Each point receives 15 corruption types at 5 severity levels.
  • Fixing torch==1.7.1 and torchvision==0.8.2 addresses small preprocessing-related standard-accuracy differences across software versions.Observed differences reached up to 0.16% on 5,000 points for the same model.
  • Repeated AutoAttack evaluations show very small robust-accuracy fluctuations across four random seeds, with runtime varying by model robustness.

F Additional analysis

Additional analyses examine robustness across datasets and threat models, then relate robustness to calibration, distribution shifts, OOD detection, fairness, privacy leakage, smoothness, and transferability.

  • Progress on adversarial defenses: 82.32% versus 0.00% robust accuracy and 95.74% versus 94.78% standard accuracy distinguish the best ℓ2 model from a standard CIFAR-10 model.The best ℓ2-robust model also leaves a relatively small standard–robust accuracy gap.
  • Robustness across distribution shifts: ℓp adversarial robustness is maintained across CIFAR-10, CINIC-10, CIFAR-10.1, and CIFAR-10-C distribution shifts.Robustness on shifted datasets highly correlates with robustness on the original CIFAR-10 dataset.
  • Calibration: Most ℓ2-robust models are underconfident and have worse calibration before and after temperature rescaling.Engstrom et al. achieves 1.41% ECE before rescaling, while Gowal et al. reaches 1.00% after rescaling, compared with the standard model’s 3.71% and 1.11%, respectively.
  • Out-of-distribution detection: ℓ2-robust models generally have OOD detection performance comparable to standard models, while Augustin et al. performs better.The Augustin et al. approach explicitly optimizes robust accuracy and worst-case OOD detection performance.
  • Fairness in robustness: Higher average robust accuracy is associated with lower relative standard deviation across classes, indicating reduced subgroup disparity.
  • Privacy leakage: Adversarial examples achieve higher membership-inference accuracy than benign images, while confidence information closely follows label-based inference accuracy for adversarial examples.
  • Smoothness: Robustly trained WRN-28-10 models are smoother than the standard model at all layers, while the most robust models are smoothest up to the last layer.
  • Smoothness: Robust accuracy has a clear relationship with Lipschitzness across a broader set of ℓ∞ models.

G Leaderboards

The leaderboards report model accuracy and robustness across threat models and common corruptions, with accompanying metadata and transferred-adversarial-example evaluations.

  • Leaderboard metadata: Leaderboard entries report clean accuracy, robust accuracy, extra training data, architecture, publication venue, and Model Zoo identifiers when available.
  • Transferability evaluations: Transferred-adversarial-example evaluations are organized by target network and source network, where attacks are evaluated on the target and generated from the source.
  • Transferability evaluations: The transferability figures evaluate ℓ∞ perturbations at 8/255 and ℓ2 perturbations at 0.5 using ten-step PGD and FGSM attacks.
  • Leaderboard scope: The benchmark includes CIFAR-10 ℓ∞, ℓ2, and corruption leaderboards, with corresponding tables for CIFAR-100 and ImageNet.
Loading 2010.09670v3…