Source-linked AI summary
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, Rujun Long, Patrick McDaniel
TL;DR
CleverHans addresses the comparability problem in adversarial robustness by standardizing attack implementations and supporting adversarial training. The report describes its attack and defense functionality, benchmark-reporting practices, and versioning, while noting scope boundaries including user-provided data preparation and attack-specific trade-offs.
Problem
Adversarial robustness benchmarks are not comparable without standardized adversarial-example implementations because high accuracy may reflect a weaker attack rather than a robust model.
Method
CleverHans provides reference implementations of attacks, interfaces for applying defenses such as adversarial training, and guidance for reporting benchmark results and versions.
Results
CleverHans supplies standardized attacks and adversarial-training functionality for robustness development and benchmarking, including a reported FGSM example with eps 0.3 and test set accuracy of 97.9%.
Takeaways & Limitations
Using CleverHans makes a high adversarial-benchmark accuracy correspond to a robust model rather than merely a weaker attack implementation.
Takeaways & Limitations
The library does not provide test datasets or preprocessing; users must prepare data and report preprocessing and model-family information for comparability.
Abstract
from arXiv · showhide
CleverHans is a software library that provides standardized reference implementations of adversarial example construction techniques and adversarial training. The library may be used to develop more robust machine learning models and to provide standardized benchmarks of models' performance in the adversarial setting. Benchmarks constructed without a standardized implementation of adversarial example construction are not comparable to each other, because a good result may indicate a robust model or it may merely indicate a weak implementation of the adversarial example construction procedure. This technical report is structured as follows. Section 1 provides an overview of adversarial examples in machine learning and of the CleverHans software. Section 2 presents the core functionalities of the library: namely the attacks based on adversarial examples and defenses to improve the robustness of machine learning models to these attacks. Section 3 describes how to report benchmark results using the library. Section 4 describes the versioning system.
1 Introduction
CleverHans provides standardized adversarial-example attacks and interfaces for developing robust models and benchmarking robustness. Standardization makes reported adversarial accuracy more comparable, while the library remains open-source and framework-flexible.
- Adversarial examples are legitimate inputs modified by small perturbations intended to mislead machine learning models.The perturbations can be imperceptible to humans or smaller than what an 8-bit representation captures for models accepting 32-bit inputs.
- CleverHans provides reference implementations for adversarial training and standardized adversarial robustness benchmarks.Without a standard attack implementation, high benchmark accuracy may reflect a weaker attack rather than a more robust model.
- The library helps developers add defenses against adversarial examples and benchmark their models’ robustness.Its interface accepts models implemented with frameworks such as Keras or without a specific model abstraction.
- CleverHans is free, open-source software licensed under the MIT license.
2 Core functionalities
CleverHans organizes adversarial-example attacks around shared interfaces and provides supporting model abstractions. The library documents the research results underlying its implementations.
- CleverHans organizes its package by modules.
- The attacks module contains a common Attack interface and implementations of several specific attacks.
- The model module defines a lightweight interface that models implement to be compatible with Attack.CleverHans includes a Keras Sequential implementation and examples for framework-independent TensorFlow models.
- The library describes research results behind the implementations in CleverHans.
2.1 Attacks
CleverHans attacks take a model and an input and return an adversarial example. The described methods include optimization-based and gradient-based procedures with configurable perturbation constraints.
- CleverHans attack algorithms take a model and input and return the corresponding adversarial example.
- L-BFGS Method: L-BFGS computes adversarial examples by approximating a box-constrained optimization problem.The approximation uses box-constrained L-BFGS optimization.
- Fast Gradient Sign Method: FGSM linearizes the training cost around an input and computes an adversarial example from that local approximation.
- Fast Gradient Sign Method: ε controls the magnitude of the perturbation introduced by FGSM.Larger ε values increase the likelihood of misclassification but make the perturbation easier for humans to detect.
- Fast Gradient Sign Method: The FGSM implementation returns an adversarial-example tensor and can clip outputs to the expected input-domain range.The method is parameterized by ε.
2.1.3 Carlini-Wagner Attack
The Carlini-Wagner attack formulates adversarial-example generation as a constrained optimization problem, while EAD extends its regularization with an elastic-net penalty. EAD empirically succeeds against strong detection and robust adversarially trained models with minimal visual distortion, but C&W is often slower than other attacks.
- Carlini-Wagner Attack: C&W searches for a small valid-range change δ that changes classification by minimizing an Lp-based objective with trade-off constant c.The success function is nonpositive exactly when the model misclassifies, and c is selected by modified binary search.
- Carlini-Wagner Attack: C&W is powerful but often much slower than other attacks.Fixing c instead of using modified binary search can speed up the attack.
- Elastic Net Method: EAD uses elastic-net regularization, with β controlling the trade-off between L1 and L2 terms, and applies ISTA shrinkage-thresholding during optimization.
- Elastic Net Method: β = 0 makes C&W L2 a special case of EAD.
- Elastic Net Method: EAD has been reported to bypass strong detection schemes and succeed against robust adversarially trained models while producing minimal visual distortion.
2.1.5 Basic Iterative Method
The library implements iterative and gradient-based attacks that perturb inputs toward misclassification or a chosen target, with configurable attack parameters and returned adversarial examples. BIM repeatedly applies small clipped gradient steps, while PGD adds random restarts; related methods include momentum and saliency-based attacks.
- 2.1.5 Basic Iterative Method: BIM extends the fast gradient method by applying small steps repeatedly while clipping each intermediate result within an ε-neighborhood of the original input.Its implementation exposes ε, the step-size per iteration, and the number of iterations.
- 2.1.6 Projected Gradient Descent: PGD is essentially projected gradient descent on the negative loss function and explores the loss landscape by restarting from many points in L∞ balls around the inputs.The implementation adds a boolean option controlling whether to apply an initial random perturbation.
- 2.1.7 Momentum Iterative Method: MIM accelerates iterative attacks by accumulating a velocity vector in the gradient direction across iterations.Its implementation shares BIM’s ε, step-size, and iteration-count parameters and adds a decay factor.
- 2.1.8 Jacobian-based Saliency Map Approach: JSMA computes Jacobian-based saliency scores, repeatedly increases the feature with the largest score, and stops when the target class is achieved or the perturbation limit is reached.The implementation returns the adversarial example, target-achievement status, and number of perturbed features.
2.1.9 DeepFool
DeepFool seeks adversarial examples with minimal Euclidean distortion by projecting inputs toward nearby separating hyperplanes and iteratively approximating this process for nonlinear neural networks. It is restricted to non-targeted attacks.
- 2.1.9 DeepFool: DeepFool cannot specify a target class and only aims to change the model’s prediction away from the original class.Its implementation exposes class-count, termination, and maximum-iteration parameters.
- 2.1.9 DeepFool: DeepFool finds least-distortion adversarial examples by projecting inputs toward the closest separating hyperplane.The method uses an approximate iterative algorithm to handle nonlinear neural networks.
2.1.10 Feature Adversaries
Feature Adversaries alter inputs to match internal representations at a selected network layer while constraining input-space changes. The section also describes SPSA as a gradient-free attack for models whose gradients are unavailable or unhelpful.
- 2.1.10 Feature Adversaries: Feature Adversaries confuse models by matching internal representations, rather than only changing the predicted class.The representation is targeted at a specified layer.
- 2.1.10 Feature Adversaries: Feature-adversarial examples minimize Euclidean distance between deep representations while constraining input distance under L∞ to less than δ.The optimization uses box-constrained L-BFGS, and the implementation targets a chosen layer.
- 2.1.11 SPSA: SPSA estimates gradients with finite differences in random directions, making it useful for non-differentiable models or cases where gradients do not point in useful directions.Its implementation exposes ε, optimization steps, learning rate, and finite-difference perturbation size.
2.2 Defenses
CleverHans defenses aim to reduce model sensitivity to small input perturbations by modifying training procedures. Adversarial training incorporates adversarial examples by averaging losses on legitimate and adversarial inputs.
- 2.2 Defenses: Defenses seek to make models smoother by limiting sensitivity to small input perturbations, making adversarial examples harder to craft.The defense implementations are placed in the library’s model-training functions.
- 2.2 Defenses: Adversarial training injects adversarial examples during training to improve model generalization.The training function accepts an adversarial-example tensor for this purpose.
- 2.2 Defenses: When adversarial examples are supplied, the training loss is the average of losses on legitimate and adversarial inputs while the rest of training remains unchanged.This combines both prediction losses in the objective used to optimize model parameters.
3 Reporting Benchmark Results
Benchmark reports should identify the CleverHans version, attack configuration, and relevant data and preprocessing details. Matching versions improves comparability with prior benchmarks, while new studies should use the latest release.
- Use the same CleverHans version as prior benchmarks to minimize differences caused by undetected behavior changes between releases.
- For new results not directly compared with previous work, use the most recent versioned CleverHans release.
- Report the CleverHans version, attack methods, and configuration parameter values used in each benchmark.
- A reproducible report can specify CleverHans v2.1.0, FGSM with eps 0.3, and 97.9% test-set accuracy.
- Because the library supplies neither test datasets nor preprocessing, users must document preprocessing and model family sufficiently for comparison.
4 Versioning
CleverHans uses semantic versioning so release numbers communicate changes relevant to reproducible benchmarks. Benchmark comparisons should account especially for major-version differences when bug fixes alter reported accuracy.
- CleverHans version numbers use the MAJOR.MINOR.PATCH semantic-versioning format.
- PATCH releases contain backwards-compatible bug fixes, MINOR releases add backwards-compatible features, and MAJOR releases change interfaces.
- A bug fix that changes benchmark accuracy is treated as an API change and triggers a MAJOR version increment in the next release.
- Academic comparisons should use CleverHans benchmark results produced with the same MAJOR version number.
- Release notes for each CleverHans version are available in the project's GitHub releases.