Source-linked AI summary
Foolbox: A Python toolbox to benchmark the robustness of machine learning models
Jonas Rauber, Wieland Brendel, Matthias Bethge
TL;DR
Machine-learning models can be fooled by nearly imperceptible input perturbations, creating a need to measure and compare their robustness. Foolbox addresses this with a Python toolbox of attacks, criteria, distance measures, and framework interfaces, using internally tuned attacks to seek minimum perturbations. It supports reproducible benchmarking through explicit reporting of versions and evaluation settings.
Problem
Machine-learning models can be fooled by almost imperceptible perturbations, motivating tools to generate adversarial examples and compare model robustness.
Method
Foolbox organizes reference and new adversarial attacks around models, criteria, and distance measures, with internal hyperparameter tuning to find minimum perturbations.
Results
Foolbox supports adversarial robustness benchmarking across multiple attacks, criteria, distance measures, and popular machine-learning frameworks.
Takeaways & Limitations
Comparable benchmarks require reporting the Foolbox version, inputs, attacks, non-default hyperparameters, criterion, and distance metric.
Abstract
from arXiv · showhide
Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at https://github.com/bethgelab/foolbox . The most up-to-date documentation can be found at http://foolbox.readthedocs.io .
1.1. Structure
Foolbox structures adversarial-example generation around the model, adversarial criterion, distance measure, and attack algorithm required to evaluate perturbations. Its Python modules mirror these elements.
- Adversarial-example generation requires a model, criterion, distance measure, and attack algorithm operating on an input and its label.
- Foolbox implements this structure through five Python modules: models, criteria, distances, attacks, and adversarial.
Models
Foolbox provides model interfaces for several popular machine-learning frameworks and standardizes the prediction and gradient operations used by attacks. It also supports combining predictions from one model with gradients from another.
- The models module interfaces with Keras, MXNet, TensorFlow, PyTorch, Theano, and Lasagne.
- Each framework interface accepts a framework-specific model representation and exposes standardized prediction and gradient methods to attacks.
- CompositeModel combines predictions from one model with the gradient of another, enabling gradient-based attacks on non-differentiable models and transfer attacks.
Criteria
Foolbox criteria define when an input-label pair is adversarial, including untargeted, targeted, probability-threshold, and top-k conditions. Criteria can also be customized, although some attacks support only particular criteria.
- Misclassification treats an input as adversarial when its predicted class differs from the original class.
- Top-k misclassification requires the original class to fall outside the top-k predicted classes.
- Original-class and target-class probability criteria use thresholds on the original or specified target class probability.
- Targeted misclassification requires the predicted class to equal a given target class.
- Users can define custom adversarial criteria, but some attacks are inherently restricted to particular criteria.
Distance Measures
Distance measures quantify adversarial perturbation size, and Foolbox implements common measures while allowing extensions. Inputs are normalized by their allowed value range to make distances scale-invariant.
- Foolbox implements mean squared, mean absolute, L0, and L∞ distance measures for vectors x and y.
- L0 counts differing elements, while L∞ measures the maximum absolute coordinate difference.
- Each input element is normalized by the difference between the smallest and largest allowed value to achieve scale invariance.
Attacks
Foolbox implements many adversarial attacks, each using a model and adversarial criterion to search for a minimum perturbation.
- Foolbox implements a large number of adversarial attacks.The paper refers readers to section 2 for an overview.
- Each attack receives a model and a criterion defining what counts as an adversarial.The default criterion is misclassification.
- Attacks tune hyperparameters internally to find the minimum perturbation.
Adversarial
Foolbox's adversarial object stores the context and result of an attack, and can support pausing and resuming long-running attacks.
- An adversarial object encapsulates the model, criterion, distance measure, original input, label, and smallest perturbation found.
- Applying an attack to an input-label pair automatically creates an adversarial object.
- Setting unpack to False returns the full object, which can be reused to pause and resume long-running attacks.
1.2. Reporting Benchmark Results
Foolbox benchmark reports should identify the software version, data, attacks, settings, criterion, and distance metric used.
- Benchmark reports should state the Foolbox version number.
- Reports should identify the input samples and attacks applied to them.
- Reports should disclose non-default hyperparameters, the criterion, and the distance metric.
1.3. Versioning System
Foolbox uses semantic versioning with benchmark-oriented distinctions between breaking changes, result-affecting compatible changes, and bug fixes. Comparable model robustness evaluations should use the same MAJOR.MINOR version and report the version number.
- Foolbox releases use MAJOR.MINOR.PATCH version numbers based on semantic versioning.
- MAJOR versions indicate API changes that break compatibility with previous versions.
- MINOR versions indicate added functionality or compatible changes that can affect benchmark results.
- PATCH versions indicate compatible bug fixes that do not affect benchmark results.
- Comparing two models requires the same MAJOR.MINOR Foolbox version, which should be reported with benchmark results.
2. Implemented Attack Methods
Foolbox implements a broad range of attack strategies, including gradient-based, score-based, decision-based, and specialized perturbation attacks. These methods use different model information and perturbation objectives, while tuning attack parameters to seek small adversarial changes.
- Gradient-based attacks: Gradient-based attacks use loss gradients to identify input directions that most affect model predictions.Implemented variants include single-step gradient and gradient-sign attacks, as well as iterative versions that update the input repeatedly.
- Gradient-based attacks: DeepFool approximates the classifier linearly and steps toward the nearest class boundary under L2 or L∞ distance.For each alternative class, it estimates the distance to the boundary and selects the smallest one.
- Optimization-based attacks: Optimization-based attacks include L-BFGS-B and SLSQP, which seek minimum adversarial perturbations under objective or constraint formulations.L-BFGS-B uses line search over a regularization parameter, whereas SLSQP can directly handle nonlinear constraints.
- Feature- and search-based attacks: Saliency-map, single-pixel, and local-search attacks select or modify input features according to their estimated influence on classification.Saliency Map uses gradients, while Single Pixel and Local Search probe sensitivity to individual pixels.
- Access-dependent attacks: Score-based attacks estimate useful directions from probabilities or logits, whereas decision-based attacks require only the model’s class decision.Foolbox includes numerical-gradient, Boundary, and Pointwise attacks for settings with limited model access.
- Noise and transformation attacks: Foolbox also tests robustness to uniform, Gaussian, salt-and-pepper, blur, and contrast-reduction perturbations, plus precomputed adversarial candidates.The noise and image-transformation attacks perform internal line searches for minimal adversarial perturbations.