PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Robustness Toolbox v1.0.0

Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian M. Molloy, Ben Edwards

arXiv:1807.01069v4cs.LGstat.ML

TL;DR

Machine-learning models are vulnerable to adversarially modified inputs, motivating tools for testing and defending model pipelines. ART provides a Python library that integrates attacks, defences, and detection across supported machine-learning frameworks. The documented release includes multiple defence approaches and attack implementations, with some methods explicitly limited by their current implementation scope.

  • Problem

    Machine-learning models face adversarial threats from deliberately modified inputs, requiring protection and testing across the machine-learning pipeline.

  • Method

    ART offers framework-independent classifier interfaces and combines adversarial attacks with model-hardening and runtime-detection defences.

  • Results

    ART implements attacks and defences including adversarial training, feature squeezing, label smoothing, spatial smoothing, and poisoning detection.

  • Takeaways & Limitations

    ART supports researchers benchmarking attacks and defences and developers composing and deploying secure machine-learning applications.

  • Takeaways & Limitations

    ART currently does not implement the ℓ1 or ℓ∞ adaptations described for universal adversarial perturbations.

Abstract

from arXiv · show

Adversarial Robustness Toolbox (ART) is a Python library supporting developers and researchers in defending Machine Learning models (Deep Neural Networks, Gradient Boosted Decision Trees, Support Vector Machines, Random Forests, Logistic Regression, Gaussian Processes, Decision Trees, Scikit-learn Pipelines, etc.) against adversarial threats and helps making AI systems more secure and trustworthy. Machine Learning models are vulnerable to adversarial examples, which are inputs (images, texts, tabular data, etc.) deliberately modified to produce a desired response by the Machine Learning model. ART provides the tools to build and deploy defences and test them with adversarial attacks. Defending Machine Learning models involves certifying and verifying model robustness and model hardening with approaches such as pre-processing inputs, augmenting training data with adversarial samples, and leveraging runtime detection methods to flag any inputs that might have been modified by an adversary. The attacks implemented in ART allow creating adversarial attacks against Machine Learning models which is required to test defenses with state-of-the-art threat models. Supported Machine Learning Libraries include TensorFlow (v1 and v2), Keras, PyTorch, MXNet, Scikit-learn, XGBoost, LightGBM, CatBoost, and GPy. The source code of ART is released with MIT license at https://github.com/IBM/adversarial-robustness-toolbox. The release includes code examples, notebooks with tutorials and documentation (http://adversarial-robustness-toolbox.readthedocs.io).

1 Introduction

ART is an open-source Python library for adversarial machine learning that unifies classifiers across major frameworks and supports composable defences. The document provides mathematical and implementation details for attacks, defences, architecture, and library modules.

  • ART provides standardized classifier interfaces for TensorFlow, Keras, PyTorch, MXNet, Scikit-learn, XGBoost, LightGBM, CatBoost, and GPy.
  • Its architecture supports combining defences such as adversarial training, data preprocessing, and runtime detection.
  • The library targets researchers benchmarking attacks or defences and developers deploying secure machine-learning applications.
  • The document explains attack and defence semantics, mathematical backgrounds, custom implementation choices, architecture, and library modules.

2 Background

Adversarial machine learning protects the machine-learning pipeline against attacks during training, testing, and inference. This section introduces evasion and poisoning threats, defence strategies, robustness metrics, and notation for classifier behavior and adversarial perturbations.

  • Adversarial machine learning addresses threats to machine-learning pipelines at training, test, and inference time.
  • Evasion attacks: Evasion attacks modify classifier inputs to cause misclassification while keeping perturbations small, with targeted and untargeted objectives.
  • Defences: Defences include model hardening through adversarial training, preprocessing, regularization, and architectural changes, alongside runtime detection.
  • Robustness metrics: Robustness metrics quantify perturbation required for misclassification or model-output sensitivity to input changes.
  • Poisoning attacks: Poisoning attacks target data collection and training when data sources and curation processes are not fully controlled by model owners.
  • Notation: Classifier notation defines inputs X, outputs Y with K classes, logits Z(x), probabilities F(x), and classification C(x) as the highest-logit class.
  • Notation: Adversarial-sample generation uses the training loss, its input gradient, and class gradients.

3 Library Modules

ART is organized into modules for classifiers, attacks, defences, detection, metrics, poisoning detection, wrappers, utilities, and versioning. The module structure supports the document’s detailed treatment of these components.

  • Attacks: The library contains modules for evasion attacks, including adversarial patches, boundary attacks, Carlini, DeepFool, Fast Gradient, NewtonFool, and projected gradient descent.
  • Classifiers: Classifier modules support black-box, decision-tree, ensemble, Gaussian-process, Keras, LightGBM, MXNet, PyTorch, Scikit-learn, TensorFlow, and XGBoost models.
  • Defences: Defence modules include adversarial training, feature squeezing, Gaussian augmentation, JPEG compression, label smoothing, spatial smoothing, and related preprocessing methods.
  • Detection and evaluation: Detection and evaluation modules cover adversarial-input detection, scanning operations, scoring functions, metrics, verification, and poisoning detection.
  • Supporting modules: The library also includes data generators, utilities, tests, visualization, and a documented versioning system.

4 Classifiers

ART defines framework-independent classifier interfaces that connect machine-learning models to attacks and defences. Its base and mixin classes expose prediction, training, preprocessing, activations, gradients, and model-specific capabilities.

  • ART abstracts classifier implementations from their underlying frameworks through a functional API supporting multiple machine-learning libraries.
  • Supported classifiers: Supported integrations include TensorFlow v1 and v2, Keras, PyTorch, MXNet, Scikit-learn, XGBoost, LightGBM, CatBoost, GPy, Python functions, and ensembles.
  • Classifier base class: Prediction returns class probabilities or logits with output shape (n, K), where n is the number of samples and K the number of classes.
  • Neural-network classifiers: The neural-network interface adds activations, layer names, channel index, and learning-phase access for internal model operations.
  • Gradient classifiers: The gradient interface connects classifiers to white-box attacks by exposing loss and class gradients, including gradients through or approximated across preprocessors.
  • Specialized classifiers: Decision-tree and ensemble interfaces expose tree access and aggregate trained classifiers so attacks can target ensembles.

5 Classifier Wrappers

ART provides classifier wrappers that standardize access to models and alter prediction or gradient computation to support varied attack and defense settings.

  • ART offers a functional API for defining wrappers around ART classifiers according to different attack strategies.
  • ClassifierWrapper exposes the properties and functions of Classifier instances while changing the constructor interface.
  • ExpectationOverTransformations averages predictions and gradients over specified random input transformations.This supports adversarial samples robust to transformations encountered in defenses or real-world synthesis and digitization.
  • QueryEfficientBBGradientEstimation estimates classifier gradients from a specified number of predictions instead of using true gradients.It emulates black-box attacks that estimate gradients through classifier queries.
  • RandomizedSmoothing modifies classifiers that perform well under Gaussian noise to provide certified robustness against L2 adversarial perturbations.

6 Attacks

ART implements a broad collection of evasion attacks through a framework-independent classifier interface, spanning gradient-based, iterative, black-box, spatial, and feature-saliency methods.

  • ART’s attack module targets Classifier objects through a framework-independent API, making attack implementations agnostic to the model’s training framework.The abstract Attack class provides a common interface and stores the target classifier.
  • The library includes FGSM, BIM, PGD, JSMA, Carlini & Wagner, DeepFool, Universal Perturbation, NewtonFool, Virtual Adversarial Method, and additional attacks.Implemented methods also include spatial transformation, Elastic Net, ZOO, Boundary, Adversarial Patch, Decision Tree, HCLU, and HopSkipJump attacks.
  • FGSM controls L1, L2, or L∞ perturbation norms in targeted and untargeted settings by modifying inputs according to classifier loss.Its strength parameter ϵ trades off attack success against perturbation size, and an extension searches incrementally up to ϵmax.
  • FGSM requires only one gradient evaluation and applies directly to batches, making it efficient for generating adversarial-training samples.
  • BIM and PGD iteratively extend FGSM, with PGD projecting the result back onto the ϵ-norm ball around the original input at every iteration.
  • JSMA iteratively modifies individual input components selected by a saliency map until targeted misclassification or a modification budget is reached.The saliency map selects components associated with increasing the target-class probability, while search and modified sets control feasible and used components.

6.7 Carlini & Wagner ℓ∞attack

This passage set describes ART’s attack implementations beyond the main attack overview, including constrained optimization, decision-boundary, universal, black-box, and transformation-based methods, with explicit scope caveats.

  • The C&W L∞ attack seeks an adversarial sample satisfying ℓ(x′) = 0 while enforcing ∥x − x′∥∞ ≤ ϵ through an invertible input transformation.The transformation maps bounded input components to an unconstrained optimization domain while preserving the L∞ constraint after inversion.
  • ART’s C&W L∞ implementation differs from the original by using a less computationally involved formulation that guarantees the stated L∞ bound.It uses the same binary line-search approach as the L2 attack.
  • DeepFool iteratively projects an input toward the nearest decision boundary in L2 norm and clips intermediate and final adversarial samples to data bounds.An overshoot parameter pushes samples across the boundary to change their classification.
  • Universal adversarial perturbations construct one constant perturbation by repeatedly refining it across inputs until a target fooling rate or iteration limit is reached.The perturbation is projected to the selected norm and budget after successful per-input updates.
  • NewtonFool performs gradient descent to decrease the original-class probability, using an adaptive step size whose behavior changes around probability 1/K.The parameter η controls the aggressiveness of the descent.
  • ART also includes virtual adversarial, spatial transformation, Elastic Net, ZOO, Boundary, Adversarial Patch, Decision Tree, HCLU, and HopSkipJump attacks.These methods include black-box attacks based on output queries, transformations shared across batches, and patches intended for natural scenes.

7 Defences

ART organizes defences around model hardening, input preprocessing, and runtime detection. Its interfaces support adversarial training and several preprocessing transformations, including feature squeezing, label smoothing, and spatial smoothing.

  • Defence categories: ART categorizes defences as model hardening, data preprocessing, and runtime detection of adversarial samples.Model hardening creates a more robust classifier, preprocessing transforms inputs or labels, and runtime detection extends the classifier with a detector.
  • Implemented defences: ART implements adversarial training alongside feature squeezing, label smoothing, spatial smoothing, JPEG compression, thermometer encoding, total variance minimization, Gaussian augmentation, and PixelDefend.The listed defences are organized under the evasion-defence module.
  • Adversarial training: Adversarial training augments the training set with adversarial samples generated by specified attack-classifier pairs, producing a hardened classifier.Multiple attacks can be rotated across batches, transferred from another model, and controlled by a ratio specifying clean-sample replacement.
  • Input preprocessing: Feature squeezing reduces input precision, such as converting normalized 8-bit image pixels to b bits where b < 8.The transformation requires no fitting and is automatically applied during classifier fitting or prediction when enabled.
  • Input preprocessing: Label smoothing replaces one-hot labels with representations whose maximum and minimum components differ less, increasing entropy and potentially reducing exploitable gradients.The user specifies ymax ∈ [0, 1].
  • Input preprocessing: Spatial smoothing applies a median filter within a local window separately to each image color channel, with reflected borders where needed.It is an image-specific defence and is automatically applied at prediction when enabled.

8 Evasion Detection

ART provides runtime detection methods for adversarial samples through a unified detector interface. Its detectors include binary input, binary activation, and subset-scanning approaches.

  • Detector types: The evasion-detection module implements BinaryInputDetector, BinaryActivationDetector, and SubsetScanningDetector.These methods share the API provided by the Detector base class.
  • Common interface: The Detector interface supports fitting, checking fitted status, and applying detection to return binary decisions for input samples.The call method applies detection to provided inputs.
  • Binary detectors: BinaryInputDetector trains a binary classifier on clean and adversarial data, labeling adversarial inputs as 1 and non-adversarial inputs as 0.After fitting, the detector is ready to detect adversarial inputs.
  • Binary detectors: BinaryActivationDetector uses activations from one specified layer of another classifier as detector inputs instead of using the original inputs directly.Its binary labels represent whether an input is adversarial.
  • Subset scanning: SubsetScanningDetector uses a fast generalized subset scan to detect anomalous patterns in categorical datasets.The method is designed for general categorical data sets.

9 Poisoning Detection

ART addresses poisoning and backdoor threats by filtering suspicious training data. Activation Clustering analyzes last-hidden-layer activations by class, reducing their dimensionality and clustering them to separate potentially poisonous samples.

  • Threat model: Poisoning attacks exploit potentially untrustworthy training data to alter model decision boundaries or reduce performance.Backdoors can preserve performance on standard inputs while causing targeted failures on attacker-chosen inputs.
  • Poison filtering: ART provides filtering defences that identify suspected poison data when the training data and model are available.The PoisonFilteringDefence interface takes a model and its corresponding training data and returns suspected poisonous points.
  • Activation Clustering: Activation Clustering detects backdoor-poisoned data by exploiting differences between target-class features and source-class features combined with a backdoor trigger.Backdoor and target samples may receive the same classification even though the network relies on different features.
  • Activation Clustering: Activation Clustering retains last-hidden-layer activations, segments them by labels, reduces dimensionality, and applies clustering to each activation segment.The algorithm flattens each activation into a one-dimensional vector before reduction and clustering.
  • Activation Clustering: k-means with k = 2 was found effective for separating poisonous from legitimate activations, but analysts must still determine which cluster contains poison.The release also provides visualization for manual review, after which the model needs corresponding repair.

10 Metrics

ART includes metrics for evaluating classifier robustness against attacks and for analyzing model smoothness or formally oriented robustness bounds. These include empirical robustness, loss sensitivity, CLEVER, and clique-based verification for tree ensembles.

  • Metric overview: ART’s metrics module assesses robustness using empirical perturbation, loss sensitivity, CLEVER, and clique-method verification for decision-tree ensembles.The metrics cover neural-network-style attack assessment and verification for tree-based models.
  • Attack-based metrics: Empirical robustness is the average minimal perturbation required for a specified attack to successfully misclassify test samples.It is computed for a classifier, untargeted attack, and test dataset using the perturbation norm used to create adversarial samples.
  • Smoothness metrics: Local loss sensitivity estimates a model’s Lipschitz continuity constant from classifier-logit gradients, with smaller values indicating smoother functions.It is attack-independent and measures model properties under small input changes.
  • Robustness bounds: CLEVER estimates a lower bound γ on the perturbation needed to change a classification within a specified ℓp norm.The estimate uses a Lipschitz constant for logit gradients sampled within an ℓp-ball around the input.
  • Robustness bounds: For untargeted attacks, CLEVER takes the minimum targeted CLEVER score over all classes different from the classifier’s output.The untargeted implementation omits the target-class parameter.
  • Tree-model verification: ART verifies robustness for gradient-boosted decision trees, random forests, and extra trees using a clique-based method.Supported examples include XGBoost, LightGBM, and Scikit-learn models.

11 Data Generators

ART’s DataGenerator interface standardizes batch-wise data loading and on-the-fly augmentation, including for datasets too large to fit in memory. Framework-specific wrappers implement this interface and integrate with classifier training functions.

  • DataGenerator interface: The DataGenerator interface standardizes data loaders and generators for batch-wise dataset processing.It supports user-defined loading and augmentation generators used with classifier fit-generator functions.
  • Use cases: Data generators are especially useful for datasets that do not fit in memory and for per-batch data augmentation.
  • DataGenerator interface: Batch retrieval returns the next data batch as (x, y).
  • Framework wrappers: ART provides standard wrappers for framework-specific data loaders, all implementing the DataGenerator interface.
  • Framework integration: With TensorFlow or Keras classifier wrappers, fit-generator training delegates directly to the framework’s fit-generator function using the generator object.

12 Versioning

ART uses semantic versioning with MAJOR.MINOR.PATCH numbers that distinguish incompatible API changes, backward-compatible functionality, and bug fixes. Consistent benchmark results require reporting the MAJOR.MINOR version used.

  • Semantic versioning: ART version numbers follow the semantic-versioning format MAJOR.MINOR.PATCH.
  • Version increments: MAJOR increments indicate incompatible API changes.
  • Version increments: MINOR increments add functionality in a backwards-compatible manner.
  • Version increments: PATCH increments indicate backwards-compatible bug fixes.
  • Reproducibility: Consistent ART benchmark results can be obtained under constant MAJOR.MINOR versions, which should be reported for published experiments.
Loading 1807.01069v4…