Source-linked AI summary
DeepRobust: A PyTorch Library for Adversarial Attacks and Defenses
Yaxin Li, Wei Jin, Han Xu, Jiliang Tang
TL;DR
DeepRobust addresses the need for a comprehensive platform for studying adversarial attacks and defenses across image and graph domains. It organizes representative algorithms and supporting interfaces into an accessible PyTorch library, whose stated purpose is to enable systematic experimentation and efficient testing of new algorithms.
Problem
Existing adversarial-learning libraries mainly emphasize image attacks, while defenses and graph-structured data receive less attention.
Method
DeepRobust provides attack and defense algorithms for image and graph data, with modular interfaces and adversarial-training procedures across these domains.
Results
The library provides a comprehensive, easy-to-use platform with representative algorithms and concrete interfaces for experimenting with adversarial attacks and defenses.
Takeaways & Limitations
DeepRobust supports systematic experimentation and efficient testing of existing and new adversarial-learning algorithms.
Abstract
from arXiv · showhide
DeepRobust is a PyTorch adversarial learning library which aims to build a comprehensive and easy-to-use platform to foster this research field. It currently contains more than 10 attack algorithms and 8 defense algorithms in image domain and 9 attack algorithms and 4 defense algorithms in graph domain, under a variety of deep learning architectures. In this manual, we introduce the main contents of DeepRobust with detailed instructions. The library is kept updated and can be found at https://github.com/DSE-MSU/DeepRobust.
1 Introduction
DeepRobust addresses the need for a comprehensive platform to study adversarial attacks and defenses across image and graph domains. It is designed to support systematic experimentation and efficient testing of new algorithms.
- Motivation: Deep learning systems used in safety-critical applications remain vulnerable to small designed perturbations called adversarial perturbations.The paper illustrates adversarial examples in image and graph domains.
- Research gap: Existing libraries mainly focus on image-domain attacks, with comparatively little attention to defenses and graph-structured data.DeepRobust is positioned as broader in both defense coverage and domain coverage.
- Contribution: DeepRobust provides representative attack and defense methods for both image and graph data.
- Organization: The manual covers adversarial-learning concepts, library design, algorithm mathematics and implementation details, and usage examples.
2 Foundations of Adversarial Attacks and Defenses
The paper organizes adversarial attacks by their goals and the attacker’s access, while grouping defenses into four broad countermeasure categories. These distinctions describe how attacks operate and how defenses respond.
- Attack foundations: Adversarial attacks seek invisible perturbations that cause classifiers to make wrong predictions on otherwise well-classified data.
- Attack goals: Poisoning attacks modify training data, whereas evasion attacks craft inputs against a fixed classifier.
- Attack goals: Targeted attacks seek a specified target label, while non-targeted attacks succeed with any wrong label.
- Attacker ability: White-box, black-box, and grey-box attacks differ according to the attacker’s access to model information and query-based interaction.White-box attackers access model architecture, parameters, and gradients; black-box attackers lack inner configuration; grey-box methods combine trained generative models with black-box use.
- Defense categories: Defenses include robustness optimization, adversarial-example detection, gradient masking, and a fourth category whose description is incomplete in the supplied passage.
3 An Overview of DeepRobust
DeepRobust is organized as image and graph components with separate attack, defense, model, evaluation, configuration, and data functionality. The library is installed as a Python/PyTorch package and is intended to support modular use.
- Overview: The overview section introduces the library’s environment requirements and overall design.
- Environment: DeepRobust works with Python ≥3.6 and PyTorch ≥1.2.0 and is installed through setup.py.
- Library structure: DeepRobust contains separate image and graph components.
- Image component: The image component includes attack, defense, network-model, evaluation, and configuration modules.
- Graph component: The graph component includes targeted attacks, global attacks, defenses, and data access for public benchmark datasets.
4 Image Component
The image component exposes representative attack algorithms through a shared interface and covers optimization-, gradient-, evolutionary-, approximation-, and black-box-based methods. These attacks differ in perturbation constraints, objectives, and query or gradient requirements.
- API and coverage: The image attack package covers nine representative algorithms: LBFGS, FGSM, PGD, CW, onepixel, DeepFool, BPDA, Universal, and Nattack.
- API and coverage: The shared attack base class centralizes initialization, input type checking, parameter parsing, and attack generation, while subclasses override algorithm-specific behavior.
- Optimization-based attacks: L-BFGS and Carlini–Wagner attacks seek minimally distorted adversarial examples, with Carlini–Wagner using a loss and binary search over c to balance target success and distance.Carlini–Wagner’s loss encourages the target class to outscore other labels, followed by line search for the least-distance example.
- Gradient-based attacks: FGSM takes a one-step gradient-sign update and clips the result to an ε-neighborhood, while PGD iterates this procedure and is often used as a defense baseline.
- Boundary-based attacks: DeepFool seeks a shortest path across the decision boundary and produces less perturbation than FGSM and PGD according to the paper.The approach extends from binary classifiers to general classifiers and l_p-norm constraints.
- Other attack strategies: Universal perturbations target misclassification across most samples, onepixel constrains the l_0 norm, BPDA approximates non-differentiable layers, and Nattack optimizes a distribution in a black-box setting.
4.2 Defense Subpackage
The image defense subpackage covers three defense categories: adversarial training, gradient masking, and detection.
- Image-domain defenses are organized into adversarial training, gradient masking, and detection.
deeprobust.image.defense.base defense
The image defense base class provides shared interfaces for initialization, parameter parsing, adversarial-sample generation, training, testing, launching defenses, and loss computation. Its derived methods implement several defense strategies, including adversarial training, gradient masking, and detection.
- deeprobust.image.defense.base defense: The base class supplies interfaces for initialization, user-defined parameters, launching defenses, and calculating training loss.The loss function may be overridden according to each defense algorithm’s requirements.
- deeprobust.image.defense.base defense: Adversarial training generates adversarial samples for robust training and trains and tests the resulting adversarial model through data loaders.The model, clean data, target labels, optimizer, and epoch settings support these operations.
- Adversarial training: FGSM training generates adversarial examples during each iteration and updates model parameters with them.
- Adversarial training: Fast improves FGSM adversarial training by adding random initialization during adversarial-sample generation.The described result is a significant robustness improvement.
- Adversarial training: PGD adversarial training replaces FGSM-generated examples with PGD examples to achieve overall high performance.
- Adversarial training: YOPO accelerates PGD training by approximating derivatives of the first N −1 layers as constant, reducing repeated full backpropagation.
- Adversarial training: TRADES minimizes natural loss plus a term that reduces classifier-output distance for inputs close in input space.Each step solves an inner maximization for x′ before updating the model parameters.
- Gradient masking: Thermometer encoding discretizes each pixel value into an l-dimensional vector to mask gradient information.
5 Graph Package
The graph package organizes targeted attacks, untargeted attacks, and defenses, covering gradient-, reinforcement-learning-, poisoning-, preprocessing-, and robust-model-based methods.
- 5 Graph Package: The graph package contains targeted attack, untargeted attack, and defense components.Its targeted-attack subpackage covers FGA, Nettack, RL-S2V, IG-Attack, and RND.
- Targeted attacks: FGA greedily selects graph-structure perturbations with the largest attack-loss gradients.
- Targeted attacks: Nettack selects perturbation candidates preserving degree distribution and feature co-occurrence, then greedily modifies the highest-scoring candidates until reaching its constraint.
- Targeted attacks: RL-S2V models black-box graph attacks as a Markov Decision Process and uses Q-learning to guide edge modifications.The attacker may modify m edges to change a target node’s predicted label.
- Targeted attacks: IG-Attack uses integrated gradients to search for adversarial edges and feature perturbations in discrete graph data.It iteratively selects the edge or feature with the strongest effect on the adversarial objective.
- Untargeted attacks: Untargeted graph attacks include Metattack, PGD, Min-max, and DICE.Metattack treats graph structure as a hyperparameter and greedily selects perturbations using the meta-gradient.
- Defenses: Graph adversarial training alternates perturbation maximization and model-loss minimization, applying separately to adjacency and attribute matrices.
- Defenses: Graph defenses include Jaccard-based edge removal, truncated SVD preprocessing, and RGCN uncertainty modeling with penalties on adversarial edges or nodes.
6 Hand-on Case Studies
The case studies demonstrate image and graph workflows for training models, launching attacks, evaluating defenses, and comparing graph defense performance on perturbed data.
- Image case studies: The examples provide demo code for each method type and show how to train image models such as CNNs on MNIST.
- Image case studies: Launching an image attack requires importing an attack class, initializing a victim model, creating a test-image dataloader, and passing them to the attack method.The output is adversarial examples.
- Image case studies: Image defense examples feed a model structure and dataloader to a defense model, producing an adversarially trained model and clean- and adversarial-data performance.
- Graph case studies: The graph attack example uses a linearized GCN surrogate and untargeted Metattack to perturb the Cora citation graph.The workflow loads Cora, trains the surrogate, sets an attack budget, and evaluates the model on test nodes.
- Graph case studies: The graph defense example applies GCN-Jaccard to a Metattack-perturbed Cora graph and tests its performance.The example also sets up a GCN model for comparison on the perturbed graph.
7 Conclusion
DeepRobust aims to provide a comprehensive, easy-to-use platform for adversarial attack and defense research, with plans for broader datasets, architectures, and continued updates.
- DeepRobust is designed as a comprehensive, easy-to-use platform for adversarial attack and defense researchers.
- Future development targets larger datasets, more model architectures, and continued inclusion of newer models.
Appendix A Environment Dependencies
The appendix identifies Table 1 as the dependencies table.
- Table 1 presents the library’s dependencies.
Appendix B Structure tree
The structure tree shows separate graph and image components, including attacks, defenses, models, utilities, examples, and tests.
- Graph structure: The repository contains a dedicated graph package with black-box attacks, attacked-data handling, datasets, defenses, and examples.
- Graph structure: Graph functionality includes global attacks such as DICE, Metattack, NIPA, and random attack modules.
- Image structure: The image-side structure includes adversarial-example assets, defenses, attack evaluation, neural-network models, training scripts, and optimization utilities.
- Testing: The repository includes tests for PGD, CW, DeepFool, FGSM, LBFGS, NAttack, OnePixel, training, and other components.