PaperlayerView exact revision on arXiv

Source-linked AI summary

How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses

Joana C. Costa, Tiago Roxo, Hugo Proença, Pedro R. M. Inácio

arXiv:2305.10862v1cs.CV

TL;DR

DNNs can be misled by small perturbations, raising concerns about their use in critical applications. This survey synthesizes attacks by adversary capacity, defenses by protection strategy, Vision Transformer findings, and evaluation resources. It concludes by comparing adversarial settings and identifying open issues, including the need for faster purification strategies.

  • Problem

    DNNs are vulnerable to adversarial examples that alter predictions, challenging their use in critical applications despite strong performance across tasks.

  • Method

    The survey organizes attacks by adversary capacity, defenses into six domains, and reviews Vision Transformers, datasets, metrics, and state-of-the-art results.

  • Results

    The survey reports that ViTs show mixed adversarial behavior: some findings indicate higher robustness than ResNet-50, while fair comparisons find vulnerability comparable to CNNs and greater vulnerability to adversarially generated patches.

  • Takeaways & Limitations

    The survey identifies unresolved robustness and trust concerns for critical applications and points to close-to-real-time adversarial purification as a future direction.

Abstract

from arXiv · show

Deep Learning is currently used to perform multiple tasks, such as object recognition, face recognition, and natural language processing. However, Deep Neural Networks (DNNs) are vulnerable to perturbations that alter the network prediction (adversarial examples), raising concerns regarding its usage in critical areas, such as self-driving vehicles, malware detection, and healthcare. This paper compiles the most recent adversarial attacks, grouped by the attacker capacity, and modern defenses clustered by protection strategies. We also present the new advances regarding Vision Transformers, summarize the datasets and metrics used in the context of adversarial settings, and compare the state-of-the-art results under different attacks, finishing with the identification of open issues.

I. INTRODUCTION

Deep neural networks achieve strong performance across many tasks but can be fooled by small, often imperceptible perturbations, creating risks for critical applications. This survey organizes adversarial attacks by attacker capacity, defenses by protection domain, and reviews architectures, examples, norms, datasets, and metrics.

  • Motivation: DNNs support complex tasks including object recognition, face recognition, and natural language processing, with applications in self-driving vehicles, malware detection, and healthcare.Their impaired functioning can severely impact users in these critical areas.
  • Motivation: Small perturbations can entirely alter DNN predictions while remaining invisible or slight to humans and increasing confidence in erroneous outputs.This vulnerability complicates the use of DNNs in critical areas.
  • Adversary capacity: Attackers are classified by access to training data, architecture, network responses, or input-output pairs, with real-world attacks often limited to model responses and costly queries.The survey distinguishes four adversary information settings and notes that restricted access diminishes attack strength.
  • Survey scope: The survey groups recent attacks by adversary capacity, contrasts black-box and white-box attacks, and organizes defenses into six domains.It also describes adversarial effects in Vision Transformers.
  • Neural network architectures: CNNs apply convolutions to image matrices before a fully connected classifier, whereas ViTs divide images into ordered patches processed by a Transformer Encoder and MLP Head.The supplied figures schematically illustrate these image-classification mechanisms.
  • Adversarial examples: Adversarial examples add perturbations that cause DNN misclassification while humans correctly classify the same image.The survey formalizes perturbation construction as an optimization problem and defines the adversarial image as X* = X + δX.
  • Attack examples and norms: L-BFGS, DeepFool, and SmoothFool illustrate progressively different perturbation patterns, while L0, L2, and Lp norms constrain the noise added to images.The examples include broad image noise, localized whale-region perturbation, and smoother disturbances.

C. Vector Norms and ϵ Constraint

Adversarial attacks constrain perturbations using p-norms and an ϵ noise budget, while attack goals and attacker knowledge define additional categories. Lower- and higher-order norms emphasize different perturbation patterns and detectability.

  • Vector Norms: p-norms map vectors to positive scalar values; adversarial attacks commonly use p = 0, 2, and ∞.L0 counts non-zero elements, L2 measures Euclidean distance, and L∞ returns the largest absolute element.
  • Vector Norms: Higher p emphasizes large errors, whereas lower p emphasizes small errors, producing different perturbation patterns.Large p supports small maximal changes across many pixels, while small p encourages larger spikes in fewer pixels.
  • ϵ Constraint: ϵ controls the amount of perturbation noise added to an image and typically decreases as task difficulty increases.Reported values are ϵ = 0.1 for MNIST, ϵ = 8/255 for CIFAR-10 and CIFAR-100, and ϵ = 4/255 for ImageNet.
  • Adversary Goals and Capacity: Attack goals include confidence reduction, untargeted misclassification, and targeted classification into a specified class.These categories describe the classifier output sought by the attacker.
  • Adversary Goals and Capacity: White-box attacks use architecture or data access, whereas black-box attacks rely on oracle responses or input-output samples.The survey characterizes Confidence Reduction White-box attacks as easiest and Targeted Black-box attacks as strongest.

III. RELATED SURVEYS

Earlier surveys established common concepts, attacks, defenses, and threat models, but differed in scope and organization. This survey emphasizes datasets, metrics, state-of-the-art results, and future directions amid limited standardization.

  • Prior Survey Coverage: Prior surveys covered adversary goals and capabilities, attacks, defenses, physical-world examples, and applications across machine-learning settings.Their organizations varied by attack stage, adversary capability, underlying defense methodology, threat-model stage, or application domain.
  • Prior Survey Coverage: Some reviews focused on computer vision, while others addressed broader machine-learning security or attacks in graphs, text, and audio.The surveys therefore differ in task coverage and analytical perspective.
  • Defense Organization: Existing surveys commonly grouped defenses by modified training or input, network modification, add-on networks, or underlying methodology.These categories recur across reviews with different taxonomies and scopes.
  • Open Gaps: Datasets and metrics remain less standardized than adversary goals, capabilities, attacks, and defenses.The survey responds by analyzing datasets and metrics and consolidating state-of-the-art results.
  • This Survey: The survey provides comparison information across prior reviews and identifies datasets for further exploration and future research directions.These additions are intended to support novice researchers and guide subsequent work.

A. White-box Settings

White-box attacks exploit model information such as gradients, architecture, or data to generate adversarial examples through diverse optimization and perturbation strategies. The surveyed methods include gradient-based, boundary-based, generative, physical-world, and transfer-oriented attacks.

  • Foundations: Adversarial examples were introduced after evidence that DNNs generalize poorly near an input, motivating attacks such as L-BFGS.The survey frames this local generalization issue as the basis for adversarial-example research.
  • Gradient-Based Attacks: FGSM generates an adversarial example in one step using the model cost function, its gradient, and the perturbation radius ϵ.The perturbation amount should remain very small to support undetectability.
  • Norm-Based Attacks: C&W attacks optimize perturbations under L2, L0, and L∞ norms, targeting low distortion or fewer altered pixels depending on the norm.Its procedures include smoothing, pixel fixing, and penalty-based optimization.
  • Iterative and Boundary Attacks: Iterative methods such as PGD, DeepFool, and BIM repeatedly optimize or adjust perturbations to increase loss or cross a decision boundary.PGD uses inner maximization and outer minimization; DeepFool seeks the minimal boundary-crossing perturbation; BIM repeatedly applies FGSM.
  • Alternative Strategies: Other white-box approaches use GANs, momentum, saliency, smoothness, physical adaptation, feature importance, or model ensembles to improve realism or transferability.These methods respectively include AdvGAN, MI-FGSM, JSM, SmoothFool, AdvCam, FIA, and MGAA.

B. Universal Adversarial Perturbations

Universal adversarial perturbations are image-agnostic vectors that can misclassify many inputs and generalize across networks. Related methods extend this idea through generative perturbations, black-box search, and standardized attack ensembles.

  • Universal Perturbations: UAPs are image-agnostic perturbations that cause misclassification with high probability across images and can generalize across neural networks.They are obtained by searching for a vector that fools almost all images drawn from a distribution.
  • Universal Perturbations: The UAP optimization updates a universal vector using the minimal per-image perturbation needed to fool the classifier.Here, Δv_i is the minimal perturbation, v is the universal perturbation, and x_i is the original image.
  • Generative Extensions: Universal Adversarial Networks learn generators whose outputs perturb arbitrary inputs and can outperform original UAPs.Their training set is inspired by UAPs and the generator produces perturbations for a given input.
  • Black-Box Settings: Black-box threat models include query-limited, partial-information, and label-only settings, motivating gradient estimation, score-based optimization, and noise robustness.The methods use Natural Evolutionary Strategies, top-k probabilities, or predicted labels depending on available information.
  • Auto-Attack: Auto-Attack combines APGD-CE, APGD-DLR, FAB, and Square Attack into a parameter-free, computationally affordable, user-independent robustness test.Its components cover white-box, norm-minimizing, and query-efficient black-box strategies without relying on a single attack type.

A. Adversarial Training

Adversarial training strengthens models by training on adversarial and clean examples, but its effectiveness depends on attack construction, computational cost, and the robustness–accuracy trade-off. The surveyed variants target these constraints through faster generation, specialized weighting, curriculum strategies, and representation-aware objectives.

  • Core approach: Adversarial training mixes clean and adversarial examples to regularize networks and can be applied across neural-network architectures.The adversarial examples may be generated with attacks such as PGD, FGSM, or C&W.
  • Known limitations: One-step adversarial training can produce label leaking, with adversarial-image accuracy exceeding clean-image accuracy when ground-truth labels are used.This finding suggests avoiding ground-truth labels when generating adversarial examples for training.
  • Known limitations: Fast single-step training can converge to a degenerate global minimum, leaving models vulnerable to black-box attacks.Ensemble Adversarial Training addresses this vulnerability by using examples crafted on other static pretrained models.
  • Robustness and accuracy: Several variants target the robustness–accuracy trade-off by adjusting losses, perturbation difficulty, or example weights during training.TRADES bounds robust error using natural and boundary error, while SAT, GI-AT, and HAT modify training difficulty or weighting to improve the trade-off.
  • Efficiency: Adversarial training has high computational cost on large datasets, motivating methods that reuse gradients or restrict propagation to accelerate example generation.Free-AT reuses gradient information, while YOPO limits forward and backpropagation to the first layer during each update.
  • Perturbation scope: Adversarial training can also address larger or more realistic perturbations by aligning predictions with an Oracle, using perceptual similarity to generate Oracle-Invariant attacks.OA-AT targets settings where perturbations are visibly larger than the usual imperceptibility constraint.

C. Use of Supplementary Networks

Supplementary-network defenses detect, reconstruct, or transform adversarial inputs before or alongside classification. The surveyed methods use detectors, denoisers, generative models, graph representations, and local implicit functions, with some approaches designed to handle unseen attacks.

  • Detection and reconstruction: MagNet detects adversarial examples using reconstruction error for strongly perturbed inputs and probability divergence for slightly perturbed inputs.Its design addresses both examples far from and close to the classifier decision boundary.
  • Detection and reconstruction: Adversary Detection Network classifies inputs as clean or adversarial, but its effectiveness is strongly correlated with the datasets and classification networks used.This dependence constrains how broadly its detection behavior can transfer.
  • Detection and reconstruction: Feature Squeezing detects adversarial examples by comparing predictions from a standard network with those from a squeezed input representation.The survey reports high detection accuracy with few false positives.
  • Generative defenses: Generative defenses such as Defense-GAN and ER-Classifier map adversarial inputs toward the benign image distribution.Defense-GAN learns original-data representations with Wasserstein GANs, while ER-Classifier uses encoder and discriminator components.
  • Generalization: Several supplementary defenses aim to generalize beyond their training attacks, including CAFD and Detector Graph.CAFD matches class-activation features, while Detector Graph uses latent neighborhood graphs and graph neural networks.
  • Input transformation: DISCO removes adversarial perturbations through localized manifold projections that predict clean RGB values from per-pixel deep features.The method receives an adversarial image and a query pixel location.

D. Change Network Architecture

Changing network architecture or preprocessing can improve robustness by suppressing adversarial features, modifying decision boundaries, or mapping inputs toward natural-image spaces. The surveyed approaches range from searched connectivity patterns to feature denoising and input randomization.

  • Architecture search: Neural Architecture Search found that densely connected patterns and convolution operations on direct connection edges improve adversarial robustness in RobNets.RobNets combine these architecture patterns into more robust networks.
  • Feature processing: Feature Denoising inserts non-local means, bilateral, mean, or median filtering followed by 1x1 Convolution and an identity skip connection into intermediate CNN layers.The blocks are designed to denoise intermediate representations rather than only the input image.
  • Input transformation: Input Random adds random resizing and random zero padding at the beginning of the classifier.Both the resized dimensions and padding placement are randomized.
  • Decision boundaries: Controlling Neural Level Sets modifies decision boundaries by relating neural-level-set sample positions to network parameters.The relation is incorporated through an additional fixed linear layer and a corresponding loss function.
  • Input transformation: Sparse Transformation Layer maps inputs into a low-dimensional quasinatural image space that approximates the natural image space and removes perturbations.The layer is placed between the input image and the network’s first layer and is attack-agnostic.
  • Feature learning: Normalization techniques including Batch Normalization were reported to increase vulnerability, motivating a framework that learns robust features before non-robust features.The surveyed finding specifically contrasts robust features with features learned when using BN.

E. Perform Network Validation

Network validation and robustness studies examine how data processing, implementation choices, testing coverage, and training design affect adversarial behavior. The surveyed evidence includes input-compression effects, formal and coverage-based testing, augmentation strategies, and model configurations associated with robust accuracy.

  • Data and perturbations: FGSM perturbation magnitude affects classification behavior nonlinearly: smaller perturbations can substantially reverse the classification drop, whereas the effect disappears as magnitude increases.The result concerns the impact of JPEG-related evaluation settings on classification performance.
  • Formal validation: Formal safety verification treats a decision as safe when finite input transformations do not change the model decision, checking layers individually with SMT.The method verifies stability over a finite transformation space.
  • Coverage-based testing: DeepXplore uses neuron coverage and multiple DNNs as cross-referencing oracles to search for inputs that trigger divergent behaviors.Its gradient-based search jointly seeks differing behaviors and high neuron coverage.
  • Coverage-based testing: DeepGauge evaluates DNN resilience with multi-granularity criteria spanning neuron- and layer-level coverage, including primary-function and corner-case behaviors.The approach targets a multi-faceted testing set.
  • Training design: Early stopping combined with semi-supervised data augmentation outperforms cutout and mixup for substantial improvements under the surveyed Overfit Reduction approach.The survey reports that cutout and mixup can fail to prevent overfitting or over-regularize models.
  • Training design: Weight decay, attack early stopping, and smooth activations materially affect robustness, while weight averaging with data augmentation can increase it.The Bag of Tricks study reports that smooth activation benefits lower-capacity models, and FDA enhances robustness with spatial composition.
  • Purification: Adversarial purification with DDPM adds noise to an adversarial image and iteratively removes it to produce a purified image.IRUGD uses generative models trained on original images and reports DDPM as most closely resembling real data among the evaluated diffusion models.
  • Robust accuracy: Larger models with Swish/SiLU activations and model-weight averaging can reliably achieve state-of-the-art robust accuracy.This conclusion comes from a systematic study of losses, model sizes, activations, unlabeled data, and related training choices.

F. Adversarial Purification

Adversarial purification uses generative models to remove perturbations, with diffusion models emerging as effective tools for recovery and robust training. The section also shows that Vision Transformers remain vulnerable, with robustness depending on attack type, evaluation fairness, and patch-level perturbations.

  • F. Adversarial Purification: Adversarial purification removes perturbations using generative models.
  • F. Adversarial Purification: DDPMs closely resemble real data and improve adversarial training when used to generate additional examples.
  • F. Adversarial Purification: DiffPure recovers clean images from adversarial inputs through a reverse generative process, while DensePure aggregates predictions from multiple denoised samples.
  • VI. Adversarial Effects on Vision Transformers: ViTs are susceptible to adversarial patch perturbations but initially demonstrate almost double the robustness of ResNet-50.
  • VI. Adversarial Effects on Vision Transformers: ViT robustness varies by attack: it increases against FGSM, PGD, MIM, and C&W relative to ResNet, but not against C&W and MI-FGSM.
  • VI. Adversarial Effects on Vision Transformers: Fairer training comparisons indicate that ViTs can be as vulnerable as CNNs, while adversarially generated patches can make ViTs more vulnerable than CNNs.

B. CIFAR-10 and CIFAR-100

The survey describes datasets spanning handwritten digits, object categories, traffic signs, and large-scale ImageNet variants for adversarial evaluation. CIFAR-100 is harder than CIFAR-10 and has been less commonly used, while ImageNet-derived benchmarks target filtered examples, common corruptions, perturbation sequences, and color distortions.

  • B. CIFAR-10 and CIFAR-100: CIFAR-10 contains ten classes with 50,000 training and 10,000 test images, whereas CIFAR-100 contains 100 classes with 600 images per class.
  • B. CIFAR-10 and CIFAR-100: CIFAR-100 is harder than CIFAR-10 because its classes are grouped into 20 superclasses across different contexts and environments.
  • B. CIFAR-10 and CIFAR-100: CIFAR-100 has been excluded from many adversarial-example studies because models trained on CIFAR-10 already demonstrate unsatisfactory results.
  • C. Street View Datasets: SVHN contains centered colored digit images with ten classes, while GTSRB contains 43 traffic-sign classes and realistic scenarios.
  • ImageNet Benchmarks: ImageNet-A contains real-world adversarially filtered images, and ImageNet-C and ImageNet-P evaluate robustness to standardized corruptions and perturbation sequences.
  • ImageNet Benchmarks: ImageNet-COLORDISTORT evaluates color distortions across RGB, HSV, CIELAB, and YCbCr color spaces.

VIII. METRICS AND STATE-OF-THE-ART RESULTS

Adversarial evaluation uses complementary metrics for classification performance, attack success, perturbation magnitude, and robustness under transformations. The survey defines clean and robust accuracy, average robustness, and destruction rate, and reports comparative results on CIFAR-10 and CIFAR-100 under standardized attacks.

  • Metrics: Fooling rate measures the percentage of adversarial examples that cause model misclassification, while destruction rate measures perturbation effects and average robustness measures resistance to adversarial examples.
  • Metrics: Accuracy counts correctly predicted samples as (TP + TN) / (TP + TN + FP + FN), with clean accuracy for original images and robust accuracy for adversarial images.
  • Metrics: Average robustness averages the estimated minimal perturbation norm relative to the input norm across the test set.
  • Metrics: Destruction rate evaluates how arbitrary transformations affect adversarial images by comparing transformed adversarial predictions with the original image labels.

B. Defense Mechanisms Robustness

The survey compares defense robustness under PGD and Auto-Attack, identifies the strongest approaches, and outlines evaluation gaps across datasets, metrics, attack settings, and architectures.

  • Robustness evaluation: Accuracy is reported as Clean Accuracy on original data and Robust Accuracy on adversarially perturbed data, with PGD and Auto-Attack used for evaluation.Auto-Attack has gained interest over PGD for assessing robustness in white- and black-box settings.
  • Robustness evaluation: Auto-Attack evaluations identify supplementary networks such as DISCO as the most resilient approach, followed by SODEF and STL.The comparison covers CIFAR-10, CIFAR-100, and ImageNet in white- and black-box settings.
  • Defense strategies: Adversarial Training demonstrates the best defense results and is recommended as a requirement when evaluating attacks and defenses.
  • Datasets and metrics: MNIST and CIFAR-10 are saturated, while CIFAR-100, ImageNet, GTSRB, SVHN, and Fashion-MNIST warrant further evaluation.CIFAR-100 and ImageNet defenses do not achieve state-of-the-art clean accuracy, reported as 91% and 95%, respectively.
  • Datasets and metrics: Current evaluations rely mainly on accuracy and standardized ϵ values, motivating metrics that quantify perturbation magnitude across datasets.Such a process would support extending adversarial attacks to datasets without standardized ϵ values.
  • Attack settings: White-box attacks receive greater attention despite limited real-world feasibility, motivating more black-box and physical-attack evaluation.The survey also anticipates increased gray-box attacks when users can partially control training data or network access.
Loading 2305.10862v1…