PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective

Gabriel Resende Machado, Eugênio Silva, Ronaldo Ribeiro Goldschmidt

arXiv:2009.03728v1cs.CV

TL;DR

Image classifiers are widely used but vulnerable to adversarial images, while effective defenses and accepted explanations remain unresolved. This survey synthesizes attacks and defenses from a defender’s perspective, introduces taxonomies, examines explanations, and gives evaluation guidance. It concludes by identifying promising directions for more robust defenses and architectures.

  • Problem

    Image classifiers used in security-critical applications are vulnerable to adversarial images, while no known efficient defense or fully accepted explanation for their existence is available.

  • Method

    The paper conducts a defender-oriented review covering attacks, defenses, explanations for adversarial examples, and guidance for designing and evaluating defenses.

  • Results

    The survey introduces updated attack and image taxonomies, a novel defense taxonomy, explanations for adversarial examples, defense-evaluation guidance, and promising future research paths.

  • Takeaways & Limitations

    Researchers should consider the reviewed taxonomies and guidance when devising and evaluating defenses against adversarial attacks.

  • Takeaways & Limitations

    Adversarial training is strongly coupled to its training attack algorithm and may not resist evasions generated by different attacks without broader adversarial training data.

Abstract

from arXiv · show

Deep Learning algorithms have achieved the state-of-the-art performance for Image Classification and have been used even in security-critical applications, such as biometric recognition systems and self-driving cars. However, recent works have shown those algorithms, which can even surpass the human capabilities, are vulnerable to adversarial examples. In Computer Vision, adversarial examples are images containing subtle perturbations generated by malicious optimization algorithms in order to fool classifiers. As an attempt to mitigate these vulnerabilities, numerous countermeasures have been constantly proposed in literature. Nevertheless, devising an efficient defense mechanism has proven to be a difficult task, since many approaches have already shown to be ineffective to adaptive attackers. Thus, this self-containing paper aims to provide all readerships with a review of the latest research progress on Adversarial Machine Learning in Image Classification, however with a defender's perspective. Here, novel taxonomies for categorizing adversarial attacks and defenses are introduced and discussions about the existence of adversarial examples are provided. Further, in contrast to exisiting surveys, it is also given relevant guidance that should be taken into consideration by researchers when devising and evaluating defenses. Finally, based on the reviewed literature, it is discussed some promising paths for future research.

1 INTRODUCTION

Deep learning has become highly effective and widespread in computer vision, including security-critical applications, yet image classifiers remain vulnerable to adversarial images. This survey reviews attacks and defenses from a defender’s perspective, explains adversarial examples, and offers evaluation guidance and future research directions.

  • Deep learning and computer vision: CNNs became state-of-the-art in computer vision and are used in medical diagnostics, autonomous vehicles, surveillance, and biometric recognition.Their adoption extends into security-critical applications.
  • The adversarial vulnerability: Adversarial attacks add subtle, optimization-generated perturbations to legitimate images, often causing highly confident incorrect CNN predictions.Image-classification attacks are the paper’s focus because they are the most common in the literature.
  • The research gap: The field lacks both a known efficient solution for securing deep-learning models and fully accepted explanations for why adversarial images exist.Earlier surveys have also become somewhat outdated because of the area’s rapid activity.
  • Survey contributions: The survey updates taxonomies for adversarial images and attacks and proposes a novel taxonomy for organizing defenses.Its coverage spans technical formalisms, attacks, defenses, and countermeasures.
  • Survey contributions: It discusses explanations for adversarial examples and provides guidance for devising and evaluating defenses.The paper also identifies promising paths for future research.

2 BACKGROUND

Deep Learning and CNNs enable high-level feature learning from raw data, with CNNs serving as state-of-the-art models for image classification. The section also introduces autoencoders and GANs as important architectures used in adversarial machine learning.

  • Deep Learning models use many neural-network layers to learn high-level abstractions from raw inputs through linear and nonlinear transformations.
  • CNNs are specialized Deep Neural Networks that currently achieve state-of-the-art performance in image classification.
  • CNNs learn image features through convolution and pooling layers before fully connected layers produce class probabilities.
  • Since 2015, CNNs have surpassed human performance in the ILSVRC classification challenge.
  • Autoencoders reconstruct inputs from compressed representations and support dimensionality reduction and data generation.
  • GANs use simultaneously trained generator and discriminator networks to model training-data distributions and can contribute to adversarial defenses.

3 ADVERSARIAL IMAGES AND ATTACKS

An adversarial image is created by adding a perturbation to a legitimate image so that the classifier crosses its decision boundary and changes its output. An optimal adversarial image additionally uses perturbations imperceptible to humans while inducing an incorrect, preferably high-confidence prediction.

  • An adversarial image x′ is formed as x′ = x + δx, where δx moves legitimate image x across the classifier’s decision boundary.
  • An optimal adversarial image combines human-imperceptible perturbations with an incorrect classifier output, preferably at high confidence.

3.1 Taxonomy of Adversarial Images

The paper classifies adversarial images by perturbation scope, visibility, and measurement. These axes distinguish image-specific from universal perturbations, human visibility, and norm- or metric-based perturbation size.

  • The proposed taxonomy organizes adversarial images by perturbation scope, visibility, and measurement.
  • Perturbation Scope: Individual-scoped perturbations are generated separately for each input, whereas universal-scoped perturbations are image-agnostic and can be applied across samples.
  • Perturbation Visibility: Optimal perturbations fool models while remaining imperceptible, indistinguishable perturbations remain ineffective, and visible perturbations are detectable by humans.
  • Perturbation Measurement: p-norms measure input-space distance between legitimate and adversarial images because human visual capability is difficult to quantify.
  • Perturbation Measurement: L0 counts modified pixels, while L∞ limits the maximum per-pixel difference between corresponding image positions.
  • Perturbation Measurement: Although L0, L1, L2, and L∞ are commonly used, some studies define custom perturbation metrics.

3.2 Taxonomy of Attacks and Attackers

The attack taxonomy describes attackers by influence, knowledge, objective, and perturbation-generation approach. It distinguishes training-time poisoning from inference-time evasion, and separates attack settings such as white-box access, targeted goals, and gradient-based crafting.

  • A threat model specifies the attacker’s knowledge, goal, and conditions under which a defense must provide security guarantees.
  • Attacker’s Influence: Causative attacks corrupt training data, whereas evasive attacks craft adversarial examples during inference to induce misclassification or gather model information.
  • Attacker’s Influence: Input/output attacks query a target model and can provide the first step toward constructing a substitute model for black-box attacks.
  • Attacker’s Knowledge: White-box attacks give attackers full access to model and defense parameters and architectures, making them usually the most powerful attack type.
  • Security Violations: Adversarial attacks may violate classifier integrity, availability, or privacy, including misclassification, denial of service, or model reverse-engineering.
  • Attack Specificity: Targeted attacks seek a predetermined incorrect class, while untargeted attacks seek any change in the classifier’s output and targeted attacks usually cost more computationally.
  • Algorithm Iterations: Sequential algorithms craft perturbations in one iteration, whereas iterative algorithms use multiple iterations and incur higher computational cost.
  • Attack Approach: Attack approaches include gradient-based, transferability or score-based, decision-based, and approximation-based methods.

3.3 Algorithms for Generating Adversarial Images

The survey presents four frequently used adversarial-image generation algorithms, emphasizing their optimization strategies, perturbation limits, and computational trade-offs.

  • FGSM: FGSM creates an adversarial image in one gradient-directed step bounded by ϵ, offering low computational cost.It perturbs the legitimate image toward the gradient direction that maximizes model error.
  • BIM: BIM iteratively applies several smaller steps α while constraining the total perturbation within an attacker-defined upper bound.Its recursive updates use clipping to keep values within the permitted interval.
  • DeepFool: DeepFool repeatedly linearizes the classifier and updates the image toward the nearest decision boundary until the perturbation causes misclassification.Each update moves along an approximately optimal direction by a small step α.
  • Carlini–Wagner Attack: The Carlini–Wagner attack uses iterative gradient descent to find a small perturbation, with hyperparameter c controlling the amount required to fool the classifier.Its cost function is defined using the logits and target class.
  • Carlini–Wagner Attack: Increasing conf can produce adversarial images that fool models with higher confidence, but usually makes perturbations larger and more perceptible.The confidence parameter can yield predictions approaching 100% for an incorrect class.
  • Attack Taxonomy: The survey organizes additional attack algorithms in Table 1, excluding attacker influence and attack computation because those axes are not applicable across the listed methods.FGSM and L-BFGS are exceptions to the stated iterative-computation pattern.

4 DEFENSES AGAINST ADVERSARIAL ATTACKS

The survey classifies defenses by objective and operational approach, then reviews representative methods and their limitations against varied or adaptive attacks.

  • Defense Motivation: A robust defense against all types of adversarial images remains an open research problem because current mechanisms generally cover specific threat models.The paper attributes this difficulty to the nonlinear, non-convex optimization underlying adversarial inputs.
  • Taxonomy of Defenses: The proposed defense taxonomy uses two axes: defense objective and defense approach.This framework categorizes both what a defense seeks to achieve and how it operationally protects the model.
  • Defense Objective: Proactive defenses seek robust classification, whereas reactive defenses detect adversarial images before they reach the application classifier.Detected images may be discarded or sent to a recovery procedure.
  • Gradient Masking: Gradient masking hinders optimization-based attacks by producing smoother or otherwise unhelpful gradients, including shattered, stochastic, and exploding or vanishing gradients.These categories arise from nondifferentiability, randomization, or very deep architectures.
  • Adversarial Training: Adversarial training retrains a classifier on legitimate and adversarial images, producing a theoretically stronger model.The augmented dataset is formed by generating adversarial variants and retaining their original labels.
  • Adversarial Training: Adversarial training is coupled to its training attacks and can become computationally inefficient when many attack algorithms and perturbation levels are required.The resulting model is not guaranteed to resist evasions generated by unseen or more complex attacks.
  • Adaptive Attackers: Gradient-masking defenses can be bypassed with nondifferentiable attacks such as BPDA or SPSA, or through black-box surrogate models exploiting transferability.Surrogate models approximate the targeted model using queried outputs.

5 EXPLANATIONS FOR THE EXISTENCE OF ADVERSARIAL EXAMPLES

The survey presents competing explanations for adversarial examples, including nonlinearity, boundary tilting, high-dimensional geometry, statistical limitations, and non-robust features. It also discusses adversarial transferability across models and the absence of a universally accepted explanation.

  • Competing explanations: Adversarial examples are explained through competing hypotheses rather than one universally accepted account.The survey discusses explanations based on model behavior, data geometry, statistical settings, and image features.
  • Geometric and nonlinear hypotheses: Szegedy et al. attribute adversarial examples to nonlinear networks and low-probability pockets near the data manifold.Later work linked these pockets to deficiencies in objectives, training procedures, and limited datasets.
  • Geometric and nonlinear hypotheses: Goodfellow et al. instead describe networks as locally linear, causing small perturbations to accumulate in a common erroneous direction.Their explanation motivated the FGSM attack.
  • Geometric and nonlinear hypotheses: Tanay and Griffin explain the phenomenon through class boundaries tilted relative to the training-data manifold.Perturbing legitimate samples toward these nearby tilted boundaries can produce adversarial images.
  • High-dimensional manifold: Gilmer et al. report that correctly classified inputs can lie close to misclassified adversarial inputs, supporting vulnerability independent of training procedure.Their synthetic-data experiments also rejected the assumption that adversarial examples come from a different distribution than legitimate data.
  • Statistical features and transferability: Other accounts frame adversarial examples as an unavoidable consequence of statistical learning or as evidence that models rely on non-robust features.The survey also defines transferability as adversarial samples misleading multiple models, including architecturally different ones.

6 PRINCIPLES FOR DESIGNING AND EVALUATING DEFENSES

The survey argues that defenses must be evaluated against explicit threat models and adaptive attackers using strong, reproducible security tests. It also contrasts empirical evaluation with provable robustness bounds and highlights their generalization limitations.

  • Threat models: Defenses should explicitly define the adversarial threat model under which they claim robustness.The model should specify relevant attack and access conditions so evaluations can be restricted to stated security requirements.
  • Adaptive evaluation: Evaluations should simulate adaptive adversaries and include every attack scenario that could bypass the claimed defense.Non-adaptive-only evaluations provide limited utility and unreliable support for robustness claims.
  • Provable evaluation: Provable evaluations can guarantee lower robustness bounds, but they commonly depend on a network architecture and a specific attack-generated example set.Such proofs may not extend to other adversarial examples outside the evaluated set.
  • Sanity tests: Sanity tests should complement defense experiments by checking legitimate-sample accuracy and searching for anomalies or missed adversarial examples.Recommended tests include reporting clean accuracy and using brute-force random searches when attacks rarely succeed.
  • Sanity tests: FGSM is generally considered too weak and untrustworthy to fully test defenses, so it is usually used for sanity tests.The survey links this limitation to its attack configuration and sequential perturbation computation.
  • Reproducibility: Publishing experimental source code, pretrained models, and hyperparameters enables reviewers to reproduce and check defense evaluations.The recommendation applies to all implementation materials used in the experiments.

7 DIRECTIONS OF FUTURE WORK

The survey identifies future work around general theoretical robustness guarantees, stronger and more efficient attacks, comparative defense studies, and modular hybrid defenses. It also calls for provable explanations of adversarial examples and transferability.

  • Theoretical robustness: A key research direction is deriving theoretical robustness bounds that generalize across attacks and threat models.Current defenses are often empirical or produce bounds tied to particular evaluation settings.
  • Explanations: Provable explanations for adversarial examples and transferability could guide defenses toward the underlying weakness in learning models.The survey states that existing explanations lack universally accepted proofs.
  • Attack efficiency: Attack algorithms should become more computationally efficient, especially for time-sensitive applications and black-box settings with limited queries.The survey identifies traffic-signal recognition and oracle-query constraints as practical examples.
  • Comparative evaluation: Comparative studies across defenses could reveal promising architectures for particular threat models.The survey notes that few existing works compare security approaches systematically.
  • Hybrid defenses: Hybrid defenses could combine multiple countermeasures as separate modules selected from a repository when inputs arrive.Each module performs a security procedure associated with its defense or preprocessing method.

8 FINAL CONSIDERATIONS

The survey concludes that adversarial vulnerability threatens safety-critical computer-vision applications and motivates a defender-oriented synthesis of attacks, defenses, explanations, and research directions. Its contributions include updated attack taxonomies, a novel defense taxonomy, and guidance for designing and evaluating defenses.

  • Scope and motivation: Adversarial vulnerability threatens the use of deep learning in safety-critical image-classification and recognition applications.The survey presents this security issue as a concern for the field’s continued development.
  • Survey scope: The paper provides a comprehensive, self-contained survey of adversarial machine learning with emphasis on the defender’s perspective.Its coverage spans machine-learning basics, adversarial examples, attacks, defenses, explanations, and future work.
  • Taxonomies: The survey updates taxonomies for adversarial images and attacks and introduces a novel taxonomy for categorizing defenses.The review includes recent defenses not covered by earlier surveys.
  • Explanations: It gathers and discusses explanations for the existence and transferability of adversarial examples.The paper treats these explanations as part of its synthesis of the literature.
  • Guidance and future work: The survey discusses evaluation policies for defenders and reviewers together with promising future research paths.The stated directions include theoretical guarantees, comparative studies, and other open problems.
  • Background: The paper includes background on CNNs, architectures ranked by ILSVRC top-5 performance from 2012 to 2017, autoencoders, and GANs.These topics establish technical context for the survey’s treatment of adversarial machine learning.

A OTHER TASKS IN ADVERSARIAL MACHINE LEARNING FOR COMPUTER VISION

Adversarial Machine Learning in Computer Vision extends beyond Image Classification to Object Detection and Semantic Segmentation. The section identifies these tasks and references relevant studies addressing them.

  • Object Detection identifies semantic objects in input images, usually enclosing each detected object with a bounding box.
  • Semantic Segmentation is another mainstream Computer Vision task approached in Adversarial Machine Learning research.
  • Table 3 surveys relevant Adversarial Machine Learning studies for Object Detection and Image Segmentation.

B STANDARD DATASETS IN COMPUTER VISION

Datasets are used to evaluate Deep Learning algorithms in Adversarial Machine Learning and Computer Vision. Table 4 summarizes popular datasets, using task abbreviations for Image Classification and Recognition, Object Detection, and Semantic Segmentation.

  • Datasets are important tools for evaluating Deep Learning algorithms in Adversarial Machine Learning and Computer Vision.
  • Table 4 summarizes some of the most used datasets in Adversarial Machine Learning and Computer Vision.
  • The dataset table uses ICR, OBJ, SGS, and N/A for Image Classification and Recognition, Object Detection, Semantic Segmentation, and Not Available.
Loading 2009.03728v1…