PaperlayerView exact revision on arXiv

Source-linked AI summary

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Naveed Akhtar, Ajmal Mian

arXiv:1801.00553v3cs.CV

TL;DR

Deep neural networks can confidently change their outputs after subtle input perturbations, raising concerns for Computer Vision systems used in practical settings. This paper surveys attack methods, evidence of vulnerability, real-world evaluations, defenses, and broader research directions. The reviewed literature indicates that adversarial attacks affect diverse models and tasks, including practical systems, while the reasons for vulnerability remain insufficiently aligned and investigated.

  • Problem

    Deep neural networks achieve high accuracy but remain vulnerable to subtle perturbations that can completely change their outputs, creating a significant concern for Computer Vision applications.

  • Method

    The paper provides a comprehensive literature survey covering attack methods, real-world conditions, analyses of attack existence, defenses, and broader research directions.

  • Results

    The reviewed literature documents adversarial vulnerability across image classification, segmentation, detection, robotics, and visual question answering, including attacks robust to physical conditions.

  • Takeaways & Limitations

    Adversarial attacks are a real threat to deep learning in Computer Vision, motivating continued study of attacks, defenses, and practical conditions.

  • Takeaways & Limitations

    The literature offers varied, often poorly aligned explanations for adversarial vulnerability, requiring more systematic investigation.

Abstract

from arXiv · show

Deep learning is at the heart of the current rise of machine learning and artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has lead to a large influx of contributions in this direction. This article presents the first comprehensive survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, we draw on the literature to provide a broader outlook of the research direction.

1 INTRODUCTION

Deep learning has become central to Computer Vision and is entering safety- and security-critical applications, but subtle adversarial perturbations can make accurate networks confidently misclassify images. The survey reviews attacks, real-world evaluations, defenses, and broader research directions.

  • Deep learning now supports challenging Computer Vision applications, including self-driving cars, surveillance, drones, robotics, and voice command recognition.
  • Small, nearly imperceptible image perturbations can completely change a neural network’s prediction while retaining high confidence.
  • Universal perturbations can fool a network on any image with high probability while remaining quasi-imperceptible to human vision.
  • The survey presents a comprehensive review of adversarial attacks, including laboratory methods, real-world conditions, analyses of attack existence, and defenses.

2 DEFINITIONS OF TERMS

The survey establishes shared terminology for adversarial attacks, their perturbations and examples, threat models, objectives, detection, and training-based defenses.

  • An adversarial example is a clean image intentionally modified to confuse or fool a machine-learning technique.
  • An adversarial perturbation is the noise added to a clean image to create an adversarial example.
  • Black-box attacks generate test-time adversarial examples without knowing the target model’s parameters.
  • A detector identifies whether an image is adversarial, while fooling rate measures the percentage of images whose predicted labels change after perturbation.
  • One-shot methods compute a perturbation in one step, whereas iterative methods repeat the computation and are often more expensive.
  • Targeted attacks seek a specific false label, whereas non-targeted attacks require only an incorrect prediction.

3 ADVERSARIAL ATTACKS

The survey reviews adversarial attack methods for common Computer Vision tasks, emphasizing technical understanding of representative approaches and distinguishing classification attacks from attacks beyond classification.

  • The reviewed attack literature mainly studies laboratory settings using standard datasets and typical Computer Vision tasks such as recognition.
  • The section is organized primarily chronologically, with technical details for popular and representative methods and brief treatment of other approaches.
  • Classification and recognition attacks are reviewed separately from methods targeting Computer Vision tasks beyond classification.

3.1 Attacks for classification

Classification attacks construct small perturbations that alter model predictions, ranging from optimization-based methods to efficient gradient-based procedures. These perturbations can remain visually similar to clean images and sometimes transfer across networks.

  • 3.1.1 Box-constrained L-BFGS: Szegedy et al.’s optimization seeks a small perturbation that changes the classifier’s output while keeping the perturbed image within [0, 1]^m.
  • 3.1.1 Box-constrained L-BFGS: Box-constrained L-BFGS minimizes a weighted perturbation norm and classifier loss, then adds the resulting perturbation to create an adversarial example.
  • 3.1.1 Box-constrained L-BFGS: The generated adversarial images can fool neural networks while appearing similar to clean images to human vision.
  • 3.1.1 Box-constrained L-BFGS: Perturbations computed for one neural network were also observed to fool multiple networks, revealing a transferability of adversarial examples.
  • 3.1.2 Fast Gradient Sign Method (FGSM): FGSM computes a one-step perturbation from the cost-function gradient, using its sign and a small ϵ to restrict the perturbation’s ℓ∞-norm.
  • 3.1.2 Fast Gradient Sign Method (FGSM): FGSM exploits the linearity of deep network models in high-dimensional space, according to Goodfellow et al.’s hypothesis.

3.1.3 Basic & Least-Likely-Class Iterative Methods

Iterative methods repeatedly adjust adversarial perturbations, while least-likely-class attacks target a specific predicted class. One-pixel attacks show that black-box methods can fool classifiers by changing extremely few pixels.

  • Basic Iterative Method: Basic Iterative Method repeatedly takes small loss-increasing steps while clipping the perturbed image within the allowed bound.BIM is equivalent to the ℓ∞ version of Projected Gradient Descent.
  • Least-Likely-Class Iterative Method: Iterative Least-likely Class Method replaces the original label with the classifier’s least-likely target label to produce targeted adversarial examples.The method extends BIM and has seriously affected Inception v3 even for very small ϵ values.
  • One Pixel Attack: 70.97% of tested images were fooled across three network models by changing just one pixel per image.The average confidence assigned to the wrong labels was 97.47%.
  • One Pixel Attack: Differential evolution generates these one-pixel attacks using only probabilistic labels from the targeted model, without network parameters or gradients.Candidate pixels are iteratively modified through parent-child fitness comparisons.

3.1.6 Carlini and Wagner Attacks (C&W)

The reviewed attacks constrain perturbation norms to make adversarial examples quasi-imperceptible and expose weaknesses in defensive distillation. Universal perturbations extend this vulnerability across images and, to some extent, across models.

  • Carlini and Wagner Attacks: Carlini–Wagner attacks constrain ℓ2, ℓ∞, or ℓ0 perturbation norms while producing quasi-imperceptible adversarial examples.The three attacks were introduced against defensive distillation.
  • Carlini and Wagner Attacks: Defensive distillation almost completely fails against the Carlini–Wagner attacks on targeted networks.Examples generated on unsecured networks also transfer well to distilled networks, enabling black-box attacks.
  • Universal Adversarial Perturbations: Universal perturbations fool any image with high probability, unlike image-specific methods such as FGSM, ILCM, and DeepFool.They remain quasi-imperceptible to human vision.
  • Universal Adversarial Perturbations: Perturbations with ℓ2 and ℓ∞ norms bounded by 4% of the respective image norms achieved fooling ratios around 0.8 or more.The iterative construction is related to DeepFool’s gradual movement toward a decision boundary.
  • Universal Adversarial Perturbations: Universal perturbations generalized across different networks, especially similar architectures, and achieved δ ≥0.5 using around 2,000 training images.This cross-image and cross-network behavior led the authors to call them, to some extent, doubly universal.
  • UPSET and ANGRI: UPSET learns n image-agnostic perturbations for n target classes, while ANGRI generates image-specific targeted perturbations.Both methods were reported to achieve high fooling ratios on MNIST and CIFAR-10.

3.1.10 Houdini

Houdini addresses attacks whose task losses are not straightforwardly differentiable, while related learned generators produce perceptually similar but misclassified images. The survey situates these methods within a broader, active, and non-exhaustive attack literature.

  • Houdini: Houdini generates adversarial examples tailored to task losses that may not be amenable to ordinary gradient-based optimization.The survey gives speech-recognition word-error rate as an example of a difficult task loss.
  • Adversarial Transformation Networks: Adversarial Transformation Networks generate examples by jointly preserving perceptual similarity and altering the targeted network’s prediction.They use feed-forward networks trained against targeted networks or network sets.
  • Black-Box Learned Attacks: Attacker networks reduced MNIST classification accuracy from 99.4% to 0.77% and CIFAR-10 accuracy from 91.4% to 6.8%.The generated examples remained perceptually indistinguishable from clean images and were evaluated in black-box attacks.
  • Scope and Resources: The review covers popular and representative attack directions but does not claim to be exhaustive because the research area is highly active.It also notes that many implementations and toolboxes are publicly available.

3.2 Attacks beyond classification/recognition

Adversarial attacks extend beyond image classification to generative models, recurrent networks, reinforcement-learning policies, semantic segmentation, and object detection. Across these settings, reviewed methods demonstrate that diverse deep-learning outputs can be manipulated by crafted inputs.

  • Generative Models: Attacks on autoencoders, VAEs, and VAE-GANs can drive generative models to produce outputs very different from those intended for the input.The survey presents this as support for adversarial examples as a general phenomenon for current deep models.
  • Recurrent Networks: Adversarial input sequences can fool recurrent neural networks, including by adapting feed-forward attacks such as FGSM.The reviewed work demonstrates successful attacks on RNNs.
  • Reinforcement Learning: Strategically timed attacks minimize reinforcement-learning rewards by perturbing only selected time steps, while enchanting attacks lure agents toward attacker-chosen behavior.FGSM-based perturbations also significantly degraded trained policies, including in black-box scenarios.
  • Segmentation and Detection: Image-agnostic perturbations significantly corrupt semantic segmentation and can remove a specific class, such as pedestrians, while leaving most segmentation unchanged.These perturbations generalized to unseen validation images with high probability.
  • Segmentation and Detection: Dense Adversary Generation treats segmentation and detection as classification over pixels, receptive fields, or object proposals.Its generated perturbations successfully fooled targeted networks and generalized across network models.

4 ATTACKS IN THE REAL WORLD

The survey shows that adversarial attacks extend beyond laboratory image classification into physical-world, face-attribute, cyberspace, robotic-vision, and visual-question-answering settings. These attacks can alter predictions under practical conditions, including changing facial attributes, fooling road-sign systems, and misleading models across viewpoints.

  • Attacks on Face Attributes: Adversarial attacks have been studied for face attributes, including changing lipstick labels and gender while preserving biometric utility for face matching.Fast Flipping Attribute attacks found robustness varies substantially across facial attributes.
  • Attacks in Physical Conditions: Physical attacks can survive camera capture and changing viewing conditions, including distance, angle, and resolution.Printed adversarial images fooled a mobile-camera classifier, while RP2 achieved high fooling ratios in practical road-sign driveby settings.
  • Road Sign Attack: Road-sign attacks achieved a 100% fooling rate on all shown images for the LISA-CNN classifier.The evaluated examples report camera distance and angle, and the classifier was trained on the LISA road-sign dataset.
  • Generic Adversarial 3D Objects: Expectation Over Transformation constructs 3D-printed objects that fool neural networks across a wide variety of angles and viewpoints.An EOT-perturbed turtle was classified as a rifle, demonstrating adversarial behavior for physical objects.
  • Cyberspace Attacks: Adversarial attacks also target outsourced and remotely hosted models, including BadNets and substitute-model attacks against commercial classifiers.Substitute-network attacks caused targeted networks from MetaMind, Amazon, and Google to misclassify 84.24%, 96.19%, and 88.94% of adversarial examples, respectively.
  • Robotic Vision & Visual QA Attacks: Attacks extend to robots and visual question answering, with adversarial examples transferable between commonly used VQA architectures.The reviewed work describes these vulnerabilities as threats to models beyond image classification.

5 EXISTENCE OF ADVERSARIAL EXAMPLES

The survey reviews competing explanations for why adversarial examples exist, including geometry, curvature, uncertainty, decision-boundary structure, and high-dimensional adversarial subspaces. Although attacks can require very small image changes, the literature does not yet agree on their underlying cause.

  • Explanations for Existence: Researchers propose multiple, partially conflicting explanations for adversarial examples, including linearity, nonlinear decision boundaries, flatness, curvature, and network flexibility.The survey notes that these viewpoints often align with local empirical observations but fall short of generalization.
  • Open Questions: The survey concludes that current literature lacks consensus on why adversarial examples exist, despite examples formed by modifying as little as one pixel.The disagreement concerns the mechanisms underlying adversarial examples rather than their observed occurrence.
  • Robustness Limits: Fundamental robustness limits have been studied using class distinguishability measures for linear and nonlinear classifiers.Fawzi et al. define distinguishability through distances between class means or second-order-moment matrices, depending on the classifier type.
  • Adversarial Subspaces: Adversarial examples may occupy contiguous high-dimensional regions whose intersections across classifiers help explain transferability.Tramer et al. estimate example dimensionality at approximately 25 in one reported analysis.
  • Decision-Boundary Geometry: Adversarial examples can arise when classification boundaries lie close to the data manifold, described through boundary tilting and deviation angle.Tanay and Griffin connect adversarial strength to the deviation angle between the data sub-manifold and classification boundary.
  • Alternative Mechanisms: Other accounts attribute adversarial vulnerability to predictive uncertainty, evolutionary stalling of decision boundaries, or geometric correlations shared across data points.These proposals respectively emphasize logit-difference statistics, correctly classified training samples near boundaries, and common directions associated with classifier boundaries.

6 DEFENSES AGAINST ADVERSARIAL ATTACKS

The survey organizes defenses into modified training or inputs, network modifications, and external model add-ons, distinguishing complete defense from detection-only approaches. Reviewed methods include adversarial training, data transformations, gradient and activation-based regularization, distillation, nonlinear architectures, and Lipschitz control, but stronger attacks can defeat some defenses.

  • Defenses follow three directions: modifying training or inputs, modifying networks, and adding external models during testing.
  • Complete defenses restore acceptable performance on adversarial examples, whereas detection-only methods identify attacks without necessarily preserving the original task.
  • Training and input modification: Adversarial training improves robustness but requires strong attacks, expressive architectures, and increased training or data size.
  • Limitations: Adversarial training can improve robustness while still allowing effective adversarial examples to be computed, and stronger attacks have emerged against earlier defenses.
  • Training and input modification: Input defenses include JPG compression, PCA compression, foveation, random resizing, random padding, data transformation, and augmentation, with compression and spatial changes sometimes harming effectiveness or accuracy.
  • Network modification: Network-based defenses include contractive networks, gradient regularization, adversarial loss minimization, distillation, nonlinear activations, and Parseval networks controlling layer-wise Lipschitz behavior.

7 OUTLOOK OF THE RESEARCH DIRECTION

The survey finds that adversarial attacks are a real and broadly applicable threat, with examples transferring across models and attacks exploiting multiple neural-network types and vision tasks. It also concludes that the reasons for vulnerability remain unsettled and that defenses require evaluation against counter-counter measures.

  • The reviewed literature indicates that adversarial attacks can severely degrade deep-learning performance across multiple Computer Vision tasks and in the physical world.
  • Successful attacks affect MLPs, CNNs, and RNNs across recognition, segmentation, and detection, indicating a general vulnerability phenomenon.
  • Adversarial examples often transfer between neural networks, especially when their architectures are similar, enabling black-box attacks.
  • The literature presents varied, poorly aligned explanations for adversarial vulnerability, motivating systematic investigation of its causes.
  • Linearity promotes vulnerability, but the survey concludes that it is not the only reason deep networks can be fooled by cheap analytical perturbations.
  • Defended models can be attacked again with counter-counter measures, so new defenses should estimate robustness against such responses.
  • Research on adversarial attacks and defenses has become highly active, with most reviewed literature appearing within the preceding two years.

8 CONCLUSION

The survey finds that adversarial attacks are a real practical threat to computer-vision deep learning, including in physical-world settings. It reviews attacks and defenses while noting prospects for improved robustness.

  • The literature shows that subtle input perturbations can completely change deep neural networks’ outputs despite their high task accuracy.
  • The survey reviews influential research on adversarial attacks and defenses for deep learning in Computer Vision.
  • Adversarial attacks threaten deep learning in practice, especially in safety- and security-critical applications.
  • Existing research demonstrates that deep learning can be effectively attacked both in cyberspace and in the physical world.
  • The survey suggests that deep learning may develop considerable robustness against adversarial attacks as research activity continues.
Loading 1801.00553v3…