PaperlayerView exact revision on arXiv

Source-linked AI summary

Physical Adversarial Attack meets Computer Vision: A Decade Survey

Hui Wei, Hao Tang, Xuemei Jia, Zhixiang Wang, Hanxun Yu, Zhubo Li, Shin'ichi Satoh, Luc Van Gool, Zheng Wang

arXiv:2209.15179v4cs.CV

TL;DR

Physical adversarial attacks threaten the reliability and security of DNN-based vision systems, while their real-world methodologies and performance lack systematic synthesis and evaluation. This survey distills a four-step workflow, introduces adversarial medium, and evaluates attacks across vision tasks using hiPAA’s six perspectives.

  • Problem

    Physical adversarial attacks threaten DNN-based vision systems’ reliability and security, but their real-world methodologies and performance lack systematic synthesis and evaluation.

  • Method

    The survey distills a four-step workflow, introduces adversarial medium, and evaluates physical attacks with hiPAA across six perspectives.

  • Results

    Across classification, detection, re-identification, and niche tasks, the survey provides comparative assessment using six hiPAA performance perspectives.

  • Takeaways & Limitations

    The survey frames adversarial mediums as central to physical attacks and provides a common basis for comparing methods across computer-vision tasks.

  • Takeaways & Limitations

    hiPPA’s fixed weighting may not reflect the differing priorities of specific application scenarios.

Abstract

from arXiv · show

Despite the impressive achievements of Deep Neural Networks (DNNs) in computer vision, their vulnerability to adversarial attacks remains a critical concern. Extensive research has demonstrated that incorporating sophisticated perturbations into input images can lead to a catastrophic degradation in DNNs' performance. This perplexing phenomenon not only exists in the digital space but also in the physical world. Consequently, it becomes imperative to evaluate the security of DNNs-based systems to ensure their safe deployment in real-world scenarios, particularly in security-sensitive applications. To facilitate a profound understanding of this topic, this paper presents a comprehensive overview of physical adversarial attacks. Firstly, we distill four general steps for launching physical adversarial attacks. Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world. These artifacts influence each step. To denote them, we introduce a new term: adversarial medium. Then, we take the first step to systematically evaluate the performance of physical adversarial attacks, taking the adversarial medium as a first attempt. Our proposed evaluation metric, hiPAA, comprises six perspectives: Effectiveness, Stealthiness, Robustness, Practicability, Aesthetics, and Economics. We also provide comparative results across task categories, together with insightful observations and suggestions for future research directions.

1 INTRODUCTION

The paper surveys physical adversarial attacks by organizing them into a four-step workflow and highlighting adversarial media as artifacts shaping the attack process. It addresses the lack of unified evaluation by systematically comparing existing methods and discussing future research directions.

  • The paper identifies the absence of unified evaluation and comparison criteria in prior physical-adversarial-attack surveys, motivating systematic performance evaluation.
  • Physical adversarial attacks follow four steps: digital perturbation generation, physical adversarial-medium manufacturing, threat-image capturing, and attacking.The workflow spans digital perturbation design, physical artifact production, real-world capture, and model attack execution.
  • The paper introduces “adversarial medium” for tangible artifacts carrying perturbations and shows that these artifacts influence perturbation form, manufacturing, and real-world applications.
  • It comprehensively reviews existing methodologies to abstract a general workflow for launching physical adversarial attacks.
  • The survey compares existing attack methods comprehensively and examines their limitations, challenges, and potential directions from a real-world application perspective.

2 PRELIMINARIES

This section introduces computer vision and DNN-based prediction, formalizes adversarial attacks as input-only perturbations that induce incorrect predictions, and defines physical adversarial attacks through a four-step process. It also distinguishes adversarial attacks from backdoor and poisoning attacks and clarifies the survey’s scope.

  • Computer Vision: Computer vision enables machines to perceive and understand the physical world, while DNN-based models achieve strong performance across tasks including classification, segmentation, and detection.AlexNet’s ILSVRC victory helped promote DNN applications in computer vision.
  • Adversarial Attacks: Adversarial attacks modify only input data, representing an adversarial sample as x′ = x + δ that causes y′ = fθ(x′) to differ from the model’s original prediction.Perturbations are typically constrained by intensity, size, and the adversarial medium.
  • Attack Taxonomy: Adversarial attacks are categorized by knowledge as white-box or black-box and by goals as targeted or untargeted.White-box attackers access model details, targeted attacks seek specified labels, and untargeted attacks seek any incorrect label.
  • Physical Adversarial Attacks: Physical adversarial attacks deploy tangible perturbations through four steps: generation, medium manufacturing, threat-image capturing, and attacking.Unlike digital attacks, physical perturbations must be sufficiently intense for real-world sensors to detect them.
  • Attack Taxonomy: Adversarial, backdoor, and poisoning attacks all seek incorrect predictions but differ fundamentally in implementation, with adversarial attacks altering neither the model nor training data.The lifecycle overview places adversarial attacks during testing or deployment, whereas backdoor and poisoning attacks involve other lifecycle stages.
  • Scope: Physical backdoor attacks exist, but they are excluded because the survey focuses specifically on physical adversarial attacks.The scope distinction is explicit even though physically realizable backdoor attacks have been proposed.

3 ADVERSARIAL MEDIUMS

The section introduces adversarial medium as the physical entity required to carry specially designed perturbations in physical adversarial attacks. It organizes existing methods by medium and distinguishes stickers from patches by application method and covered area.

  • 3 ADVERSARIAL MEDIUMS: Adversarial medium denotes the physical entity that carries specially designed perturbations in every physical adversarial attack method.The concept is introduced because all physical attack methods require such an entity.
  • 3 ADVERSARIAL MEDIUMS: The paper identifies nine adversarial mediums and organizes surveyed physical attack methods by medium and chronology.Tables 3–5 compile the methods, while Table 2 enumerates the mediums.
  • 3 ADVERSARIAL MEDIUMS: Stickers attach to host objects and may cover them entirely, whereas patches can be stuck or hung and occupy only a small area.The distinction concerns both application method and relative covered area.

4 EVALUATION METRIC

The section introduces hiPAA, a comprehensive metric for evaluating physical adversarial attacks across six dimensions, addressing inconsistent case-by-case assessments. It also describes dimension-specific evaluation procedures and notes limitations from scenario-dependent weighting and cross-task comparability.

  • Motivation: The evaluation addresses inconsistent prior assessments caused by costly adversarial-medium production, manufacturing variability, and difficulty standardizing real-world conditions.These factors make physical attack results difficult to replicate and compare consistently.
  • 4 EVALUATION METRIC: hiPAA evaluates physical adversarial attacks across Effectiveness, Stealthiness, Robustness, Practicability, Aesthetics, and Economics.The metric aggregates these six components into a unified assessment framework.
  • hiPAA weighting: The framework assigns weights of 0.3 to Effectiveness, 0.2 each to Robustness and Stealthiness, and 0.1 each to Practicability, Aesthetics, and Economics.The six weights sum to 1 and are tiered according to component importance.
  • Evaluation dimensions: Effectiveness measures attack-induced performance degradation, robustness considers cross-model, cross-scenario, and related dynamic conditions, and stealthiness assesses human-perceived invisibility.Stealthiness is evaluated through participant ratings on a 5-point Absolute Category Rating scale.
  • Evaluation dimensions: Practicability is rated by implementation feasibility, Economics uses manufacturing and deployment costs, and Aesthetics evaluates human-perceived naturalness when adversarial textures remain visible.Stealthiness and aesthetics are separated by whether the adversarial texture is intended to be invisible or visibly aligned with human aesthetics.
  • Limitations: hiPAA remains limited by scenario-specific weighting and inconsistent cross-task metrics, motivating task-specific protocols for more comparable physical-attack assessments.The evaluation recruited 30 consenting adults to rate stealthiness, aesthetics, and practicability, while scores were normalized between 0 and 1.

5 VICTIM TASKS

This section categorizes physical adversarial attacks by victim task, identifying 15 subtasks organized into four main groups and presenting their chronological milestones.

  • 5 VICTIM TASKS: The survey identifies 15 physical adversarial-attack subtasks, grouped into classification, detection, re-identification, and other tasks.Figure 2 presents the task overview, while Table 2 marks milestone works chronologically.

5.1 Attacks on Classification Tasks

Physical attacks on classification tasks span general image, traffic-sign, light-based, camera-based, and 3D-object methods. Evaluation results indicate that progress has not consistently improved, partly because existing assessments lack comprehensiveness.

  • Overview: Research attention concentrates on general and traffic-sign classification because of their fundamental importance and downstream impact.The survey directs readers to Figures 5 and 6 for visual examples of these attack categories.
  • General classification: Adversarial patches can cause high-confidence misclassification, including changing a banana into a toaster when placed beside it.The adversarial patch was highlighted for its simple production, easy deployment, and high attack potency.
  • General classification: Physical classification attacks also use full-image perturbations, designed illumination, camera-lens stickers, and adversarial 3D objects.Examples include Phy-FGSM attacks that remain effective after printing and recapturing, light-based attacks, translucent lens stickers, and a turtle misclassified as a rifle across viewpoints.
  • General classification: General-classification attack performance has not continuously improved, and CPAttack trades strong effectiveness and economics against neglected robustness.The survey attributes this issue partly to the lack of a comprehensive evaluation framework.
  • Traffic sign classification: Traffic-sign classification is a safety-critical application, with RP2 initiating attacks by affixing black-and-white blocks onto signs.The survey presents traffic-sign attacks as relevant to autonomous-driving scene understanding and decision-making.

5.2 Attacks on Detection Tasks

Physical adversarial attacks on detection tasks primarily evade detectors by altering vehicles, concealing people, or disrupting traffic-sign and related detection systems. Vehicle attacks are evaluated across six hiPAA perspectives, while person attacks extend from visible patches to wearable, generative, and thermal-infrared media.

  • Vehicle detection: Vehicle-detection attacks apply distinctive exterior patterns, using photorealistic simulation and neural rendering to generate camouflage that addresses occlusion and long-distance conditions.DTA combines photorealistic rendering with differentiability, while FCA renders nonplanar textures over the full vehicle surface.
  • Vehicle detection: Vehicle attacks perform poorly in stealthiness, and no method excels across all six perspectives; AdvLight is robust but economically weak.Large, single-color vehicles make external alterations noticeable, limiting stealthiness across methods.
  • Person detection: 1.00 to 0.28: adversarial patches reduced Tiny YOLO detector accuracy in real-world person detection.The attack was among the first proposed for real-world person detectors.
  • Person detection: Person-detection attacks conceal people with adversarial patches, T-shirts, invisibility cloaks, cartoon-style textures, GAN-generated samples, and expandable multi-angle textures.AdvT-shirt uses a TPS-based transformer to accommodate non-rigid deformation caused by pose changes.
  • Person detection: Thermal-infrared person detection remains less explored, motivating AdvBulbs and related material-based patches that study physical attack factors such as size, shape, and placement.These attacks target security-relevant thermal systems used in autonomous driving and night surveillance.

5.3 Attacks on Re-Identification Tasks

Physical attacks on re-identification tasks target face-recognition and person re-identification systems using printable or wearable adversarial media. Reported methods enable evasion or impersonation while emphasizing real-world effectiveness, reproducibility, and visual inconspicuousness.

  • Face Recognition Systems: Face-recognition attacks use printable eyeglasses, patches, hats, and makeup to reduce recognition similarity or enable evasion and impersonation.Methods include non-printability-constrained eyeglasses, gray patches applied as eyeglasses or stickers, forehead stickers on hats, and natural-looking eye makeup.
  • Face Recognition Systems: AdvMakeup applies natural-looking eye-region makeup and uses task-driven fine-grained meta-learning to deceive face-recognition systems inconspicuously.Its makeup generation module adds natural eye shadow over the orbital region.
  • Person Re-ID: AdvPattern is identified as the first and only physical attack on person re-identification, supporting both evasion and impersonation through distinct optimization objectives.The method uses adversarial patches with special patterns in the physical world.

5.4 Attacks on Other Tasks

Physical adversarial attacks extend beyond mainstream tasks to eight niche vision tasks, with performance compared using the hiPAA metric. Reported attacks affect crowd counting, monocular depth estimation, segmentation, steering prediction, and object tracking in real-world settings.

  • 5.4 Attacks on Other Tasks: Eight niche tasks are covered, including optical flow estimation, steering angle prediction, crowd counting, segmentation, object tracking, monocular depth estimation, image captioning, and X-ray detection.Table 11 compares attack methods for these tasks using the hiPAA metric.
  • 5.4 Attacks on Other Tasks: Crowd-counting attack PAP causes overcounting by up to 100 on 80% of samples by maximizing the victim model’s loss.The attack uses a perceptual adversarial patch in the physical world.
  • 5.4 Attacks on Other Tasks: MDE attack OAP achieves over 6 meters mean depth-estimation error and 93% ASR in downstream tasks using optimized rectangular patch placement.Despite its effectiveness, OAP relies on 2D image patches.
  • 5.4 Attacks on Other Tasks: A 1m × 2m adversarial patch optimized with pixel-wise cross-entropy disrupts physical semantic-segmentation predictions.The patch is optimized against a pretrained ICNet model.
  • 5.4 Attacks on Other Tasks: Physical attacks also target steering prediction and object tracking, using visually resilient roadside signs, monitor backgrounds, or patches to mislead autonomous systems and trackers.Tracking attacks can cause systems to prioritize the background or neglect the originally tracked object.

6 DISCUSSION · 6.1 Current Challenges

The discussion identifies unresolved gaps in physical adversarial attacks despite the field’s diversity and growth, emphasizing digital–physical domain gaps and unreliable real-world evaluation. It calls for further research and more reliable, controllable evaluation setups.

  • 6 DISCUSSION: Despite increasing publication activity, physical adversarial attacks still contain gaps requiring further exploration through discussion of current challenges and opportunities.
  • 6 DISCUSSION: Physical adversarial attacks span steering angle prediction, object tracking, and image captioning, illustrating the field’s broad scope across computer vision tasks.
  • 6.1.1 Existing Domain Gaps: The attack workflow designs perturbations digitally, deploys them physically, and executes attacks digitally, requiring transformations between digital and physical domains.
  • 6.1.1 Existing Domain Gaps: Existing research has begun modeling digital-to-physical transformation, including Jan et al.’s D2P network for transforming images between domains.
  • 6.1.2 Uncontrollable Evaluation Settings: Real-world evaluations commonly use manufactured adversarial mediums, but dynamic environments and subjective crafting factors make results uncontrollable.
  • 6.1.2 Uncontrollable Evaluation Settings: Uncontrollable factors include clothing material and printing quality, motivating future evaluation setups that are reliable and controllable.

6.2 Future Work

Future work should improve physical adversarial attacks by addressing domain gaps, reporting experimental conditions, enhancing realistic simulation, evaluating all six hiPAA dimensions, and expanding to underexplored vision tasks.

  • Addressing Domain Gaps: Future attacks should address both digital-to-physical and physical-to-digital domain gaps to better connect manufactured perturbations, camera transformations, and DNN inputs.The first gap includes printing losses and color-reproduction limitations, while the second concerns transformations from physical perturbations into digital images.
  • Improving Experimental Reporting: Researchers should disclose adversarial-medium production details and physical testing conditions, including materials, size, manufacturing, lighting, backgrounds, and shooting distance.These details are described as indispensable for comprehensive evaluation.
  • Enhancing Simulation: Future work should improve physical-environment simulation to enhance robustness under dynamic conditions while addressing simulators’ non-differentiability during end-to-end optimization.Unreal Engine and Unity provide controllable lighting, backgrounds, camera distances, and view angles, but their non-differentiability limits optimization use.
  • Comprehensive Evaluation: Physical adversarial attacks should be evaluated across all six hiPAA perspectives rather than optimizing individual dimensions or isolated tradeoffs.The survey identifies effectiveness, stealthiness, robustness, practicability, aesthetics, and economics as the six evaluation perspectives.
  • Expanding Task Coverage: Future research should extend physical adversarial attacks to underexplored tasks beyond established applications such as person detection, traffic sign detection, and face recognition.Monocular depth estimation is cited as an example of a critical real-world driving task that has recently received an adversarial patch attack.

7 CONCLUSION

The paper surveys physical adversarial attacks on computer vision, motivated by concerns about deep neural network reliability and security. It covers multiple task categories while emphasizing adversarial mediums and comprehensive evaluation.

  • Physical adversarial attacks raise security concerns about the reliability of deep neural networks.
  • The survey covers physical adversarial attacks across classification, detection, re-identification, and niche computer vision tasks.
  • Its overview focuses on adversarial mediums and comprehensive evaluation of real-world attacks.
Loading 2209.15179v4…