PaperlayerView exact revision on arXiv

Source-linked AI summary

Defending Against Physically Realizable Attacks on Image Classification

Tong Wu, Liang Tong, Yevgeniy Vorobeychik

arXiv:1909.09552v2cs.LGcs.AIcs.CVeess.IVstat.ML

TL;DR

The paper asks how image classifiers can be defended against physically realizable attacks, given the limited effectiveness of conventional robust-learning methods. It evaluates adversarial training and randomized smoothing, proposes rectangular occlusion attacks with efficient attack-computation approaches, and finds that adversarial training using this model achieves high robustness against the studied physical attacks.

  • Problem

    Conventional adversarial training and randomized smoothing have limited effectiveness against prominent physically realizable attacks, motivating a defense model better matched to those threats.

  • Method

    The paper models attacks as adversarially chosen rectangular occlusions, develops algorithms to compute them, and uses the resulting attacks for adversarial training.

  • Results

    Adversarial training with rectangular occlusion attacks achieves high robustness against several prominent physical attacks and is significantly more robust than l_p-based defenses.

  • Takeaways & Limitations

    Matching the training threat model to the structure of physical attacks provides an effective generic defense within the studied attack settings.

  • Takeaways & Limitations

    The evaluation uses digital representations of physically realizable attacks and does not model issues such as viewpoint variation or printability.

Abstract

from arXiv · show

We study the problem of defending deep neural network approaches for image classification from physically realizable attacks. First, we demonstrate that the two most scalable and effective methods for learning robust models, adversarial training with PGD attacks and randomized smoothing, exhibit very limited effectiveness against three of the highest profile physical attacks. Next, we propose a new abstract adversarial model, rectangular occlusion attacks, in which an adversary places a small adversarially crafted rectangle in an image, and develop two approaches for efficiently computing the resulting adversarial examples. Finally, we demonstrate that adversarial training using our new attack yields image classification models that exhibit high robustness against the physically realizable attacks we study, offering the first effective generic defense against such attacks.

1 INTRODUCTION

The paper evaluates whether conventional robust-learning defenses withstand physically realizable attacks and introduces rectangular occlusion attacks with adversarial training as a more effective defense.

  • 1 INTRODUCTION: The work addresses a broader defense problem arising because deep neural networks can be systematically fooled by carefully perturbed images.The paper places its contribution within a substantial literature on robust-model learning and adversarial-input detection.
  • 1 INTRODUCTION: Adversarial training and randomized smoothing show limited effectiveness against physically realizable eyeglass-frame and stop-sign sticker attacks.Their effectiveness can vary by setting, but they do not provide high robustness despite moderate performance against conventional norm-bounded attacks.
  • 1 INTRODUCTION: Rectangular occlusion attacks model an adversary placing a small, adversarially crafted rectangle at a chosen image location.The model is intended to capture common physical attacks more directly than conventional l_p-bounded models.
  • 1 INTRODUCTION: Adversarial training against the proposed attack produces models significantly more robust to physical attacks than defenses based on l_p attack models.The comparison is against adversarial training and randomized smoothing methods that leverage conventional l_p-bounded attack models.

2 BACKGROUND

The paper frames physical attacks as object modifications that remain inconspicuous yet induce misclassification, then examines principled defenses against representative attacks. Its analysis uses digital simulations of three physical attacks and compares robust learning with randomized smoothing.

  • Physical attacks: Physical attacks modify photographed objects rather than digital images, while remaining low-suspiciousness and causing misclassification by state-of-the-art networks.The paper operationalizes low suspiciousness as modifying only a small part of the object with noise-like real-world changes.
  • Threat model: Because the goal is defense, the paper simulates digital representations of physical attacks while ignoring viewpoint robustness, printability, and other implementation issues.For example, stop-sign stickers are represented as digital stickers on stop-sign images.
  • Physical attacks: The study considers eyeglass-frame noise, adversarial stickers on stop signs, and adversarial patches as physically realizable attack examples.The eyeglass-frame and stop-sign examples are illustrated in Figure 1.
  • Defenses: The paper focuses on robust learning and randomized smoothing as principled defense categories, using representative state-of-the-art approaches from each.Robust learning minimizes a robust loss, while randomized smoothing adds isotropic Gaussian noise during training and prediction.
  • Defenses: Adversarial training approximates robust optimization with PGD-generated inputs, whereas randomized smoothing predicts using class probabilities induced by Gaussian-noise perturbations.The robust-learning approach takes gradient-descent steps on the associated adversarial inputs, and smoothing constructs a classifier from a noisy base classifier.

3 ROBUSTNESS OF CONVENTIONAL ROBUST ML METHODS AGAINST PHYSICAL ATTACKS

The section tests conventional robust ML methods against physically realizable eyeglass-frame and stop-sign attacks. Across these settings, adversarial training and randomized smoothing provide limited or inconsistent physical robustness, despite stronger robustness to the l_p attacks used for training.

  • 3.1 ADVERSARIAL EYEGLASSES IN FACE RECOGNITION: Under stronger eyeglass-frame attacks, adversarial accuracy falls below 40%, while the best adversarial-training models also lose about 20% clean accuracy.The strongest adversarial-training variant uses ϵ = 8; 7 and 50 PGD iterations perform similarly.
  • 3.1 ADVERSARIAL EYEGLASSES IN FACE RECOGNITION: Randomized smoothing is ineffective against eyeglass frames, with attacked accuracy below 20% for relatively weak attacks and near 0% for sufficiently strong attacks.
  • 3.2 ADVERSARIAL STICKERS ON STOP SIGNS: Against stop-sign stickers, conventional adversarial training is at best marginally better than the original model, with robust accuracy under 50% for stronger attacks.Training with ϵ = 32 was ineffective on clean data and was omitted from the plots.
  • 3.2 ADVERSARIAL STICKERS ON STOP SIGNS: Randomized smoothing performs inconsistently across stop-sign experiments, but its best variant reaches slightly above 60% accuracy on stronger attacks while losing nearly 40% relative to clean data.The best variant uses σ = 0.25 and significantly outperforms adversarial training in this setting.
  • 3.3 DISCUSSION: The results support attack-model mismatch as a key explanation: conventional robust models are much more robust to the l_p-bounded attacks they were trained against than to physical attacks.

4 PROPOSED APPROACH: DEFENSE AGAINST OCCLUSION ATTACKS (DOA)

The proposed defense models physically motivated threats as rectangular occlusion attacks and trains classifiers against them. This yields Defense against Occlusion Attacks, using efficiently computed adversarial occlusions as an alternative to conventional robust ML for physical attacks.

  • 4.1 ABSTRACT ATTACK MODEL: RECTANGULAR OCCLUSION ATTACKS (ROA): Rectangular occlusion attacks place an adversarially chosen rectangle anywhere in an image and allow bounded l∞ noise inside it, enforcing spatial contiguity.The model abstracts physical stickers and is acknowledged not to capture every real occlusion shape or size.
  • 4.2 COMPUTING ROA ATTACKS: ROA computation searches for a rectangle location and then generates fine-grained adversarial perturbations restricted to that region.The basic method exhaustively evaluates possible upper-left corners and applies PGD inside the selected rectangle.
  • 4.2 COMPUTING ROA ATTACKS: A gradient-based acceleration selects C candidate locations with the largest input-gradient magnitudes before performing exhaustive search over that subset.
  • 4.3 DEFENSE AGAINST OCCLUSION ATTACKS: Adversarial training with ROA produces Defense against Occlusion Attacks classifiers proposed as an alternative defense against physically realizable attacks.The paper states that this defense is adequate for its purposes.

5 EFFECTIVENESS OF DOA AGAINST PHYSICALLY REALIZABLE ATTACKS

DOA, adversarial training under the rectangular occlusion threat model, is substantially more robust than conventional methods against eyeglass, stop-sign, and patch attacks in the reported digital evaluations.

  • 5.1 DOA AGAINST ADVERSARIAL EYEGLASSES: DOA achieves significantly greater robustness than adversarial training and randomized smoothing against the eyeglass frame attack.The gradient-based heuristic performs slightly worse than exhaustive search but remains far better than conventional robust ML approaches.
  • 5.1 DOA AGAINST ADVERSARIAL EYEGLASSES: For rectangles with roughly equal area, changing dimensions has minimal impact when enough PGD iterations are used inside the rectangle.Figure 4 compares 100 × 50 and 70 × 70 rectangles using exhaustive search and a gradient-based heuristic.
  • 5.2 DOA AGAINST THE STOP SIGN ATTACK: DOA reaches over 90% robust accuracy against the stop sign attack with exhaustive search and approximately 85% with the gradient-based variant.These results use square rectangles covering approximately 5% of the 32 × 32 images and 30 PGD iterations.
  • 5.2 DOA AGAINST THE STOP SIGN ATTACK: DOA retains 100% stop-sign classification accuracy on clean data and approximately 95% accuracy on the full traffic-sign classification task.These outcomes accompany the robust-accuracy results for the square-rectangle variant.
  • 5.3 DOA AGAINST ADVERSARIAL PATCH ATTACKS: DOA remains highly robust against adversarial patches covering 20% of the image, while adversarial training and randomized smoothing perform poorly once coverage reaches 10% or more.The comparison is reported for face recognition, with further results on both datasets in Appendix F.

6 CONCLUSION

The conclusion finds conventional robustness methods relatively ineffective against physically realizable image attacks, whereas rectangular occlusion attacks combined with adversarial training achieve high robustness across several prominent examples.

  • 6 CONCLUSION: Rectangular occlusion attacks coupled with adversarial training achieve high robustness against several prominent physically realizable attacks.The authors identify this combination as a promising path toward practical robustness in downstream computer-vision applications.
  • 6 CONCLUSION: Conventional methods for robust image classification tend to be relatively ineffective against physically realizable attacks.The conclusion frames this limitation as motivating the proposed threat model and training approach.

A DESCRIPTION OF DATASETS AND DEEP LEARNING CLASSIFIERS

The experiments use face-recognition and traffic-sign datasets with adapted convolutional classifiers, while conventional robustness methods are evaluated against norm-bounded attacks.

  • Datasets: The face-recognition data comprise 10 selected VGGFace subjects, processed into 224 × 224 images and split into training, validation, and test sets.The final splits contain 3178 training, 922 validation, and 470 test images.
  • Classifiers: Face recognition uses a transfer-learned VGGFace architecture whose classifier is changed to 1024 → 1024 → 10 outputs for the selected subjects.The convolutional layers retain the VGGFace structure.
  • Datasets: The traffic-sign experiments use 16 selected LISA sign categories, with 32 × 32 preprocessing and a stop-sign test set containing 40 images.The selected data include 3509 training and 1148 validation examples.
  • Robustness evaluation: The appendix evaluates curriculum adversarial training and randomized smoothing against l∞ and l2 attacks on both datasets.The cited tables cover curriculum training on face recognition and randomized smoothing against face-recognition l2 attacks.
  • Robustness evaluation: The experiments distinguish digital perturbations applied across images from the physically realizable stop-sign attack setting.The traffic-sign appendix uses adversarial perturbations to all sign images rather than only physical stop-sign attacks.

B.2 TRAFFIC SIGN CLASSIFICATION

On traffic-sign classification, conventional robustness methods remain effective against standard perturbations, while the section evaluates their balance between clean accuracy and robustness and illustrates physical occlusion failures.

  • Traffic-sign robustness: For traffic signs, curriculum adversarial training is relatively robust and retains non-adversarial effectiveness when ϵ < 32.These results are reported for l∞ attacks with varying attack strength.
  • Traffic-sign robustness: For randomized smoothing, ϵ ≤1 generally yields robust traffic-sign performance, with σ = 0.5 balancing non-adversarial accuracy and robustness.The cited result concerns l2 attacks on the smaller traffic-sign images.
  • Rectangular occlusion examples: In face-recognition examples, ROA can make the image harder for human observers to recognize by occluding upper-lip and eye regions.The cited examples use a 100 × 50 rectangle.
  • Rectangular occlusion examples: The ROA examples show 7 × 7 rectangles causing Speedlimit45 and Stop signs to be predicted as Speedlimit30 and Yield, respectively.The examples place the rectangular occlusion directly on the traffic signs.

D DETAILED DESCRIPTION OF THE ALGORITHMS FOR COMPUTING THE RECTANGULAR OCCLUSION ATTACKS

The rectangular occlusion attack searches for a high-loss rectangle location and then generates high-bound perturbations inside it, using either exhaustive or gradient-guided search.

  • Attack construction: ROA places a fixed-dimension rectangle anywhere in the image and permits high-bound l∞ noise inside the rectangle.The rectangle imposes a contiguity constraint unlike unconstrained l0-style perturbations.
  • Exhaustive search: Exhaustive search evaluates candidate upper-left positions and retains the location producing the highest loss.A stride parameter reduces the number of positions considered.
  • Gradient-based search: Gradient-based search ranks candidate regions by summed squared gradient magnitude, then evaluates only the top C locations.The retained candidate with the greatest adversarial loss becomes the selected sticker position.
  • Search efficiency: Gradient-based search is motivated as a cheaper alternative because exhaustive location search requires many costly neural-network loss evaluations.Gradient magnitude serves as a regional sensitivity measure for narrowing the search.
  • Attack construction: After locating the rectangle, the method applies l∞ PGD restricted to that region to generate adversarial noise.The implementation varies PGD iterations and uses corresponding learning rates.

F ADDITIONAL EXPERIMENTS WITH DOA

Additional experiments vary rectangle sizes, search procedures, and PGD iterations, and compare DOA with adversarial training and randomized smoothing on physical attacks.

  • Face-recognition experiments: The face-recognition experiments similarly vary PGD iterations for 100 × 50 and 70 × 70 regions against the eyeglass-frame attack.Figures 9 and 10 report exhaustive and gradient-based search variants.
  • Traffic-sign experiments: DOA evaluations vary PGD iterations for exhaustive and gradient-based searches on 10 × 5 and 7 × 7 traffic-sign rectangles.The cited tables cover both search variants and both rectangle sizes.
  • Patch attacks: The additional experiments compare conventional defenses and DOA against adversarial patch attacks on face-recognition and traffic-sign data.Separate tables report comparisons and defense-specific results for both datasets.
  • Defense comparisons: Additional tables report adversarial-training, curriculum-training, randomized-smoothing, and DOA results for adversarial patch attacks.The reported evaluations span both face-recognition and traffic-sign datasets.
  • Qualitative comparisons: The example panels juxtapose original inputs, attacked images, conventional-defense predictions, and DOA predictions.The face-recognition examples identify DOA predictions as correct in the displayed comparisons.

H EFFECTIVENESS OF DOA METHODS AGAINST l∞ATTACKS

DOA is largely unhelpful against l∞-bounded PGD attacks on face recognition and traffic sign classification because it assumes modifications affect only a small scene region.

  • DOA is largely unhelpful against l∞-bounded attacks on both face recognition and traffic sign classification.The evaluation uses 50-step DOA against 20-iteration l∞ attacks.
  • DOA assumes the attacker modifies only a relatively small proportion of the scene, approximately 5%.This contiguity and sparsity assumption reflects the intended physical plausibility of the threat model.

I EFFECTIVENESS OF DOA METHODS AGAINST l0 ATTACKS

Against l0-style and patterned occlusion attacks, DOA improves robustness and generalizes across several physical attack shapes, including suspicious large masks.

  • DOA improves classification robustness against JSMA l0 attacks in both face recognition and traffic sign classification.The comparison is shown in Figure 13 against the original model.
  • DOA generalizes successfully to unions of triangles and circles, larger triangles, and heart-shaped attack patterns.The evaluation covers three additional occlusion patterns.
  • The larger triangle and heart masks remain covered despite occupying substantial image areas, including 8% of the face for the heart pattern.The heart pattern is described as particularly suspicious because of its size.
Loading 1909.09552v2…