Source-linked AI summary
Robust Physical-World Attacks on Deep Learning Models
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song
TL;DR
Physical systems increasingly use DNNs despite adversarial visual perturbations that can induce dangerous behavior, while robust physical attacks and standardized evaluations remain limited. The paper introduces RP2 and a two-stage lab/field evaluation, showing targeted misclassification of real road signs under changing viewpoints, including with stickers on a moving-vehicle test. It also demonstrates a single-sticker attack on a microwave.
Problem
DNNs used in physical systems are vulnerable to adversarial visual modifications, while robust physical perturbations face environmental variability and physical-world evaluation lacks a standardized methodology.
Method
RP2 samples physical and synthetic transformations, masks perturbations onto the target object, and evaluates them through lab and field tests.
Results
RP2 produced robust physical perturbations, including road-sign attacks with 100% poster success in stationary and drive-by tests against LISA-CNN and 87.5% sticker success on GTSRB-CNN video frames.
Takeaways & Limitations
Low-cost, physically realizable modifications can reliably cause classification errors under widely varying distances and angles, motivating attention to physical-world adversarial risk.
Takeaways & Limitations
RP2 is evaluated with access to the target classifier’s architecture and weights, while black-box evaluation remains an open question.
Abstract
from arXiv · showhide
Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations.Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm,Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. Witha perturbation in the form of only black and white stickers,we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8%of the captured video frames obtained on a moving vehicle(field test) for the target classifier.
1. Introduction
RP2 addresses the difficulty of making physical adversarial perturbations robust to changing viewpoints and imperfect real-world conditions. The paper introduces a two-stage evaluation and demonstrates targeted misclassification using practical perturbations on road signs and other objects.
- Motivation: DNN vulnerabilities matter because physical systems such as cars, UAVs, and robots may be misled by adversarial visual modifications.Such modifications can cause controlled systems to behave unexpectedly and potentially dangerously.
- Challenge: Environmental variability, especially changing camera distance and angle, is the main challenge for robust physical perturbations.Additional constraints include sensor imperfections, changing backgrounds, and imperfect fabrication.
- Approach: RP2 samples physical dynamics and synthetic transformations, then constrains perturbations to the target object using a mask.The mask supports visible but inconspicuous shapes resembling graffiti rather than modifying the surrounding background.
- Evaluation: The evaluation uses lab tests across distance and angle configurations and field tests involving a moving vehicle under uncontrolled conditions.The methodology was proposed because physical adversarial perturbations lacked a standardized evaluation procedure.
- Results: 100% success was reported for poster attacks in stationary and drive-by tests against LISA-CNN, while sticker attacks reached 80% in stationary conditions.The classifiers achieved 91% accuracy for LISA-CNN and 95.7% for GTSRB-CNN on their respective test sets.
- Results: 87.5% success was reported for sticker attacks on extracted video frames against GTSRB-CNN, and a single sticker caused Inception-v3 to classify a microwave as a phone.The attacks used physically realizable modifications, including black-and-white stickers and object-constrained posters.
2. Related Work
Prior work established digital and some physical adversarial examples, but often relied on synthetic transformations or relatively stable conditions. RP2 extends this line by constraining perturbations to real objects and modeling broader physical variability.
- Adversarial examples: Adversarial examples alter inputs to induce incorrect untargeted predictions or a specified targeted label.The related-work discussion frames these examples using a classifier, an input, and a perturbation constrained by a distance measure.
- Digital attacks: Early methods such as fast-gradient and optimization-based attacks primarily developed digital adversarial examples.These approaches modify image inputs rather than necessarily altering physical objects.
- Printed attacks: Printed-image attacks and synthetic-transformation methods demonstrated physical vulnerability but did not modify the physical objects themselves.RP2’s object-constrained poster attack reproduces this setting while confining perturbations to the sign surface.
- 3D-printed objects: Prior 3D-printed-object work differed by using synthetic transformations alone and small-scale replicas, whereas RP2 models physical and synthetic transformations and modifies true-sized objects.The comparison identifies both modeling and object-realism differences.
- Stable conditions: Physical attacks on eyeglasses demonstrated success under relatively stable pose, distance, angle, and lighting conditions.The related work contrasts those stable conditions with the broader environmental variability considered by RP2.
- Scope: Road-sign detector attacks were reported as unsuccessful, while this paper focuses on classifiers and leaves detector attacks out of scope.This establishes a task boundary rather than a direct comparison of identical attack settings.
3. Adversarial Examples for Physical Objects
The paper frames robust physical attacks as requiring perturbations that survive changing viewpoints, fabrication, and other environmental conditions. It develops an object-constrained optimization approach that combines physical-world sampling, printable-color constraints, transformation alignment, and mask placement.
- Environmental Conditions: Physical attacks must remain effective across changing conditions, especially camera distance and viewing angle.These variations can reduce perturbation effectiveness in real-world settings.
- Single-Image Optimization: The attack optimizes a perturbation δ added to an image x so the target classifier predicts the chosen target class y*.The formulation uses a distance term and a targeted classification constraint, with ||δ||p specifying the distortion measure.
- Robust Physical Perturbation: RP2 samples transformed instances from a distribution combining experimental data and synthetic transformations to model physical variability.Road-sign variations include distance, angle, lighting, cropping, brightness, and spatial transformations.
- Spatial Constraints: A mask restricts perturbations to the target object's surface and makes them visible but inconspicuous.The mask contains ones where perturbations are optimized and zeroes elsewhere, and can be shaped like graffiti.
- Mask Placement: Mask placement is selected by probing the object's physical features because different positions can affect attack effectiveness.The proposed pipeline begins by computing perturbations with L1 regularization using a mask covering the entire sign surface.
- Fabrication Constraints: The objective includes a non-printability term to account for printer color reproduction errors before the perturbation is printed and applied to the object.The final perturbation is printed, cut out, and placed on the target object.
4. Experiments
The experiments evaluate RP2 with controlled lab and moving-vehicle field tests across varying distances and angles, then measure targeted physical attacks on road signs and everyday objects.
- Evaluation methodology: The evaluation uses stationary lab tests and drive-by field tests to assess perturbations across camera distances and angles.Field recordings simulate a vehicle approaching a sign at speeds between 0 mph and 20 mph.
- Evaluation methodology: Attack success counts only when the perturbed image is misclassified and its clean counterpart at the same distance and angle is correctly classified.This controls for errors caused by factors other than the added perturbation.
- Results for LISA-CNN: 100% of poster-printed Stop-sign images were targeted as Speed Limit 45 by LISA-CNN, with 80.51% average target confidence.Unperturbed signs were classified into the true class in 100% of baseline tests.
- Results for LISA-CNN: 73.33% targeted attack success was achieved on a Right Turn sign across 15 distance/angle configurations, despite four non-target classifications.The remaining failures were still misclassified as other warning-sign classes.
- Results for LISA-CNN: Drive-by targeted success reached 100% for the subtle poster attack and 84.8% for camouflage abstract-art stickers on LISA-CNN.The clean Stop sign was correctly classified in all sampled video frames.
- Results for GTSRB-CNN and Inception-v3: Additional attacks achieved 80% targeted success on GTSRB-CNN, 90% on a microwave with Inception-v3, and 71.4% targeted success on a coffee mug.The coffee mug also had a 100% untargeted success rate.
5. Discussion
The discussion contrasts RP2’s white-box setting with unresolved black-box evaluation and examines how image cropping affects attack success. It also notes evidence that improved RP2 can attack detectors without cropping.
- Black-Box Attacks: RP2 assumes access to the target classifier’s network architecture and model weights, making the evaluated attack a white-box attack.The authors use this strongest attacker model to analyze requirements for successful attacks and inform future defenses.
- Black-Box Attacks: Evaluating RP2 in a black-box setting remains an open question.
- Image Cropping and Attacking Detectors: 70% targeted and 90% untargeted attack rates were observed against LISA-CNN, while GTSRB-CNN achieved 60% targeted and 100% untargeted rates under pseudo-random cropping.The evaluation included cropping conditions in which most of the sign was guaranteed to remain in the image.
- Image Cropping and Attacking Detectors: An improved version of RP2 reportedly attacks object detectors without requiring image cropping.
6. Conclusion
The conclusion presents RP2 as an algorithm for physically realizable adversarial perturbations and pairs it with lab and drive-by evaluation. Road-sign classification serves as the safety-relevant, naturally noisy application domain.
- Conclusion: RP2 generates robust, physically realizable adversarial perturbations.
- Conclusion: A two-stage experimental design using lab and drive-by tests supports understanding physical adversarial examples when objects themselves are perturbed.
- Conclusion: Road-sign classification is targeted because of its importance for safety and its naturally noisy physical setting.