Source-linked AI summary
Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon
Yiqi Zhong, Xianming Liu, Deming Zhai, Junjun Jiang, Xiangyang Ji
TL;DR
Adversarial examples pose risks for safety-sensitive machine-learning deployments, while existing physical attacks can require target access or use conspicuous artificial patterns. This paper proposes a black-box shadow-based optical attack and evaluates it in simulated and real-world traffic-sign settings. The attack is reported to work in both digital and physical settings, while the authors identify lighting, targeting, and optimization limitations.
Problem
Physical-world adversarial attacks must be assessed for risk, but sticker-based methods can require target access or valid-color printing and existing optical patterns are conspicuous.
Method
The paper proposes a black-box light-based physical-world attack that generates naturalistic, stealthy perturbations using shadows.
Results
The attack was successfully applied in both digital and physical settings, with evaluations conducted in simulated and real-world environments.
Takeaways & Limitations
Shadows can mislead machine-learning vision systems into producing erroneous decisions and therefore represent a threat to system reliability.
Takeaways & Limitations
The attack requires a strong single light source, is difficult with multiple or poor lighting, and cannot explicitly conduct targeted attacks.
Abstract
from arXiv · showhide
Estimating the risk level of adversarial examples is essential for safely deploying machine learning models in the real world. One popular approach for physical-world attacks is to adopt the "sticker-pasting" strategy, which however suffers from some limitations, including difficulties in access to the target or printing by valid colors. A new type of non-invasive attacks emerged recently, which attempt to cast perturbation onto the target by optics based tools, such as laser beam and projector. However, the added optical patterns are artificial but not natural. Thus, they are still conspicuous and attention-grabbed, and can be easily noticed by humans. In this paper, we study a new type of optical adversarial examples, in which the perturbations are generated by a very common natural phenomenon, shadow, to achieve naturalistic and stealthy physical-world adversarial attack under the black-box setting. We extensively evaluate the effectiveness of this new attack on both simulated and real-world environments. Experimental results on traffic sign recognition demonstrate that our algorithm can generate adversarial examples effectively, reaching 98.23% and 90.47% success rates on LISA and GTSRB test sets respectively, while continuously misleading a moving camera over 95% of the time in real-world scenarios. We also offer discussions about the limitations and the defense mechanism of this attack.
1. Introduction
The paper motivates a stealthier physical-world adversarial attack because existing approaches can be impractical, conspicuous, or difficult to realize. It proposes shadow-based perturbations and evaluates them in simulated and real-world settings.
- Motivation: Physical-domain attacks matter because adversarial inputs can cause erroneous decisions in safety-sensitive applications.The paper identifies autonomous driving and medical diagnosis as examples where such errors may be dangerous.
- Limitations of Existing Attacks: Sticker-pasting attacks can require physical access to targets and may be limited by imperfect reproduction of printed colors.These constraints are presented as practical troubles for physical-world deployment.
- Limitations of Existing Attacks: Existing laser- and projector-based optical patterns are artificial, conspicuous, and easily noticed by humans.The paper contrasts these methods with its goal of naturalistic and stealthy perturbations.
- Proposed Attack: The paper proposes a black-box physical-world attack that uses shadows as naturalistic and stealthy light-based perturbations.The method targets traffic sign recognition and uses a common natural phenomenon rather than printed stickers or artificial optical patterns.
- Evaluation: The study evaluates the method in simulated and real-world environments and discusses attack limitations and defenses.These evaluations are presented as evidence of the method’s effectiveness across both settings.
2. Related Work
Related work covers digital and physical adversarial examples, black-box attack settings, and physical-world realizability. Recent non-invasive optical attacks avoid accessing targets but use artificial laser or projected patterns.
- Adversarial Examples: Digital adversarial perturbations modify model inputs under norm constraints, while physical attacks add perturbations to objects themselves.The paper distinguishes these domains by where the perturbation is applied and how imperceptibility is constrained digitally.
- Attack Settings: Black-box attacks query a target model without access to its internal structure, using transferability, model reconstruction, or both.The paper contrasts this setting with white-box attacks, where gradients can be fully used.
- Physical Realizability: Physical-world adversarial examples must remain effective despite camera-induced variations and practical capture constraints.Earlier printed attacks demonstrated transfer to photographs, but noise-based perturbations were described as impractical in the physical world.
- Optical Attacks: Non-invasive optical attacks illuminate, laser-disturb, or project patterns onto targets without physically accessing the objects.The cited methods use high-frequency light, laser beams, or projectors to affect recognition.
3. Approach
The approach models a shadow by its location and illumination value, then optimizes a triangular shadow perturbation for black-box misclassification while improving robustness to physical variation. It uses LAB-space lightness modeling, EOT, and PSO-based optimization, with prediction stabilization for physical deployment.
- Shadow Perturbation Modeling: The attack represents the adversarial example as a shadow defined by a target-object mask, polygon region, and shadow coefficient.The shadow is applied to the intersection of the polygon and target mask, with the coefficient controlling its value.
- Shadow Location: Triangles are used for the shadow region because they are sufficient for successful attacks while remaining more natural and easier to implement than complex polygons.More complex polygons may improve success rates but make the shadow look unnatural and harder to realize physically.
- Shadow Value: The model simulates shadows in LAB space by multiplying only the L channel within the shadow region by coefficient k, leaving A and B unchanged.Measurements on SBU Shadow report mean shadow-to-non-shadow ratios of 0.43 for L, 0.99 for A, and 0.90 for B.
- Black-box Optimization: PSO replaces zeroth-order gradient estimation because discrete coordinates and Boolean region membership make ZOO gradients unreliable in this attack.Particles represent candidate solutions and update their positions using individual and population information without gradients.
- Expectation Over Transformation: EOT randomizes image transformations and shadow coefficient k so the optimized perturbation remains effective despite camera and physical-domain variation.The transformations include downsampling, brightness adjustment, perspective transformation, motion blur, and random k values; practice approximates the expectation with 10 transformed samples plus the original.
- Prediction Stabilization: Prediction stabilization reruns PSO after initial optimization to improve stability of untargeted misclassification under physical viewing variation.Experiments report similar prediction stability to targeted attacks.
4. Experiments
Experiments evaluate shadow-based attacks in digital, physical, and scheduled settings, including robustness-oriented design choices. The attack remains effective across varied lighting and moving-camera scenarios, while simpler polygons and five random restarts balance effectiveness and overhead.
- 4.2. Evaluation in Digital Domain: 98.23% and 90.47% attack success rates were achieved on LISA and GTSRB test sets respectively when k = 0.43.The value k = 0.43 is the mean shadow value in the SBU Shadow dataset.
- 4.2. Evaluation in Digital Domain: Only dozens or hundreds of queries were needed to obtain valid adversarial examples in the black-box setting.
- 4.3. Evaluation in Physical Domain: Shadow attacks remained effective outdoors in daylight and indoors or at night when an artificial light source generated the shadow.The indoor experiment used a flashlight in a dark stairwell.
- 4.3. Evaluation in Physical Domain: 100.00% of outdoor video frames were misclassified, with 95.91% predicted as speed limit 35.The video contained 220 frames and simulated a camera moving from far to near the sign.
- 4.4. Scheduled Attack: Scheduled attacks succeeded on all eight German speed limit signs, lasting from dozens of seconds to several minutes.Images were generated every second from 8:25 to 8:35 around a scheduled time of 8:30.
- 4.5. Ablation Study: Triangles were sufficient for successful adversarial examples, while using five random restarts improved success rate with only marginal gains beyond five.The simpler polygon reduces implementation and computational burden, and n = 5 was selected in practice.
- 4.6. How to Defend against the Shadow Attack?: Random-shadow adversarial training improved robustness and attack difficulty, at the expense of slightly decreased clean-sample accuracy.The comparison uses robustness as the probability that the attack fails.
5. Limitation
The attack is constrained by its lighting requirements and cannot explicitly perform targeted attacks. These limitations reduce effectiveness in some environments and restrict attack control.
- Lighting constraints: A strong single light source is required to produce significant shadow perturbations.Poor lighting makes k large, causing little difference between attacked and clean images.
- Lighting constraints: Multiple light sources also make it difficult to produce effective shadow perturbations.
- Targeting constraint: The attack cannot explicitly conduct targeted attacks, likely because shadows produce relatively single perturbation directions.Outdoor experiments nevertheless achieved comparable prediction stability to targeted attacks.
6. Conclusion
The paper presents shadow-based optics as an adversarial attack that works in digital and physical settings. It concludes that natural shadows can mislead machine-learning vision systems, while acknowledging drawbacks for future work.
- Conclusion: The proposed optics-based attack uses the natural phenomenon of shadow to produce harmful perturbations on targets.
- Conclusion: The attack can be successfully applied in both digital and physical settings.
- Conclusion: The work shows that shadows can mislead machine-learning vision systems into producing erroneous decisions.
- Conclusion: The authors identify drawbacks and plan to make the attack more practical in future research.