Source-linked AI summary
Backdoor Attack in the Physical World
Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, Shu-Tao Xia
TL;DR
Static-trigger backdoor attacks assume identical trigger appearance and location during training and testing, but physical-world digitization can violate that assumption. The paper analyzes this vulnerability and proposes transformation-based defense and training enhancement, finding that trigger changes can sharply reduce attack performance while enhanced attacks can remain effective under such changes.
Problem
Existing backdoor attacks commonly use static triggers, although trigger location and appearance may differ between training and testing in the physical world.
Method
The paper studies trigger location and appearance, then uses input transformations for defense and random transformations of poisoned training images for attack enhancement.
Results
Changing trigger location or appearance can sharply degrade static-trigger attacks, while enhanced attacks may still succeed in physical-world settings where standard attacks fail.
Takeaways & Limitations
Backdoor evaluation and defense should account for transformation vulnerability rather than assuming identical training and testing triggers.
Takeaways & Limitations
The analysis considers a setting where users cannot fully control the model training process.
Abstract
from arXiv · showhide
Backdoor attack intends to inject hidden backdoor into the deep neural networks (DNNs), such that the prediction of infected models will be maliciously changed if the hidden backdoor is activated by the attacker-defined trigger. Currently, most existing backdoor attacks adopted the setting of static trigger, $i.e.,$ triggers across the training and testing images follow the same appearance and are located in the same area. In this paper, we revisit this attack paradigm by analyzing trigger characteristics. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training. As such, those attacks are far less effective in the physical world, where the location and appearance of the trigger in the digitized image may be different from that of the one used for training. Moreover, we also discuss how to alleviate such vulnerability. We hope that this work could inspire more explorations on backdoor properties, to help the design of more advanced backdoor attack and defense methods.
1 INTRODUCTION
The paper argues that static-trigger backdoor attacks are vulnerable when trigger location or appearance changes between training and testing, limiting their reliability in physical settings. It proposes transformation-based defense and training enhancement to address this vulnerability.
- Motivation: Static-trigger attacks may degrade sharply when the trigger’s location or appearance differs between training and testing.The paper identifies trigger inconsistency as a vulnerability relevant to physical-world attacks.
- Research question: The study examines trigger location and appearance to determine whether changed testing triggers can still activate hidden backdoors.
- Defense: A testing-time transformation such as flipping or scaling is proposed as a defense against static-trigger attacks.The transformation changes the trigger’s location or appearance before prediction.
- Attack enhancement: Randomly transforming all poisoned training images is proposed to improve attack robustness to trigger changes.The enhancement is designed to be naturally combined with existing backdoor attacks.
- Physical-world relevance: The proposed enhancement is connected to physical attacks, where enhanced attacks may succeed while standard backdoor attacks fail.
2 THE PROPERTY OF EXISTING ATTACKS WITH STATIC TRIGGER
The paper characterizes static backdoor triggers by their location and appearance, then shows that small changes to either can sharply reduce attack performance.
- Backdoor attack setup: The attack model trains on poisoned images generated from a trigger and benign image, together with benign samples, using a target label.The infected model is intended to retain benign-image performance while responding maliciously to triggered inputs.
- Trigger characteristics: A backdoor trigger is characterized independently by its location and appearance within the minimum covering box.Location uses the bottom-right pixel position of the box, while appearance uses pixel values and arrangement for non-zero α entries.
- The effect of location: ASR drops from 100% to below 50% when the trigger location shifts by only 2–3 pixels during inference.The experiment preserves trigger appearance while changing only its location.
3 TRANSFORMATION-INSPIRED DEFENSE AND ATTACK ENHANCEMENT
The paper proposes transforming test images to disrupt static triggers and randomly transforming poisoned training images to improve attack robustness against trigger changes.
- Transformation-based defense: The transformation-based defense preprocesses each test image with a transformation such as flipping or scaling before prediction.It predicts T(x) rather than the original image x.
- Transformation-based defense: The defense is efficient, attack-agnostic, data-free, and model-free, making it suitable for third-party model APIs.It requires only transforming the test image and does not require additional samples or model modification.
- Transformation-based enhancement: The enhanced attack approximates unknown inference transformations using widely adopted transformation families with bounded parameter domains.Each transformation parameter lies within a domain constrained by a maximum transformation size.
- Transformation-based enhancement: Because training over all transformed variants is computationally expensive, the enhancement samples one transformation configuration for each poisoned image at each training step.The sampled transformed poisoned images are trained alongside benign images.
- Physical backdoor attacks: The proposed enhancement targets physical attacks by modeling differences introduced when digitizing devices capture testing images.The paper relates these differences to transformations such as spatial transformations.
4 EXPERIMENT
The experiments evaluate transformation-based defenses and enhanced attacks on CIFAR-10, showing that transformations can strongly reduce attack success while randomized training transformations improve robustness.
- 4.1 TRANSFORMATION-BASED DEFENSE: Table 1 compares clean accuracy and attack success rate across backdoor defenses on CIFAR-10.
- 4.1 TRANSFORMATION-BASED DEFENSE: More than 90% ASR reduction was achieved in all cases with ShrinkPad using a 4-pixel shrinking size.Flip also defended BadNets and Blended Attack, but not Consistent Attack because its trigger is symmetric.
- 4.1 TRANSFORMATION-BASED DEFENSE: The proposed defense generally had higher clean accuracy and lower ASR than the preprocessing-based Auto-Encoder, and was comparable to Fine-Pruning and Neural Cleanse.
- 4.2 ATTACK ENHANCEMENT: Enhanced attacks generally achieved higher ASR than corresponding standard attacks against spatial-transformation defenses.The only exception was Consistent Attack+ under Flip defense; fewer poisoned images also made Consistent Attack+ less favorable to random triggers.
5 CONCLUSION
The paper finds that existing backdoor attacks are vulnerable to transformations and proposes a transformation-based enhancement to reduce that vulnerability and connect it to physical attacks.
- 5 CONCLUSION: The paper reveals transformation vulnerability in existing backdoor attacks and proposes an enhancement to reduce it.
- 5 CONCLUSION: The proposed enhancement is linked to physical attacks and is intended to motivate more advanced backdoor attack and defense methods.