Source-linked AI summary
The Translucent Patch: A Physical and Universal Attack on Object Detectors
Alon Zolfi, Moshe Kravchik, Yuval Elovici, Asaf Shabtai
TL;DR
Prior physical attacks on object detectors required access to each target object, making multi-object concealment dependent on patching each one. This paper optimizes a translucent universal patch for the camera lens to suppress a selected class while limiting effects on other classes. The physical patch failed to detect 42.27% of stop-sign instances while retaining nearly 80% detection of untargeted-class instances.
Problem
Earlier physical attacks required applying a patch to the target object, and hiding multiple objects required patching each one.
Source: §1 · p. 1Method
Results
The method had the best trade-off among evaluated attacks between target-class misdetection and untargeted-class detection; physically, it failed to detect 42.27% of stop-sign instances while detecting nearly 80% of untargeted-class instances.
Source: §5.2 · p. 8, §6 · p. 9Takeaways & Limitations
The experiments highlight a risk to autonomous-car ADAS from an adversary who can apply a patch to the camera.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Physical adversarial attacks against object detectors have seen increasing success in recent years. However, these attacks require direct access to the object of interest in order to apply a physical patch. Furthermore, to hide multiple objects, an adversarial patch must be applied to each object. In this paper, we propose a contactless translucent physical patch containing a carefully constructed pattern, which is placed on the camera's lens, to fool state-of-the-art object detectors. The primary goal of our patch is to hide all instances of a selected target class. In addition, the optimization method used to construct the patch aims to ensure that the detection of other (untargeted) classes remains unharmed. Therefore, in our experiments, which are conducted on state-of-the-art object detection models used in autonomous driving, we study the effect of the patch on the detection of both the selected target class and the other classes. We show that our patch was able to prevent the detection of 42.27% of all stop sign instances while maintaining high (nearly 80%) detection of the other classes.
1. Introduction
Physical patches have typically required access to each target object, while camera-based attacks on object detection raise the additional challenge of manipulating many candidate boxes. The paper places a translucent universal patch on the camera lens and reports reduced stop-sign detection while preserving detection of other classes.
1. Introduction
- Earlier physical attacks required access to each target object, and hiding multiple objects required a patch on each one.Source: §1 · p. 1
- The proposed lens-mounted patch uses gradient-based optimization to hide all instances of a selected class, limit effects on untargeted classes, and remain printable and inconspicuous.Source: §1 · p. 1
- Unlike the cited camera-based attack on image classification, this work targets object detection, where thousands of candidate boxes per output pixel complicate the attack.Source: §1 · pp. 1–2
- The physical patch failed to detect 42.27% of stop-sign instances while still detecting nearly 80% of untargeted-class instances.The corresponding digital attack decreased stop-sign average precision by 42.47%.Source: Figure 1 · p. 1, §1 · p. 2
- The paper also reports transferability from a surrogate model and a design process that considers printing and patch-placement constraints.Source: §1 · p. 2
2. Related Work
Related work spans digital and physical attacks across computer-vision tasks, attacker knowledge, and perturbation types. This paper distinguishes its lens-mounted physical attack from methods that alter the targeted object, while also examining effects on untargeted classes.
2. Related Work
- Prior attacks are categorized by task, attack type, attacker knowledge, and perturbation type; the review is organized around attack type because the proposed work is physical.Source: §2 · p. 2
2.1. Digital Attacks
- Digital attacks expanded from classification to universal perturbations and segmentation, but digitally tampered inputs do not naturally transfer to the physical world.Source: §2.1 · p. 2
2.2. Physical Attacks
- Physical attacks have altered objects with stickers, printed signs, clothing, or patches, including attacks on classifiers and object detectors.Source: §2.2 · p. 2
- The proposed work instead perturbs the camera lens to attack object detectors and evaluates effects on non-target classes, an aspect not addressed by Li et al. [15].Source: §2.2 · pp. 2–3
3. Method
The method optimizes a printable, translucent region-level patch placed on the camera lens to hide a selected class while preserving detection of other classes. It represents the patch with configurable shapes and optimizes their parameters using a custom loss and backpropagation.
3. Method
- The lens-mounted universal patch is optimized to hide target-class instances while minimizing effects on untargeted-class detection.The attacker is assumed to have direct access to the camera lens.Source: §3 · p. 3
3.1. Patch Design
- The patch is a four-channel image composed of configurable blurry oval shapes with position, radius, shear, and color parameters.Its position-dependent alpha channel makes shapes more opaque near their centers and less opaque toward their edges.Source: §3.1 · pp. 3–4
- A predefined total-variation constraint shapes the alpha channel, while alpha blending models the effect of applying the translucent patch to an image.Manual parameter choices also limit patch noise and approximate a printed patch.Source: §3.1 · pp. 3–4
3.2. Patch Optimization
- The optimization combines target-class confidence and bounding-box IoU objectives with untargeted-class detection and printability losses, then minimizes the weighted total loss using backpropagation.The authors use grid search for loss weights and allow two weights to be zero during hyperparameter search.Source: §3.2 · pp. 4–5
4. Motivating Example: Tesla Use Case
Physical translucent patches placed on a Tesla Model X front camera deceived its ADAS in two proof-of-concept tests. A cyan patch changed the perceived color of a red traffic light, while a red patch hid a stop sign.
4. Motivating Example: Tesla Use Case
- A physical on-sensor patch demonstrated a camera-based attack on an object detection model in a Tesla Model X ADAS proof of concept.The patches were fully colored and not generated by the proposed attack.Source: §4 · p. 5
- The demonstration applied patches to the front camera rather than directly to the traffic-sign targets.Source: §4 · p. 5
- A cyan patch made the ADAS interpret a red traffic light as green, while a red patch prevented a stop sign from being detected.Source: §4 · p. 5
5. Evaluation
The physical lens patch missed 42.27% of stop-sign instances while detecting nearly 80% of untargeted instances. Digital tests also found target-class suppression, limited untargeted-class impact, and transfer to other detectors.
5. Evaluation
- Evaluation used digital alpha blending and a printed lens patch, with YOLOv5 evaluated white-box and YOLOv2 and Faster R-CNN evaluated as black-box models.The driving-related data combined about 1,750 stop-sign images; the test set used LISA images, while training and validation used BDD and MTSD.Source: §5 · pp. 5–6, §5.2 · p. 8
5.1. Digital Attacks
- With eight shapes and αmax = 0.4, PATCH reduced YOLOv5 AP to 52.7%, a decrease of over 42.47% from its 95.17% best performance.Among the examined patches, PATCH had the greatest effect on the stop sign class.Source: §5.1 · p. 7, Figure 6 · p. 8
- Untargeted-class AP was 82.69%, a 10.07% difference from the random patch and better than fully colored patches.The detector’s untargeted-class AP was 100% on this dataset because its ground-truth labels were generated by the detector.Source: §5.1 · p. 7
- As the number of shapes increased, AP decreased until overlapping shapes limited further reduction; increasing αmax also lowered AP but could harm untargeted detection and patch imperceptibility.Table 1 presents AP as a function of shape count, and Table 2 presents AP as a function of αmax; the text reports no specific table values for these trends.Source: §5.1 · p. 7, Table 1 · p. 7, Table 2 · p. 8
- A patch optimized on YOLOv5 reduced stop-sign AP to 57.36% on YOLOv2 and 54.53% on Faster R-CNN, while untargeted-class AP remained close to each detector’s best performance.The patch used eight shapes and αmax = 0.4 and was tested on detectors other than the surrogate model.Source: §5.1 · p. 8, Table 3 · p. 8
5.2. Physical attacks
- 42.27% of stop sign instances were missed by the physical patch, while nearly 80% of untargeted instances were still detected.The comparison is against the detector without a patch; random patches had inadequate stop-sign fooling, while fully colored patches performed poorly on untargeted classes.Source: §5.2 · p. 8, Table 4 · p. 8
6. Conclusion
The paper presents a physically realizable camera-lens attack that does not require direct access to the targeted object. Its experiments show a target-class versus untargeted-class trade-off, and the authors identify risks for autonomous-car ADASs.
6. Conclusion
- The authors present a translucent camera-lens patch and custom optimization process for a physically realizable attack without direct access to the targeted object.The patch design considers real-world constraints.Source: §6 · pp. 8–9
- The method achieved the best trade-off among evaluated attacks between stop-sign misdetection and detection of untargeted classes.The experiments demonstrated that a specific class can be prevented from being detected while other classes remain detectable.Source: §6 · p. 9
- The study highlights a risk to autonomous-car ADASs from adversaries placing patches on cameras and calls for countermeasures such as anomaly detection or active sensor checks.Source: §6 · p. 9