Source-linked AI summary
On Physical Adversarial Patches for Object Detection
Mark Lee, Zico Kolter
TL;DR
Object-detection patch attacks had generally required modifying or overlapping target objects. This paper constructs and evaluates a physical non-overlapping patch attack, showing that it can strongly suppress YOLOv3 detections across images and webcam input.
Problem
Object-detection adversarial attacks had required manipulating the object being attacked, limiting attacks that leave scene objects unchanged.
Method
The paper designs a physical adversarial patch using projected gradient descent, expectation over transformations, and an object-detection-specific loss, then evaluates it on YOLOv3.
Results
YOLOv3 drops from 55.4 to single digit mAP, and the printed patch suppresses detections in real-time webcam input without overlapping target objects.
Takeaways & Limitations
The attack opens new physical threat vectors that can suppress detections without requiring modification of individual scene objects.
Abstract
from arXiv · showhide
In this paper, we demonstrate a physical adversarial patch attack against object detectors, notably the YOLOv3 detector. Unlike previous work on physical object detection attacks, which required the patch to overlap with the objects being misclassified or avoiding detection, we show that a properly designed patch can suppress virtually all the detected objects in the image. That is, we can place the patch anywhere in the image, causing all existing objects in the image to be missed entirely by the detector, even those far away from the patch itself. This in turn opens up new lines of physical attacks against object detection systems, which require no modification of the objects in a scene. A demo of the system can be found at https://youtu.be/WXnQjbZ1e7Y.
1. Introduction
The paper develops a physical adversarial patch for object detection that suppresses detections without modifying target objects. It demonstrates the attack on YOLOv3 across digital and real-time webcam settings.
- Prior physical object-detection attacks required placing the patch over the object being attacked.
- The proposed patch suppresses all previously detected objects, including objects relatively far from the patch.
- The attack uses projected gradient descent, expectation over transformations, and a loss designed for object detection systems.
- The method is demonstrated on YOLOv3 by evaluating post-attack mAP on COCO and attacking real-time webcam input physically.
- The attack could suppress detections such as pedestrians, cars, and street signs by placing a crafted sign on a sidewalk rather than altering each object.
2. Related Work
Related work established adversarial patches for classification and object detection, but physical detection attacks generally overlapped target objects. The paper instead studies non-overlapping patches and compares them with DPatch while using YOLOv3 as the demonstration detector.
- Adversarial patch attacks were introduced for classifiers as localized, robust, universal perturbations applied by masking image pixels.
- Physical adversarial examples for object detection generally used objects overlapping targets to change their class or suppress detection.
- This approach specifically creates adversarial patches that do not overlap objects of interest.
- DPatch also avoids target overlap, but was tested only digitally and did not clip patches to the valid [0, 1] image range.
- YOLO is a one-shot detector that processes an S × S grid, predicting bounding boxes, confidence scores, and class probabilities; YOLOv3 is used here.
3. Methodology
The paper formulates a universal physical patch attack for YOLOv3 by maximizing detection loss over images and patch transformations, then evaluates it against DPatch in unclipped, clipped, and physical settings. The proposed method suppresses detections more effectively, including under random placement and webcam conditions.
- Clipped attack: Under box constraints, DPatch remains weakly adversarial and plateaus at relatively high mAP, whereas the proposed method achieves single-digit mAP values.The clipped experiments apply rotations, scaling, translations, and brightness changes, with translations allowing arbitrary image locations.
- Attack formulation: The attack optimizes a universal patch by maximizing loss for original targets across sampled images and transformations.The patch application function transforms and masks the patch, while maximization occurs outside the expectation over samples and transformations.
- Experimental setup: The method uses YOLOv3 pretrained on COCO 2014, with mAP-50 evaluated under confidence thresholds 0.001, 0.1, and 0.5.The corresponding unpatched mAP-50 values are approximately 55.4%, 50.3%, and 40.9%.
- Unclipped attack: For unclipped attacks, the proposed update reaches approximately 0 mAP after 5 steps, while DPatch reaches roughly 3 mAP after 50 steps.The unclipped comparison places both 120 × 120 patches at the image top-left; Table 1 evaluates the entire validation set across confidence thresholds.
- Clipped attack: With random placement, the clipped patch achieves as low as 7.2 mAP and captures semantically meaningful detector-salient patterns such as zebra stripes.The patch also attracts most region proposals in the clipped ROI analysis.
- Physical attack: A printed patch suppresses real-time YOLOv3 webcam detections across varied positions, lighting, and orientations, and can disable moving-object detections when the patch remains stable.Its influence weakens for farther objects; side placement may require enlargement and can fail against sufficiently confident detections.
4. Conclusion
The paper introduces a physical patch attack that suppresses YOLOv3 detections without requiring the patch to overlap target objects.
- The attack reduces YOLOv3 from 55.4 to single-digit mAP and outperforms DPatch in the untargeted setting.