Source-linked AI summary
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
Jiajun Lu, Hussein Sibai, Evan Fabry, David Forsyth
TL;DR
The paper asks whether physical adversarial examples remain effective when objects are viewed from changing distances and angles, a concern for autonomous-vehicle safety. It runs attacks against classifiers and YOLO, then photographs perturbed stop signs across viewing conditions. The experiments suggest that most frames are classified correctly because adversarial effects do not reliably persist across scale and viewpoint.
Problem
Physical adversarial examples can survive printing and photography, but prior experiments largely overlook changing distances and angles relevant to moving-platform object detection.
Method
The paper generates adversarial examples with several attacks against classifiers and YOLO, then prints perturbed stop signs and photographs them from varied distances and angles.
Results
Physical adversarial examples do not reliably fool object detectors across different distances and angles; destruction rates are high and increase with distance.
Takeaways & Limitations
A moving autonomous vehicle should correctly identify most frames of a perturbed stop sign, while attacks effective across many viewing conditions remain an open question.
Takeaways & Limitations
The detector attack modifies bounding boxes, object confidences, and class confidences together, potentially producing relatively weak, easier-to-detect adversarial examples; the angle experiment was also simple.
Abstract
from arXiv · showhide
It has been shown that most machine learning algorithms are susceptible to adversarial perturbations. Slightly perturbing an image in a carefully chosen direction in the image space may cause a trained neural network model to misclassify it. Recently, it was shown that physical adversarial examples exist: printing perturbed images then taking pictures of them would still result in misclassification. This raises security and safety concerns. However, these experiments ignore a crucial property of physical objects: the camera can view objects from different distances and at different angles. In this paper, we show experiments that suggest that current constructions of physical adversarial examples do not disrupt object detection from a moving platform. Instead, a trained neural network classifies most of the pictures taken from different distances and angles of a perturbed image correctly. We believe this is because the adversarial property of the perturbation is sensitive to the scale at which the perturbed picture is viewed, so (for example) an autonomous car will misclassify a stop sign only from a small range of distances. Our work raises an important question: can one construct examples that are adversarial for many or most viewing conditions? If so, the construction should offer very significant insights into the internal representation of patterns by deep networks. If not, there is a good prospect that adversarial examples can be reduced to a curiosity with little practical impact.
1. Introduction
Adversarial perturbations can fool neural networks and even survive short-range physical capture, raising safety concerns for autonomous vehicles. This paper tests whether such attacks persist across distances and angles and finds they generally do not.
- Small, carefully chosen perturbations can cause neural networks to misclassify images while remaining imperceptible to humans.
- Physical adversarial examples have been demonstrated by printing perturbed images and photographing them for neural-network classification.
- Prior physical attacks used short-distance photographs, typically taking only one photo per image.
- These findings raise serious safety and security concerns when neural networks control safety-critical systems such as autonomous vehicles.
- Experiments in this paper show that physical adversarial examples do not generalize across several distances and angles for object detectors.Most frames captured by a car approaching perturbed stop signs are classified correctly.
2. Methods for Generating Adversarial Examples
The paper evaluates several gradient-based attacks against classifiers and extends adversarial-example generation to the YOLO object detector. It also tests strong perturbations while identifying weaknesses in the detector attack formulation.
- The experiments use weak and strong attacks with varied hyperparameters to test two neural networks with different architectures.
- Fast Sign method: The Fast Sign method perturbs pixels along the image-space direction that most increases the linearized cross-entropy cost under the l∞ norm.The hyperparameter ϵ controls the distance between the original and adversarial images.
- Fast Sign method: The experiments set ϵ = 0.2 × 255 = 51, a large perturbation intended to test whether strong attacks remain adversarial across distances and angles.
- Iterative methods: The iterative method applies smaller updates repeatedly and clips pixels after each iteration to keep the result within the original image’s l∞ ϵ-ball.The experiments use α = 10 and 20 iterations, described as heuristic choices.
- L-BFGS method: L-BFGS searches for a perturbation minimizing c|R| + J(X + R, yfool), balancing perturbation size against classification as a fooling class.Unlike the iterative method, it estimates step size with L-BFGS and applies no pixel clipping.
- Attacking object detectors: The detector attack replaces the classifier’s one-hot target with a vector whose elements equal 1/lpoutput, driving detector outputs toward no detected object.The attack perturbs bounding-box coordinates, object confidences, and class confidences together, which may make examples weaker and easier to detect.
3. Physical Adversarial Examples
Physical adversarial perturbations were evaluated under changing camera distances and angles using printed stop signs, detector and classifier attacks, and retaken photographs. The experiments suggest that camera movement and scale changes disrupt attacks across viewing conditions, so most frames remain correctly classified.
- Experimental rationale: Camera movement changes the image pattern seen by the model, severely disrupting most physical adversarial attacks.The study focuses on varying camera distance and also reports results for camera rotation.
- Experimental setup: The experiments generated adversarial examples against both a YOLO detector and a traffic sign classifier, then printed them and measured detection rates from simulated driving conditions.The setup used 180 stop-sign photos captured at varied distances, angles, and lighting conditions, with evaluations on digital and retaken physical images.
- Scale effects: High-frequency detector perturbations were more visible nearby but could not be preserved at far distances, whereas low-frequency classifier perturbations remained more visible from farther away.The differing frequency patterns help explain why attack effectiveness varies with viewing scale.
- Physical results: Clean-photo detection was 30%, while photographed clean cropped signs reached 65% at 0.5m versus 11% for clean cropped digital images.The authors attribute this increase to the high-contrast natural background in the physical retake photos.
- Image preparation effects: Cropping decreased detection because it removed background features, while concentrating perturbations on the sign made the cropped attack stronger digitally.Printing and photographing the cropped images nevertheless increased detection rates relative to clean cropped digital images.
- Distance-dependent effectiveness: For the Iterative method, relative detection increased from 45% at 0.5m to 96% at 1.5m, while destruction rate increased from 32% to 86%.The results indicate that perturbations are most effective at particular distances rather than consistently across distances.
- Implication for autonomous vehicles: A rolling majority vote over video frames should identify the stop sign because attacks are weaker at distances beyond the detector’s maximum effective range.The paper’s central safety argument is that misdetection occurs only over a limited distance range.
4. Experiment in Real Life
The real-life experiments tested printed adversarial stop signs from a moving car across changing distances and angles. Misclassifications were rare, and most frames correctly detected the signs.
- Experimental setup: The experiment used printed original and adversarial stop signs attached to an actual stop sign, viewed by a windshield-mounted camera during repeated drives.Video was recorded at 2 frames/second while the detector ran in real time.
- Observations: Printed signs were first detected from a closer distance than actual stop signs because they were smaller, blurrier, and less reflective.
- Observations: Rare misclassifications occurred as distance and angle changed while the car approached, with the signs eventually disappearing from the frames.The camera was not close enough to recognize the perturbation reliably before the sign disappeared.
- Observations: One printed perturbed stop sign was misdetected as a sports ball in only two frames and correctly detected in the rest.The authors attribute the two-frame error more likely to low resolution and natural detection error than to the perturbation.
- Angle experiment: In a laboratory angle experiment, both clean and perturbed signs were correctly classified from 20cm to 2m, except one close-range capture angle that misdetected the perturbed sign as a toilet.The clean sign remained correctly detected at that angle.
5. Discussion
The discussion finds that physical adversarial effects are inconsistent across viewing conditions, while highlighting methodological limitations and open questions about broader attacks.
- The detector attack perturbed bounding-box, object-confidence, and class-confidence objectives together, potentially making the examples weaker and easier to detect.The authors propose targeting only stop-sign class-confidence values in future work.
- Multiple images from the same setting help distinguish random detector errors from consistent adversarial effects.The method relies on detector errors being comparatively random while adversarial effects are consistent.
- Most frames correctly detected the perturbed stop sign across varying distances, with only two frames misclassified as a sports ball.The isolated misdetections may reflect low resolution and natural detector error rather than the perturbation.
- A simple angle experiment suggested that viewpoint can change adversarial effectiveness, but a comprehensive verification remains future work.Distance effects were supported by many experiments, whereas angle effects received only preliminary testing.
- Using a drastically larger epsilon improved attack rates and clarified the paper’s point but made comparisons with existing literature difficult.Experiments with smaller epsilon were reportedly even less effective in the physical world.
- The findings leave open whether adversarial examples can remain effective across many or most viewing conditions.The answer could inform understanding of deep-network pattern representations or determine whether physical attacks have limited practical impact.
6. Conclusion
The paper empirically evaluates physical adversarial stop-sign attacks across distances and angles using multiple attack methods, classifiers, and an object detector. It finds that these perturbations cannot reliably fool detectors across varied viewing conditions and generally become less effective as distance increases.
- Across varied distances and angles, physical adversarial perturbations could not reliably fool object detectors.The conclusion is based on controlled stop-sign experiments using three attack methods against both a classifier and an object detector.
- The destruction rate was high in most cases and increased as camera distance increased.The authors measured the fraction of perturbed versions that no longer remained adversarial at each distance.
- A preliminary angle experiment showed that camera angle can also change the effectiveness of an adversarial perturbation.The authors characterize the angle result as preliminary rather than comprehensive.