Source-linked AI summary
LaVAN: Localized and Visible Adversarial Noise
Danny Karmon, Daniel Zoran, Yoav Goldberg
TL;DR
The paper asks whether visible adversarial noise can remain confined to a tiny region outside the main object while fooling image classifiers. It learns localized patches and finds that they transfer across images and locations, causing high-confidence misclassification in Inception V3; the authors leave robustness across architectures and training approaches for future work.
Problem
Existing work largely studies imperceptible image-wide noise or visible noise covering prominent object features, leaving small visible patches outside the main object less explored.
Method
The paper learns visible adversarial noise patches in a white-box setting and applies them across images and locations without covering the main object.
Results
2% of the image pixels is sufficient for patches that transfer across images and locations and cause a state-of-the-art image classifier to misclassify to arbitrary target labels with very high success rates.
Takeaways & Limitations
The results expose a classifier blind spot: the network can misclassify even when gradients often do not regard the small patch as the main cause.
Takeaways & Limitations
The patches are model-specific, and the paper leaves susceptibility differences across architectures and robustness through adversarial auxiliary losses for future research.
Abstract
from arXiv · showhide
Most works on adversarial examples for deep-learning based image classifiers use noise that, while small, covers the entire image. We explore the case where the noise is allowed to be visible but confined to a small, localized patch of the image, without covering any of the main object(s) in the image. We show that it is possible to generate localized adversarial noises that cover only 2% of the pixels in the image, none of them over the main object, and that are transferable across images and locations, and successfully fool a state-of-the-art Inception v3 model with very high success rates.
1. Adversarial Noise
The paper studies visible adversarial noise confined to a small image region without covering the main object, contrasting it with imperceptible image-wide noise and object-covering visible attacks. Such patches can transfer across images and locations while fooling Inception V3, revealing a potential architectural blind spot.
- Adversarial Noise: The paper targets visible adversarial noise confined to a bounded area covering up to 2% of pixels without covering the main object.This contrasts with imperceptible noise spread across the image and visible noise placed over prominent object features.
- Adversarial Noise: The patches cause a state-of-the-art Inception V3 classifier to misclassify with very high confidence despite avoiding the main object.The result holds for both network-domain and image-domain noise, with higher success rates for network-domain noise.
- Adversarial Noise: Gradient inspection shows the network usually does not identify the noised patch as the main cause of misclassification, sometimes assigning it hardly any blame.This finding contrasts with the hypothesis that adversarial patches are more salient to the network than real-world objects.
- Adversarial Noise: The localized patches can be transferred across many images and locations while remaining specific to the model used for training.The paper characterizes this as universal across images and locations but white-box model-specific.
- Adversarial Noise: These results highlight an interesting blind spot in current state-of-the-art network architectures.The paper frames adversarial examples as useful for studying classifier strengths, weaknesses, and blind spots beyond their security implications.
2. Localized noise for a single image and location
The single-image, single-location setup optimizes a masked replacement patch toward a target class using gradients from a trained classifier. In experiments, localized network-domain patches achieved high targeted success, but their effectiveness depended strongly on the exact image and location.
- Setting and Method: The first setup generates visible localized noise specific to one image and one location within that image.The patch is later formed by replacing the masked region rather than adding noise across the entire image.
- Setting and Method: The method optimizes a noise patch to increase the target-class probability using a stochastic gradient-based algorithm.It assumes access to a trained model that assigns class probabilities and seeks an image classified as the target class.
- Setting and Method: The localized image is constructed with a binary mask that preserves the original image outside the patch and replaces the masked region with noise.Element-wise multiplication combines the unmasked input with the masked noise patch.
- Setting and Method: The loss moves predictions toward the target class and away from the highest-scored class using pre-softmax network activations.The authors state that this decouples class outputs and speeds convergence.
- Experiments and Results: The experiments use 42x42 patches on 299x299 Inception V3 inputs, with success defined as target confidence of at least 90%.The patch is roughly 2% of the image pixels and is placed around corners to reduce the chance of covering the main object.
- Experiments and Results: 79% of 110 configurations produced localized network-domain noise meeting the high-confidence targeted criterion.Relaxing success to target argmax raised success to 91%, while avoiding the source class reached 98%.
- Shortcomings: 75 out of 100 image-target-location triplets produced high-confidence targeted noise, but shifting the patch by one pixel restored the original classification with high confidence.Moving the patch to another image also failed unless a border surrounding roughly 25% of each side of the source image was transferred.
3. Transferable localized noise
The paper develops transferable localized noise patches that are trained across random images and locations, then evaluates their robustness across target classes, positions, and unseen images. These patches remain effective despite covering only a small image region, with network-domain noise generally outperforming image-domain noise.
- Method: Transferable noises are trained by applying the same patch to randomly sampled images and locations while optimizing toward a shared target class.Generation stops after the target class reaches probability ≥0.9 for 30 consecutive iterations.
- Experiments: The study generates transferable patches for 14 target classes in both network and image domains.Network-domain patches are rescaled to image range only for visualization.
- Location robustness: 83% of tested locations produced target-class confidence ≥0.9, while 97% prevented prediction of the original source class.These results were averaged across 14 targets and 100 test images.
- Cross-image transfer: On unseen images, 43% of attempts reached target confidence ≥0.9, 89% made the target the most likely class, and 100% prevented the original source prediction.The evaluation used 100 ImageNet images excluded from noise generation, with patches placed in the bottom-right corner.
- Image domain: Image-domain patches achieved target confidence ≥0.9 in 28.3% of attempts, made the target class most likely in 74.1%, and caused source-class misclassification in 78.9%.Their success rates were lower than those of network-domain patches but remained effective.
- Class dependence: Success varied substantially by source and target class, with globally structured classes such as volcano hypothesized to be harder targets than locally structured classes such as baseball.The authors qualify this observation because only a subset of ImageNet classes was tested.
4. Noise perceived by the network
The paper examines whether localized patches are salient to the classifier by reversing their effects through image-wide gradient updates. The gradients often emphasize ordinary object features rather than the patch, suggesting that the network can misclassify without explicitly highlighting the attacking region.
- Motivation: Small transferable localized patches can fool a state-of-the-art classifier into misclassifying images.The patches are evaluated as visible localized noises rather than imperceptible image-wide perturbations.
- Gradient analysis: Gradient-based fixes often place substantial activity outside the noise patch while restoring the original source class.This pattern appears when optimizing toward the source class and when optimizing away from the target class.
- Stealthiness examples: In a Sports Car image classified as iPod at 98%, gradients emphasize the car front and surroundings while almost completely ignoring the noise patch.The same stealthy gradient pattern persists for both network-domain and image-domain noises.
- Interpretation: The observed gradients contrast with the proposal that adversarial patches are more salient to the network than real-world objects.The authors describe the noise as effective yet, in some combinations, stealthy according to target gradients.
- Quantification: The most active 42x42 gradient-fix patch very rarely overlaps the 42x42 noise patch under both MAX and SUM scoring.The analysis covers 2,800 noised images formed from 28 noises applied to 100 test images.
5. Related Work
Prior work largely studies imperceptible image-wide perturbations or visible patterns placed over salient objects. LaVAN instead localizes visible noise away from the source object and uses smaller patches than the closely related adversarial-patch setting.
- Image-wide perturbations: Many earlier attacks use imperceptible noise covering the entire image, including the salient object.This includes attacks on image classification, segmentation, and pose estimation.
- Object-covering attacks: Other localized attacks place visible patterns directly on salient regions such as faces or traffic signs.Examples include patterned glasses for face recognition and rectangular patches on traffic signs.
- LaVAN setting: LaVAN localizes visible pixels to a specific region without covering the source-class object.The paper contrasts this setting with prior object-covering attacks.
- Relation to adversarial patches: Compared with related adversarial patches, LaVAN shows that relaxing physical-printability requirements permits substantially smaller visible-noise regions.The paper also reports that gradients do not identify the attacking patch as the source of misclassification.
6. Conclusions
The paper demonstrates that tiny, visible patches can reliably induce targeted misclassification without covering the main object, while often remaining inconspicuous to the network’s own gradients. This exposes a security concern and motivates further study of network vulnerabilities and robustness.
- 2% of image pixels is sufficient for visible patches to induce misclassification to arbitrary target labels without covering the main salient object.The same patch can be applied across arbitrary images and locations with very high success rates.
- Target-class resemblance appears in patch features such as texture, fur, body parts, and global shape, despite the patch being much smaller than the main object.
- Gradient-based analyses find the noised patch is often not particularly salient to the network compared with other image locations.This result holds even though the patch can resemble the target class and, in the image-domain case, resembles ordinary network stimuli.
- The network’s apparent failure to detect that it is being fooled may make these attacks difficult to protect against in both image-domain and network-domain settings.
- The findings motivate future research on architectural susceptibility and adversarial auxiliary losses for improving robustness.