Source-linked AI summary
Foveation-based Mechanisms Alleviate Adversarial Examples
Yan Luo, Xavier Boix, Gemma Roig, Tomaso Poggio, Qi Zhao
TL;DR
Adversarial examples expose failures of CNNs on visually imperceptibly perturbed images. The paper revises the global-linearity account using ImageNet analyses and tests foveation-based processing, finding that foveation nearly restores accuracy without perturbations. The authors attribute this to robustness to object scale and position not generalizing to perturbation transformations.
Problem
Adversarial examples are visually imperceptible perturbations that cause CNNs to fail, while the hypothesis that CNNs act as linear classifiers requires revision for ImageNet settings.
Method
The paper analyzes ImageNet CNNs for local linearity and applies foveation mechanisms that select image regions while discarding other regions.
Results
Foveation almost recovers the accuracy of CNNs without adversarial perturbations across clutter conditions and foveation types, including perturbations calculated with foveation considered.
Takeaways & Limitations
Foveation reduces the perturbation’s effect because robustness to object scale and position does not generalize to transformations of the perturbation.
Takeaways & Limitations
The conclusion assumes foveation does not negatively affect CNN performance and notes that human perception’s robustness to these perturbations remains unresolved.
Abstract
from arXiv · showhide
We show that adversarial examples, i.e., the visually imperceptible perturbations that result in Convolutional Neural Networks (CNNs) fail, can be alleviated with a mechanism based on foveations---applying the CNN in different image regions. To see this, first, we report results in ImageNet that lead to a revision of the hypothesis that adversarial perturbations are a consequence of CNNs acting as a linear classifier: CNNs act locally linearly to changes in the image regions with objects recognized by the CNN, and in other regions the CNN may act non-linearly. Then, we corroborate that when the neural responses are linear, applying the foveation mechanism to the adversarial example tends to significantly reduce the effect of the perturbation. This is because, hypothetically, the CNNs for ImageNet are robust to changes of scale and translation of the object produced by the foveation, but this property does not generalize to transformations of the perturbation. As a result, the accuracy after a foveation is almost the same as the accuracy of the CNN without the adversarial perturbation, even if the adversarial perturbation is calculated taking into account a foveation.
1 INTRODUCTION
The paper revises the hypothesis that adversarial examples arise because CNNs behave globally as linear classifiers. It proposes that CNNs are locally linear around recognized objects and shows that foveation-based mechanisms can substantially alleviate adversarial examples.
- Adversarial examples are visually imperceptible perturbations that can make CNNs misclassify an image’s object category.
- The paper challenges the global-linearity hypothesis by analyzing several ImageNet CNN architectures that do not assume fixed object position or scale.
- CNNs act locally linearly in image regions containing recognized objects, while they may act non-linearly in other regions.
- Foveation selects an image region for CNN processing and discards information from the remaining regions.
- Across different clutter conditions and foveation types, foveation almost recovers the CNN’s accuracy without adversarial perturbations, even when perturbations account for foveation.
2 EXPERIMENTAL SET-UP
The experiments use ImageNet validation images and three pretrained CNN architectures, generating minimum-norm adversarial perturbations with BFGS and Sign methods. The setup also examines perturbation perceptibility, spatial position, and qualitative differences between perturbation types.
- Dataset: The evaluation uses 50,000 ILSVRC 2012 validation images across 1,000 object categories, with top-5 prediction error as the accuracy measure.
- CNN Architectures: Experiments evaluate AlexNet, GoogLeNet, and VGG using publicly available pretrained Caffe models.
- Generation of Adversarial Perturbations: Adversarial perturbations are approximate minimum-norm perturbations that induce top-5 misclassification, generated with BFGS and Sign algorithms.
- Generation of Adversarial Perturbations: BFGS perturbations concentrate around the object, whereas Sign perturbations spread across the image, and each CNN architecture produces a different perturbation.
- Perceptibility and Position: Perturbation perceptibility is assessed by varying L1 norm per pixel and L∞ norm until perturbations become visible and begin occluding the target object.
- Perceptibility and Position: The target object’s position is represented using its manually annotated ILSVRC 2012 ground-truth bounding box.
3 REVIEW OF THE LINEARITY OF CNNS FOR IMAGENET
ImageNet experiments show that adversarial effects depend on the generating architecture and perturbation location, challenging a global linear-classifier account. The results support local linearity around recognized objects and nonlinearity elsewhere.
- ImageNet’s varied object scales and positions motivate revisiting adversarial-example hypotheses developed in settings with more fixed object placement.
- The review varies perturbation norms and compares perturbations generated on one CNN architecture with their effects on another.
- Adversarial-example properties: Increasing perturbation norm generally causes more misclassification, while BFGS produces more misclassification than Sign for L1 norm per pixel.
- Adversarial-example properties: When perturbations remain no more than slightly perceptible, accuracy is between 10% and 20%, depending on the CNN, and some images lack imperceptible adversarial perturbations.
- Adversarial-example properties: Perturbations generated for the evaluated CNN cause the most misclassification, while perturbations transferred from another architecture are less damaging; constrained accuracy is between 40% and 60%.
- Role of the Target Object Position: Masking perturbations to the object reduces accuracy similarly to the full perturbation, whereas background-only masking decreases accuracy significantly less, by about 40% of accuracy.
- Local Linearity: The local-linearity test finds f(ϵ⋆) unrelated to f(x + ϵ⋆) − f(x), although some local linearity appears near the clean image along the perturbation direction.
- Local Linearity: The proposed hypothesis states that CNNs are locally linear around recognized-object regions but may be non-linear elsewhere.
4 FOVEATION-BASED MECHANISMS ALLEVIATE ADVERSARIAL EXAMPLES
Foveation selects image regions for CNN processing, reducing the perturbation’s effect while preserving object-related responses. Across cluttered and uncluttered settings, foveations substantially restore adversarial-example accuracy, including when perturbations account for foveation.
- Foveation mechanism: Foveation selects a region for CNN processing, discarding other image information while potentially changing object scale and position.The transformed input retains a fixed size even when the selected region varies.
- Underlying hypothesis: The method assumes object representations remain robust to foveation-induced scale and position changes, whereas this robustness does not generalize to perturbations.Consequently, foveation reduces perturbation alignment without comparably harming clean-image alignment.
- Results: Foveation produces significant accuracy improvements, and the effects of clutter removal or averaging multiple foveations are smaller than the effect predicted by Hypothesis 2.This comparison supports attributing the improvement primarily to reduced perturbation effectiveness.
- Results: Accuracy after foveation is almost the same as without adversarial perturbation, improving from 0% to more than 70% in all reported cases.Object Crop MP gives the largest improvement, likely because it preserves the target object while removing clutter.
- Results: For imperceptible or nearly imperceptible perturbations, foveations improve accuracy by 30% to 40% in both setups, but larger perturbations reduce accuracy.The hypothesis assumes operation in a linear region; larger perturbations can enter nonlinear regions or occlude the object.
- Results: A different foveation can again improve accuracy even when the adversarial perturbation was calculated for a foveated input.1 Shift MP-Object and Embedded MP-Object substantially improve perturbations calculated for MP-Object.
5 CONCLUSIONS
The paper concludes that CNNs behave as high-dimensional linear classifiers near recognized objects, while foveation reduces adversarial perturbations because object-transformation robustness does not extend to perturbations.
- CNNs act as high-dimensional linear classifiers in the vicinity of images containing objects recognized by the CNN.
- Foveation decreases adversarial perturbations’ effects on classification scores.
- CNN robustness to object transformations does not generalize to transformations of adversarial perturbations.
- The authors suggest that systems integrating information from multiple fixations may be more robust to adversarial examples.
A.1 THE LINEARITY OF CNNS FOR IMAGENET
ImageNet experiments revise the always-linear CNN hypothesis: local linearity depends on the image region, and the proposed linear approximation can fit classification-score changes better than global linearity.
- ImageNet results with Sign perturbations show the same tendency previously observed for BFGS perturbations.
- Changing perturbation placement and norm reveals accuracy patterns for BFGS and Sign across different CNN architectures.
- For recognized-object positions, f(ϵ⋆) is unrelated to f(x + ϵ⋆) − f(x), challenging the hypothesis that CNNs are always too linear.
- The local-linearity approximation has much smaller error than the hypothesis that CNNs are always linear classifiers.
A.2 FOVEATION-BASED MECHANISMS ALLEVIATE ADVERSARIAL EXAMPLES
Foveation substantially alleviates adversarial examples across perturbation strengths and evaluation settings, although stronger perturbations reduce accuracy and require larger norms to cause misclassification after foveation.
- Removing clutter or averaging multiple foveations has a much smaller effect than the effect attributed to Hypothesis 2.
- Increasing perturbation norm decreases post-foveation accuracy because occlusions may move the CNN into a nonlinear region.
- After foveation, the perturbation norm needed for misclassification increases by about 5 times for BFGS and 5 to 8 times for Sign.
- The figures evaluate accuracy across CNNs while varying perturbation norms, masked perturbation placement, local linearity, and foveation configurations.
- Foveation mechanisms are evaluated using top-5 accuracy with minimum perturbations.
B GENERATION OF THE ADVERSARIAL PERTURBATION
The paper defines adversarial perturbations as minimum-norm image changes that cause misclassification and generates them with L-BFGS-based BFGS or gradient-sign procedures.
- An adversarial perturbation ϵ causes misclassification when added to image x, and x + ϵ is the adversarial example.
- The minimum perturbation ϵ⋆ minimizes the selected norm over all perturbations that cause misclassification.
- BFGS minimizes the L1 norm plus a loss function using box-constrained L-BFGS and back-propagated gradients.
- The BFGS procedure further uses line search to reduce the norm of the perturbation returned by L-BFGS.
- Sign perturbations use sign(∇loss(ϵ, {x, ℓ})) and then apply line search to minimize the norm needed for misclassification.
C PERCEPTIBILITY OF THE ADVERSARIAL PERTURBATION
The section uses qualitative examples to assess when adversarial perturbations become perceptible. Visibility depends on perturbation method and norm, with thresholds used in subsequent analysis.
- Figures 15–20 vary perturbation magnitude using L1 norm per pixel and L∞ norm to illustrate perceptibility.Figures 15, 17, and 19 show L1 changes; Figures 16, 18, and 20 show L∞ changes.
- For BFGS, perturbations become visible above 15 in L1 norm per pixel and 100 in L∞ norm.The difference is attributed to BFGS perturbation density being concentrated mainly in the target-object region.
- For Sign, perturbations become visible at about 15 for both norms because they are spread evenly across the image.
- The figures identify perturbations by the generating network—AlexNet, GoogLeNet, or VGG—and by the BFGS or Sign method.