Generative Adversarial Perturbations

Omid Poursaeed; Isay Katsman; Bicheng Gao; Serge Belongie

Source-linked AI summary

Generative Adversarial Perturbations

Omid Poursaeed, Isay Katsman, Bicheng Gao, Serge Belongie

arXiv:1712.02328v3Dec 6, 2017cs.CVcs.CRcs.LGcs.NEstat.ML

TL;DR

Deep neural networks can be misled by small, natural-looking adversarial perturbations, motivating attacks that are effective across settings without relying on slow iterative inference. The paper trains generative models to produce universal and image-dependent perturbations for targeted and non-targeted attacks across classification and semantic segmentation. It reports high fooling rates with small norms, faster inference than iterative methods, and transfer across target models.

Problem
Deep neural networks are vulnerable to adversarial examples, creating reliability and security concerns for visual tasks.
Method
The paper uses trainable generative neural networks to create universal and image-dependent perturbations for targeted and non-targeted classification and semantic segmentation attacks.
Results
The models achieve high fooling rates with small perturbation norms across four attack types and tasks, while also transferring across target models.
Takeaways & Limitations
Similar generative architectures can fool classification and semantic segmentation models without designing separate attack methods for each task.
Takeaways & Limitations
The universal-perturbation formulation assumes that most natural images are fooled, with an optional relaxation requiring incorrect predictions for most pixels.

Abstract

from arXiv · show

In this paper, we propose novel generative models for creating adversarial examples, slightly perturbed images resembling natural images but maliciously crafted to fool pre-trained models. We present trainable deep neural networks for transforming images to adversarial perturbations. Our proposed models can produce image-agnostic and image-dependent perturbations for both targeted and non-targeted attacks. We also demonstrate that similar architectures can achieve impressive results in fooling classification and semantic segmentation models, obviating the need for hand-crafting attack methods for each task. Using extensive experiments on challenging high-resolution datasets such as ImageNet and Cityscapes, we show that our perturbations achieve high fooling rates with small perturbation norms. Moreover, our attacks are considerably faster than current iterative methods at inference time.

1. Introduction

The paper frames adversarial examples as a security and reliability concern and introduces generative models for universal or image-dependent, targeted or non-targeted attacks. These models improve attack performance while making inference substantially faster than iterative approaches.

Deep neural networks are vulnerable to adversarial examples that resemble natural images but are crafted to mislead pretrained models.The paper identifies potential consequences for visual-task applications and security.
The framework covers universal and image-dependent perturbations for targeted and non-targeted classification and semantic-segmentation attacks.It evaluates perturbations using L∞ and L2 norms.
Generative models replace iterative methods for learning universal perturbations and improve state-of-the-art performance.
Targeted universal perturbations are presented as an effective attack despite requiring one pattern to induce a specific target prediction.The paper describes this setting as especially challenging because both universality and target matching are required.
Perturbations can be generated in the order of milliseconds, making the attacks considerably faster than iterative and optimization-based methods at inference time.

2. Related Work

Prior work generated universal perturbations by iterative aggregation or activation optimization, while image-dependent attacks used gradient-based optimization. These approaches involve limitations including weaker results, slow inference, or restricted attack settings.

Earlier universal attacks iteratively aggregate image-dependent perturbations, while data-independent methods optimize network activations without training data.The data-independent approach avoids training data but reports weaker results than the iterative method.
Optimization-based image-dependent attacks use perturbation-norm and model-loss objectives with repeated gradient updates in pixel space.
FGSM is fast because it takes one gradient-sign step, but its linear approximation often produces sub-optimal results.
Iterative attacks were extended to semantic segmentation and object detection, applying gradient ascent until target correctness reaches zero or a maximum iteration is reached.
The literature also includes defenses against adversarial attacks, although this paper focuses on efficient attack generation.

3. Generative Adversarial Perturbations

The paper formulates adversarial-example generation as trainable mappings that produce universal or image-dependent perturbations for classification and segmentation, under targeted or non-targeted objectives. Generators transform random patterns or input images into norm-constrained perturbations, avoiding iterative inference and supporting attacks against multiple networks.

Problem formulation: Adversarial examples are images close to natural inputs but designed to cause a pretrained network to misclassify them, including pixel-wise errors for semantic segmentation.The framework models adversarial examples using a small distance from a natural image and incorrect predictions; segmentation extends this requirement across pixels.
Generative framework: The framework covers universal and image-dependent perturbations for targeted and non-targeted attacks in classification and semantic segmentation, measured with L∞ and L2 norms.This unifies four attack settings: targeted universal, non-targeted universal, targeted image-dependent, and non-targeted image-dependent perturbations.
Universal perturbations: Universal perturbations are generated by mapping a random pattern through a trainable network, scaling the output to a prescribed norm, and adding it to clipped natural images.The generator approximates a mapping from random patterns to perturbations that fool most natural images without directly depending on a dataset image.
Image-dependent perturbations: Image-dependent attacks use a generator to produce an additive perturbation from each input image, with norm control providing better control over perturbation magnitude.The perturbed image is formed by adding the scaled perturbation to the input and clipping the result before evaluation.
Inference: At inference time, the image-dependent generator operates without the pretrained model, eliminating iterative gradient computations and enabling fast perturbation generation.The training uses losses analogous to the universal case, while the generator alone produces adversarial examples after training.
Multiple networks: A single generative model can be trained to mislead multiple target networks by combining their fooling losses.The paper also reports transferability of generated universal attacks across different models.

4. Experiments on Classification

The classification experiments evaluate generative perturbations across universal and image-dependent, targeted and non-targeted attacks. The models achieve strong fooling performance, transfer across networks, and millisecond-scale inference.

Experimental settings: The experiments cover universal and image-dependent perturbations for targeted and non-targeted classification attacks.ImageNet validation experiments use fixed L2 or L∞ perturbation constraints and pretrained classifiers.
Universal perturbations: For non-targeted universal attacks, the generative approach outperforms [35] in most cases under L2 and L∞ constraints.The L2 threshold is 2000, while the L∞ threshold is 10 for images in the [0, 255] range.
Universal perturbations: 52.0% average target accuracy is achieved for targeted universal perturbations at L∞= 10 across 10 randomly sampled ImageNet classes.The model uses a ResNet generator to fool Inception-v3, and the universal pattern can resemble the target class.
Inference speed: The attacks generate perturbations in the order of milliseconds, addressing the slow inference of iterative and optimization-based methods.Prior attacks may take several minutes per image, whereas the proposed attacks are designed to be both fast and accurate.
Image-dependent perturbations: 89.1% average target accuracy is achieved for targeted image-dependent perturbations at L∞= 10 across 10 randomly sampled classes.The average inference time is 0.28 ms per image for fooling Inception-v3.
Transferability and multiple networks: Joint optimization over VGG-16 and VGG-19 improves transferability compared with training on a single target network.The transferability evaluation trains on one network or jointly on both and tests across models.

5. Experiments on Semantic Segmentation

The segmentation experiments apply related generative architectures to Cityscapes and FCN-8s for targeted and non-targeted attacks. They report strong targeted fooling, lower mean IoU for image-dependent perturbations, and measurable inference times.

Cross-task applicability: The segmentation architectures are presented as generalizable across classification and segmentation, while extension beyond those tasks remains future work.Current segmentation attacks are described as iterative, task-specific, and slow at inference.
Experimental settings: The experiments evaluate generative attacks on FCN-8s using Cityscapes images downsampled from 2048 × 1024 to 1024×512 pixels.The study considers both targeted and non-targeted segmentation attacks with L∞ as the metric.
Universal perturbations: The targeted universal segmentation attack uses a desired target label map and reports success rate as categorical accuracy between prediction and target.The method uses the classification-style training architecture and loss function, with FCN-8s selected for comparison with [34].
Universal perturbations: The proposed targeted universal method outperforms [34] for most perturbation norms on FCN-8s.The generator creates a universal perturbation that resembles the target label map.
Image-dependent perturbations: Relaxing universality leads to higher success rates for targeted image-dependent segmentation perturbations.The image-dependent perturbations contain patterns from both the target and the input image.
Image-dependent perturbations: Image-dependent perturbations achieve smaller mean IoU than universal perturbations in non-targeted segmentation attacks.The reported average inference times are 132.82 ms for U-Net and 335.73 ms for the ResNet generator.

6. Discussion and Future Work

The framework generates four classes of adversarial perturbations and remains effective across classification and semantic segmentation, while supporting transfer across target models. The appendix figures visualize targeted and non-targeted, universal and image-dependent perturbations under several norm settings.

The framework covers targeted and non-targeted attacks with both universal and image-dependent perturbations across classification and semantic segmentation.
The perturbations achieve high fooling rates in the small perturbation norm regime and can transfer across different target models.
Similar generator architectures fool both classification and semantic segmentation models, reducing the need for task-specific attack methods.
Figures 10–13 visualize universal and image-dependent perturbations for targeted and non-targeted attacks, including L2 = 2000 and L∞= 10 settings.

7. Appendix

The appendix examines inference time for image-dependent generators and robustness to Gaussian filtering. The reported results indicate greater blur resistance than I-FGSM for most classification settings and reasonable robustness for segmentation.

Inference time is reported only for image-dependent perturbations because universal perturbations require only addition to the input during inference.
Table 8 reports average inference time per image and generator architectures for image-dependent classification tasks using Inception-v3.
Table 9 reports average inference time per image and generator architectures for targeted image-dependent segmentation perturbations using FCN-8s.
For most σ values, the method is more resistant to Gaussian blur than I-FGSM in non-targeted image-dependent classification perturbations.
The perturbations are reasonably robust to Gaussian blur in the segmentation task.