PaperlayerView exact revision on arXiv

Source-linked AI summary

NAG: Network for Adversary Generation

Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, R. Venkatesh Babu

arXiv:1712.03390v2cs.CVcs.AIcs.LG

TL;DR

Existing approaches usually produce one adversarial perturbation at a time, leaving the distribution of perturbations insufficiently explored. NAG uses a GAN-inspired generator trained with fooling and diversity objectives to model that distribution for a classifier. The resulting perturbations achieve state-of-the-art fooling rates, broad variety, and excellent cross-model generalizability.

  • Problem

    Existing methods typically generate a single perturbation for a classifier, while exploring the broader adversarial-perturbation manifold is important for understanding transferability and building robust models.

  • Method

    NAG trains a GAN-inspired generator with fooling and diversity objectives to model the unknown distribution of adversarial perturbations for a given classifier.

  • Results

    The generated perturbations achieve state-of-the-art fooling rates, exhibit wide variety, and show excellent cross-model generalizability.

  • Takeaways & Limitations

    Modeling the perturbation distribution provides a way to study adversarial manifolds and generate diverse adversarial examples for robust-model training.

Abstract

from arXiv · show

Adversarial perturbations can pose a serious threat for deploying machine learning systems. Recent works have shown existence of image-agnostic perturbations that can fool classifiers over most natural images. Existing methods present optimization approaches that solve for a fooling objective with an imperceptibility constraint to craft the perturbations. However, for a given classifier, they generate one perturbation at a time, which is a single instance from the manifold of adversarial perturbations. Also, in order to build robust models, it is essential to explore the manifold of adversarial perturbations. In this paper, we propose for the first time, a generative approach to model the distribution of adversarial perturbations. The architecture of the proposed model is inspired from that of GANs and is trained using fooling and diversity objectives. Our trained generator network attempts to capture the distribution of adversarial perturbations for a given classifier and readily generates a wide variety of such perturbations. Our experimental evaluation demonstrates that perturbations crafted by our model (i) achieve state-of-the-art fooling rates, (ii) exhibit wide variety and (iii) deliver excellent cross model generalizability. Our work can be deemed as an important step in the process of inferring about the complex manifolds of adversarial perturbations.

1. Introduction

Universal adversarial perturbation methods typically learn one perturbation, leaving the broader adversarial manifold underexplored. NAG addresses this gap with a GAN-inspired generator trained to model that distribution while producing diverse, transferable perturbations.

  • Motivation: Universal adversarial perturbations can fool many images and networks, creating risks for deployed systems and motivating robust-model development.Their image-agnostic and cross-model behavior is especially relevant to safety- and privacy-sensitive applications.
  • Motivation: Existing methods generally learn only a single perturbation for a classifier, despite evidence that adversarial perturbations occupy broader contiguous regions.This limits exploration of the perturbation manifold and motivates distribution modeling.
  • Approach: NAG introduces a GAN-inspired generative model that captures the unknown distribution of adversarial perturbations for a given classifier.The framework avoids parametric assumptions and does not require known samples from the target perturbation distribution.
  • Contributions: The proposed fooling objective achieves state-of-the-art fooling performance across multiple CNN architectures trained for object recognition.The contribution is presented as a novel objective for crafting universal adversarial perturbations.
  • Contributions: The learned model captures perturbation distributions and generates diverse perturbations with high fooling capacity and excellent cross-model generalizability.These properties support studying adversarial manifolds and generating varied examples for robust-model training.

2. Proposed Approach

The proposed approach uses a GAN-like generator to model a classifier’s distribution of adversarial perturbations, replacing GAN discrimination with classifier-based fooling and diversity objectives.

  • 2. Proposed Approach: A GAN-like generator maps latent vectors to adversarial perturbations for a given classifier, modeling their unknown distribution without parametric assumptions.The target CNN replaces the discriminator and remains fixed while the generator is trained.
  • 2. Proposed Approach: The fooling objective reduces the target classifier’s confidence in benign predictions on perturbed inputs, encouraging label flips.It uses the target CNN’s predictions on clean and perturbed samples to train the generator.
  • 2. Proposed Approach: The diversity objective separates representations produced by different generated perturbations at a classifier layer, discouraging the generator from learning only a limited subset.For paired perturbations, the method increases feature distance, with the final loss combining fooling and diversity terms.
  • 2. Proposed Approach: The final training loss gives equal importance to fooling and diversity objectives through λ = 1.The loss is defined as the summation of Lf and λLd.
  • 2. Proposed Approach: The generator uses five deconvolutional layers, while tanh output scaling restricts perturbations to the prescribed range for quasi-imperceptible noise.Experiments keep the generator architecture unchanged across target CNNs while learning separate adversarial distributions.
  • 2. Proposed Approach: Training forms benign, adversarial, and shuffled adversarial batches, using shuffled perturbations to compute the diversity component through target-CNN representations.Only perturbations are shuffled; the data samples remain fixed.

3. Experiments

Experiments evaluate the generator’s fooling rates, data dependence, diversity, latent-space smoothness, and performance when modeling perturbations for multiple target CNNs. The model outperforms UAP on mean fooling rate, produces diverse perturbations, and achieves strong multi-model and black-box fooling.

  • 3.1. Perturbations and the fooling rates: 65.68 mean fooling rate versus 57.66 for UAP, an 8% improvement across the reported attack entries.The generator exceeded UAP in 33 of 36 entries where UAP reported fooling rates.
  • 3.3. Diversity of perturbations: Samples from the learned distributions varied visually across target CNNs and across repeated draws from the same generator.The multi-target examples use perturbations sampled from a distribution learned to fool all seven target CNNs.
  • 3.1. Perturbations and the fooling rates: Increasing training data raised white-box and black-box fooling rates for the ResNet-152 model.Black-box rates were averaged over GoogLeNet, VGG-19, and ResNet-50.
  • 3.3. Diversity of perturbations: 48.6% relative higher diversity than UAP was observed, with 50 categories accounting for 95% of predicted labels versus 173 for UAP.The analysis attributes this diversity to the proposed diversity loss encouraging exploration of the adversarial manifold.
  • 3.5. Modelling adversaries for multiple targets: Interpolating between latent points produced smoothly changing perturbations whose intermediate samples retained high fooling capability.Ten intermediate points were evaluated for ResNet-152 over 50,000 ILSVRC 2014 validation images.
  • 3.5. Modelling adversaries for multiple targets: 80.07% average fooling rate was achieved across seven target CNNs using perturbations sampled from a jointly learned distribution.This was around 8% higher than the best mean fooling rate of 72.62% obtained by an individual network.
  • 3.6. Black-box attacks for ensemble generator: The ensemble generator achieved higher mean black-box fooling rates than generators trained for individual target CNNs on three held-out models.The ensemble was trained on VGG-F, GoogLeNet, VGG-16, and ResNet-50, while CaffeNet, VGG-19, and ResNet-152 were held out.

4. Related Works

Prior work established that structured and image-agnostic adversarial perturbations can threaten deep classifiers and support black-box attacks. Building on GAN-based distribution learning, this paper models universal adversarial perturbations for a target classifier.

  • Image-agnostic perturbations and cross-model generalization expose vulnerabilities in deep neural networks and enable black-box attacks.
  • GANs provide a framework for learning data distributions and generating samples, motivating the proposed adversarial-perturbation generator.
  • The proposed architecture models the distribution of universal adversarial perturbations for a given classifier using a GAN-inspired framework.

5. Conclusion

The paper presents a first generative approach for learning the distribution of adversarial perturbations without training samples from the unknown target distribution. Its generated perturbations achieve strong fooling rates and cross-model generalization while supporting robustness studies.

  • The framework is presented as the first generative approach to model adversarial-perturbation distributions for a given CNN classifier.
  • The generator captures an unknown target distribution without training samples from that distribution.
  • The method achieves state-of-the-art fooling rates and strong cross-model generalizability, indicating severe susceptibilities in current deep learning models.
  • Modeling the perturbation distribution enables formal studies aimed at building robust systems.

1. Traversing the manifold of perturbations

The learned generator supports smooth traversal through the manifold of adversarial perturbations by mapping interpolated latent vectors to perturbations. The demonstrated perturbations target GoogLeNet and are evaluated by fooling rates over ILSVRC validation images.

  • Ten intermediate latent points between random z1 and z2 are mapped through the learned generator into perturbations for GoogLeNet.
  • Interpolation in latent space produces perturbations with smooth transitions, demonstrating structure in the learned distribution.
  • The generated perturbations are evaluated on 50000 ILSVRC 2014 validation images using their corresponding fooling rates.

2. Generator details

The generator maps latent-space vectors to image-sized perturbations through a fully connected layer followed by several deconvolutional layers. Its architecture is adapted from improved GAN-training techniques.

  • The generator maps latent-space vectors to image-sized perturbations using an initial fully connected layer and several deconvolutional layers.
  • The architecture is adapted from prior techniques designed to improve GAN training.

3. Modelling adversaries for Multiple target classifiers

The learned generator models perturbations that can simultaneously fool multiple target classifiers and captures substantial visual diversity. Its architecture transforms a latent vector into 224 × 224 perturbations.

  • The multi-target generator captures perturbations that simultaneously fool seven target CNNs while exhibiting wide visual variety.
  • The learned distribution provides sample perturbations for the multi-target setting rather than a single fixed perturbation.
  • The generator network transforms a 10-dimensional latent vector z into a perturbation of size 224 × 224.

4. Representations for diversity objective

The diversity objective separates adversarial and shuffled-adversarial feature representations while preserving fooling performance. Softmax representations were selected because they captured relatively high variation visually.

  • The diversity objective indirectly separates perturbation pairs by projecting adversarial and shuffled-adversarial feature representations apart.Fixed target-CNN transformations guide the generator toward diverse perturbations that still fool the target.
  • Softmax, conv4, and conv5 representations produced nearly identical fooling rates, differing by less than 2%.The optimization was repeated 10 times, with low standard deviations across representations.
  • Softmax representations were chosen for the full model because visual inspection showed relatively high variation in the captured perturbations.

5. Sample benign and corresponding perturbed images

Different generated perturbations produce different predicted labels when added to the same or different benign images. The multi-target generator demonstrates this diversity across four perturbations and across target-classifier settings.

  • Different perturbations added to CaffeNet images resulted in different predicted labels for the corresponding adversarial images.
  • Increasing the fooling-objective weight raised fooling rate from 76.26% to 89.04%, while dominant labels decreased from 348 to 253.The results used ResNet-50 on ILSVRC-2014 validation images and show the trade-off between fooling and diversity.
  • Four perturbations added to one benign image produced four different predicted labels under GoogLeNet.The generator was trained to fool all seven target CNNs.
Loading 1712.03390v2…