PaperlayerView exact revision on arXiv

Source-linked AI summary

A General Framework for Adversarial Examples with Objectives

Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter

arXiv:1801.00349v2cs.CVcs.CR

TL;DR

Adversarial-example research typically constrains similarity to the original image, although applications may require additional objectives, including imprecise ones. The paper proposes AGNs, generator networks that emit adversarial examples satisfying desired objectives. Across face and handwritten-digit recognition, AGNs support physical eyeglass attacks with improved robustness, inconspicuousness, and scalability over prior approaches.

  • Problem

    Most adversarial-example research constrains perturbed images mainly by similarity, while applications require additional objectives that prior methods handle through custom modifications.

  • Method

    AGNs train a generator neural network to emit diverse adversarial examples that satisfy multiple desired objectives, including objectives specified through examples.

  • Results

    AGNs generate physical eyeglass attacks with improved robustness, inconspicuousness, and scalability, and also fool a handwritten-digit classifier.

  • Takeaways & Limitations

    AGNs provide a flexible, general approach for generating diverse adversarial examples across face and handwritten-digit recognition, including objectives that are difficult to specify precisely.

  • Takeaways & Limitations

    OpenFace networks remained vulnerable to naïve dodging attempts, and straightforward data augmentation did not improve their robustness.

Abstract

from arXiv · show

Images perturbed subtly to be misclassified by neural networks, called adversarial examples, have emerged as a technically deep challenge and an important concern for several application domains. Most research on adversarial examples takes as its only constraint that the perturbed images are similar to the originals. However, real-world application of these ideas often requires the examples to satisfy additional objectives, which are typically enforced through custom modifications of the perturbation process. In this paper, we propose adversarial generative nets (AGNs), a general methodology to train a generator neural network to emit adversarial examples satisfying desired objectives. We demonstrate the ability of AGNs to accommodate a wide range of objectives, including imprecise ones difficult to model, in two application domains. In particular, we demonstrate physical adversarial examples---eyeglass frames designed to fool face recognition---with better robustness, inconspicuousness, and scalability than previous approaches, as well as a new attack to fool a handwritten-digit classifier.

1 INTRODUCTION

Adversarial examples are easy to find but prior methods usually optimize similarity alone, whereas real applications require additional objectives. The paper introduces AGNs, which generate diverse adversarial examples meeting such objectives, and demonstrates them on physical eyeglasses and face recognition.

  • DNNs can be systematically fooled by mildly perturbed inputs that appear visually indistinguishable from benign images.
  • Prior adversarial-example attacks generally imposed few objectives beyond similarity to the original input.Physical applications may require modeling additional objectives, such as constraints on realizability.
  • AGNs train an attack generator neural network to produce successful adversarial instances satisfying specified objectives.The framework is designed to replace ad hoc objective handling with a general generation process.
  • AGNs generate many diverse adversarial examples, supporting both novel attacks and labeled negative inputs for classifier training.
  • AGNs target robustness, inconspicuousness, and scalability objectives for eyeglasses attacking VGG and OpenFace face-recognition systems.The evaluated robustness settings include changes in lighting and viewing angle, as well as specific defenses.
  • AGNs achieve approximately 70% versus 31% average impersonation success and can accommodate robustness to illumination changes.The paper also reports that inconspicuousness can be specified through labeled examples rather than a mathematical formulation.

2 RELATED WORK

Related work includes optimization-based, universal, physical, non-visual, and defensive approaches to adversarial examples. The paper distinguishes AGNs by targeting multiple, stricter, and potentially imprecise objectives through a general generation method.

  • Early and subsequent attacks sought imperceptible or high-confidence misclassification, often by optimizing perturbation norms, changed pixels, or classification losses.
  • Carlini–Wagner loss increases the target-class probability while decreasing the probabilities of other classes.The loss is defined as Losscw = max{Lc(x + r) : c ≠ ct} − Lct(x + r).
  • Generative attacks related to AGNs create perturbed images with small-norm changes, whereas AGNs impose stricter spatial constraints and multiple objectives.The paper’s examples require adversarial eyeglasses to occupy a small specific region while looking realistic and causing misclassification.
  • Universal perturbations fool many images, while this work explores universal attacks that are both inconspicuous and confined to a small region.
  • Objective-specific digital attacks use customized algorithms for precise outputs, whereas AGNs address objectives that may be imprecise.
  • Prior physical eyeglass attacks used ad hoc objectives, while this paper adds a general framework and user-study evaluation of inconspicuousness.The paper reports improved robustness and inconspicuousness, plus scalability and robustness against defenses.
  • Related defenses train robust classifiers, detect adversarial inputs, or transform inputs to sanitize them.These approaches respectively modify training, classify adversarialness, or obfuscate attack-relevant gradients.

3 A NOVEL ATTACK AGAINST DNNS

This section introduces AGNs, which train neural networks to generate realistic adversarial artifacts that fool a target DNN while satisfying objectives such as inconspicuousness. The framework replaces direct, iterative input modification with generator-based attack production and supports targeted or untargeted misclassification under a white-box threat model.

  • Attack objectives: AGNs support both targeted attacks, which increase a target class probability, and untargeted attacks, which decrease the correct class probability.The classification loss is selected according to the attack type and is maximized during generator training.
  • Threat model: The method assumes white-box access to the target DNN’s feature space, architecture, and parameters, while excluding poisoning or alteration of its training data.The adversary can alter only inputs presented for classification.
  • Framework: The framework replaces iterative tweaking of benign inputs with neural networks that generate adversarial outputs meeting desired objectives.Generated artifacts are designed to resemble a reference set when objectives such as inconspicuousness are difficult to specify precisely.
  • Framework: AGNs train a generator to produce realistic artifact images that mislead a target DNN, unlike GANs, which only learn to generate realistic samples.An AGN combines a generator, discriminator, and pre-trained classifier.
  • Training: Training balances realism and misclassification objectives while updating the generator and discriminator; the target classifier’s weights remain fixed.The discriminator distinguishes realistic from generated samples, whereas the fixed classifier represents the system attacked at test time.

4 AGNS THAT FOOL FACE RECOGNITION

The authors build a printable eyeglass-generation pipeline by filtering and preprocessing real designs, pretraining a generator and discriminator, and evaluating attacks against VGG- and OpenFace-based face-recognition networks.

  • 4.1 Collecting a Dataset of Eyeglasses: 8,340 eyeglass images remained after filtering 26,520 collected images with a classifier trained on 250 hand-labeled examples.The filter achieved 100% precision and 65% recall on the hand-labeled set.
  • 4.1 Collecting a Dataset of Eyeglasses: The authors transferred frame patterns onto a fixed silhouette to simplify alignment with face images and mapped colors into the printer’s gamut for physical realizability.The fixed shape preserves variation in colors and patterns while making alignment more efficient.
  • 4.2 Pretraining the Generator and the Discriminator: They used a Deep Convolutional GAN architecture and selected a 25-dimensional latent space with 64 × 176-pixel output images for sharp, diverse eyeglasses.The generator and discriminator were pretrained for 200 epochs so the generator already produced real-looking eyeglasses.
  • 4.3 DNNs for Face Recognition: The evaluation used four face-recognition DNNs spanning VGG and OpenFace architectures, with small and large models trained for physically realizable dodging and impersonation.The VGG-derived models use fully connected layers and softmax outputs, while OpenFace produces 128-dimensional face descriptors.
  • 4.3 DNNs for Face Recognition: VGG10 and VGG143 reached 100% and 98% held-out accuracy, while naïve dodging succeeded at most 4.60% and impersonation remained below 0.01%.Thresholds preserved those accuracies while reducing both models’ false-positive rates to 0%.
  • 4.3 DNNs for Face Recognition: OpenFace models were more vulnerable to naïve attacks than VGG models, with OF10 and OF143 achieving 100% and 85.50% accuracy, respectively.The authors attribute this difference to the limited capacity of the OpenFace network, which has fewer parameters.

5 EVALUATING AGNS

AGNs were evaluated across digital, physical, universal, transfer, inconspicuousness, luminance-robustness, and digit-recognition attacks. The experiments found strong physical success, universal evasion, asymmetric transferability, and generation of diverse human-comprehensible adversarial digits.

  • 5.1 Attacks in the Digital Domain: Detector-aware AGN attacks retained similar dodging and impersonation success rates, but detector evasion reduced attack inconspicuousness.This separates classifier attack success from the visual inconspicuousness objective.
  • 5.2 Attacks in the Physical Domain: 97% versus 67%: AGNs achieved higher physical-domain dodging success than CCS16, while impersonation reached 70% versus 31%.The differences were statistically significant for dodging (p = 0.03) and impersonation (p < 0.01).
  • 5.2 Attacks in the Physical Domain: 96% average dodging success under luminance changes showed robustness comparable to changing-pose conditions.The reported impersonation success rate under changing luminance was 61%.
  • 5.3 Universal Dodging Attacks: 92% and 94% of remaining subjects evaded VGG143 and OF143 when generators trained on 100 subjects produced 10 eyeglasses.Training on five subjects and generating five eyeglasses allowed more than 50% of remaining users to dodge either network.
  • 5.4 Transferability: Transfer was asymmetric: OpenFace-to-VGG dodging succeeded in only 10–12% of digital attempts, whereas VGG-to-OpenFace succeeded in at least 63%.Physical transfer showed the same directionality, at 20–28% versus 44–52%.
  • 5.6 AGNs Against Digit Recognition: 5,004 adversarial digits were retained from 600,000 generated images after misclassification and human-comprehensibility filtering.The DNN misclassified 8.34% of generated samples before the comprehensibility filter.

6 DISCUSSION AND CONCLUSION

AGNs provide a general methodology for generating adversarial examples that satisfy additional objectives across physical and nonphysical recognition domains. They improve physical eyeglass attacks across robustness, inconspicuousness, and scalability while producing diverse examples useful for robustness evaluation and defense.

  • AGNs improve physical eyeglass attacks over prior work in robustness, inconspicuousness, and scalability.The demonstrated robustness includes changes in imaging conditions and specific defenses, while scalability concerns the number of eyeglasses needed in different contexts.
  • AGNs accommodate objectives ranging from precisely describable constraints to imprecise objectives such as inconspicuousness.The framework can be trained from examples describing a desired objective.
  • AGNs apply across face and handwritten-digit recognition and may extend to other applications.The paper specifically demonstrates AGNs on face and handwritten-digit classifiers; broader applications are presented as an expectation.
  • AGNs generate multiple diverse adversarial examples for one benign sample, supporting model-robustness evaluation and potentially adversarial training.The passage identifies diversity as useful for evaluating robustness and for incorporating examples into defenses.
Loading 1801.00349v2…