PaperlayerView exact revision on arXiv

Source-linked AI summary

Synthesizing Robust Adversarial Examples

Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok

arXiv:1707.07397v3cs.CV

TL;DR

Standard adversarial examples often fail under real-world transformations, leaving their physical robustness uncertain. The paper introduces EOT to synthesize examples over chosen transformation distributions and applies it to printed 3D objects. Its results demonstrate robust adversarial objects in the physical world.

  • Problem

    Standard adversarial examples often lose their adversarial effect under viewpoint changes and other natural transformations, limiting evidence of their physical-world robustness.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    EOT optimizes adversarial examples over a chosen transformation distribution, including transformations that model 3D rendering.

    Source: §1.2 · p. 2, §2.1 · p. 2

  • Results

    The authors fabricate 3D adversarial objects that a standard ImageNet classifier classifies as chosen target classes across varied angles, viewpoints, and lighting conditions.

    Source: §1 · p. 1, §5 · p. 8

  • Takeaways & Limitations

    The results demonstrate that robust adversarial examples and objects exist in the physical world.

    Source: §5 · p. 8

    EOT can fail when the allowed perturbation is too small or the chosen transformation distribution is too broad.

    Source: §3.5 · p. 7

Abstract

from arXiv · show

Standard methods for generating adversarial examples for neural networks do not consistently fool neural network classifiers in the physical world due to a combination of viewpoint shifts, camera noise, and other natural transformations, limiting their relevance to real-world systems. We demonstrate the existence of robust 3D adversarial objects, and we present the first algorithm for synthesizing examples that are adversarial over a chosen distribution of transformations. We synthesize two-dimensional adversarial images that are robust to noise, distortion, and affine transformation. We apply our algorithm to complex three-dimensional objects, using 3D-printing to manufacture the first physical adversarial objects. Our results demonstrate the existence of 3D adversarial objects in the physical world.

1. Introduction

Standard adversarial examples often lose their effect under viewpoint changes and natural distortions, leaving their physical-world robustness uncertain. The paper introduces EOT and applies it to 2D and 3D examples, including fabricated physical adversarial objects.

1.1. Challenges

  • Standard adversarial examples often fail under varying viewpoints, lighting, and camera noise, while prior physical-world work had not clearly generalized to 3D objects.The 3D setting adds rotation and perspective projection beyond the affine transformations used to approximate viewpoints for 2D images.Source: §1 · p. 1, §1.1 · p. 2

1. Introduction

  • The paper demonstrates robust physical-world adversarial objects, including a 3D-printed turtle classified as a rifle across viewpoints.The target class for the turtle was selected at random.Source: §1 · p. 1

1.2. Contributions

  • Expectation Over Transformation (EOT) synthesizes examples adversarial across a chosen distribution of transformations.The paper applies EOT to 3D rendering and fabricates physical objects that fool classifiers in the physical world.Source: §1.2 · p. 2

2. Approach

The approach optimizes adversarial examples over a chosen distribution of transformations rather than a single untransformed input. It extends this framework from 2D distortions to rendered 3D objects, with an objective that constrains expected perceived distance.

2. Approach

  • The paper combines EOT with differentiation through the 3D rendering process to generate adversarial objects end to end.EOT is presented as a general framework for examples robust over a chosen transformation distribution.Source: §2 · p. 2

2.1. Expectation Over Transformation

  • EOT models a chosen distribution of transformations and constrains the expected effective distance between transformed adversarial and original inputs.The distance is defined from the classifier’s perspective, which matters when transformations change the input’s domain or codomain, such as rendering a texture.Source: §2.1 · pp. 2–3

2.2. Choosing a distribution of transformations

  • The EOT framework leaves the transformation distribution, distance metric, and optimization method open to choice.The paper uses this flexibility to generate 2D examples, 3D models, and physical-world adversarial objects.Source: §2.2 · p. 3

2.2.1. 2D CASE

  • For 2D examples, the transformation distribution approximates distortions from printing and photographing an image using random affine transformations.These transformations are differentiable, allowing straightforward application of EOT.Source: §2.2.1 · p. 3

2.2.2. 3D CASE

  • For 3D examples, EOT transforms an object’s texture into rendered poses that incorporate rendering, lighting, rotation, translation, and perspective projection.The rendering is represented as Mx + b, and the renderer supplies the coordinate map and background for each pose.Source: §2.2.2 · p. 3

2.3. Optimizing the objective

  • The objective uses expected transformed-space distance, with LAB-space ℓ2 distance as a proxy for perceptual distance, and is optimized with projected gradient descent.The Lagrangian formulation constrains the search space without requiring a complex projection.Source: §2.3 · p. 3

3. Evaluation

The evaluation tests EOT across simulated 2D and 3D transformations and physical-world effects, finding robust adversarial examples in all three settings. Its discussion qualifies that robustness depends on the chosen transformation distribution and that physical printing introduces color error.

3.1. Procedure

  • The procedure measures adversariality as the probability of classification as the target under a sampled transformation, alongside source-class accuracy and per-pixel ℓ2 distance.Simulation evaluations use randomly sampled transformations; physical evaluations use manually captured images across viewpoints.Source: §3.1 · p. 4

3.2. Robust 2D adversarial examples

  • 96.4% mean adversariality was achieved across 1,000 2D examples evaluated over 1,000 sampled transformations each.The transformations include rescaling, rotation, brightness changes, Gaussian noise, and translation; a fixed Lagrangian parameter constrained visual similarity.Source: §3.2 · p. 4

3.3. Robust 3D adversarial examples

  • Across 200 adversarial examples evaluated using 100 random transformations sampled from the distribution per example, EOT produced objects with mean adversariality of 83.4% and a long left tail, showing it usually produces highly adversarial objects.Source: §3.3 · p. 5

3.4. Physical adversarial examples

  • Physical-world EOT approximates rather than captures the true transformation distribution, incorporating camera noise, lighting effects, and printing color inaccuracies.Commercial full-color printing showed variable color accuracy between prints.Source: §3.4 · p. 5
  • Across 100 photos of each object, both printed adversarial objects were classified as their target class in a majority of viewpoints, and proxy-optimized objects remained adversarial under real physical conditions.The proxy distribution models camera noise, lighting, and per-channel color inaccuracies, among other transformations.Source: §3.4 · pp. 5–6, Table 3 · p. 6

3.5. Discussion

  • Robustness depends on the chosen transformation distribution: larger distributions generally require larger perturbations, and robust 3D examples generally need larger texture changes than 2D examples.The algorithm provides no guarantees for observations outside the chosen distribution.Source: §3.5 · pp. 6–7
  • Although printing introduces significant color error, modeling it coarsely allowed EOT to produce robust physical-world objects; higher-fidelity printing could reduce required ℓ2 perturbation.The paper also notes that generated perturbations are not always imperceptible, though the visual constraint preserves identifiability as the original class.Source: §3.5 · p. 7
  • When the target class is not top-ranked, the classifier often predicts a semantically similar class rather than the source class.For example, the turtle-to-rifle object was often labeled “revolver,” “holster,” or “assault rifle.”Source: §3.5 · p. 7

4. Related Work

The related work covers single-viewpoint adversarial attacks, defenses, and physical-world examples. It distinguishes EOT’s goal of robustness over transformations from methods targeting other forms of universality or fixed photo sets.

4.1. Adversarial examples

  • Earlier white-box attacks target single-viewpoint inputs, while black-box approaches use substitute models or gradient estimation; universal perturbations instead generalize across images, not transformations.The authors report that universal perturbations, like single-image perturbations, are not inherently robust to transformations.Source: §4.1 · pp. 7–8

4.2. Defenses

  • Many proposed defenses remain vulnerable to iterative optimization-based attacks, and some input-transformation defenses can be circumvented through EOT.Source: §4.2 · p. 8

4.3. Physical-world adversarial examples

  • Kurakin et al. (2016) showed printed-image attacks under approximately axis-aligned views, while Sharif et al. (2016) optimized over head-on portraits and addressed printability with a color map.Their approaches did not provide EOT’s robustness over transformations, and Kurakin et al. did not synthesize targeted examples or extend to 3D.Source: §4.3 · p. 8
  • Evtimov et al. (2017) used manually captured photos for 2D attacks, while EOT was also applied to adversarial patches and real-world attacks have been shown for speech-to-text.Evtimov et al.’s approach has no clear 3D translation and requires preprocessing many photos for each example.Source: §4.3 · p. 8

5. Conclusion

The paper presents EOT as a general-purpose method for adversarial examples robust over chosen transformation distributions. It reports fabricating 3D objects that a standard ImageNet classifier assigns to a chosen target across varied viewpoints and lighting.

  • Low-cost 3D printing produced physical adversarial objects classified as a chosen target across varied angles, viewpoints, and lighting by a standard ImageNet classifier.The authors conclude that such examples and objects are a practical concern for real-world systems.Source: §5 · p. 8

A. Distributions of Transformations

The appendix specifies the transformation distributions used for the 2D, 3D, and physical-world cases.

  • The 2D, 3D, and physical-world experiments use specified transformation-distribution parameters, reported in Tables 4–6.The passage does not provide the parameter values themselves.Source: A. Distributions of Transformations · p. 11

B. Robust 2D Adversarial Examples

The section presents a random sample of the paper’s two-dimensional adversarial examples.

C. Robust 3D Adversarial Examples

The section presents samples of 3D adversarial examples and a histogram summarizing adversariality across all 200 examples.

D. Physical Adversarial Examples

The section documents physical 3D adversarial objects with photographs of a turtle and baseball, alongside example images and transformation distributions for 2D and 3D cases.

Loading 1707.07397v3…