PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Pixel Norm-Balls: Parametric Adversaries using an Analytically Differentiable Renderer

Hsueh-Ti Derek Liu, Michael Tao, Chun-Liang Li, Derek Nowrouzezahrai, Alec Jacobson

arXiv:1808.02651v2cs.LGcs.CVcs.GRstat.ML

TL;DR

Pixel norm-balls measure perturbations that are divorced from real-world image variation. The paper perturbs lighting and geometry through a physically based differentiable renderer, reporting realistic parametric adversaries and their utility for adversarial data augmentation evaluated against real photographs.

  • Problem

    Pixel norm-balls rely on pixel differences that are divorced from real-world variation, limiting their coverage of real-world security scenarios.

    Source: §1 · p. 2

  • Method

    The approach perturbs physical image-formation parameters, including lighting and geometry, using a differentiable renderer that propagates pixel gradients to parametric space.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Parametric perturbations yield realistic images and support adversarial data augmentation, which the authors evaluate against real photographs for robustness to real-world deformations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Rendered adversarial data augmentation offers an initial route toward testing and training for perturbations that can occur naturally in the real world.

    Source: §1 · p. 2

    The renderer assumes locally constant texture, and the authors identify materials, visibility changes, and shape skeletons as extensions for future work.

    Source: §6 · p. 9

Abstract

from arXiv · show

Many machine learning image classifiers are vulnerable to adversarial attacks, inputs with perturbations designed to intentionally trigger misclassification. Current adversarial methods directly alter pixel colors and evaluate against pixel norm-balls: pixel perturbations smaller than a specified magnitude, according to a measurement norm. This evaluation, however, has limited practical utility since perturbations in the pixel space do not correspond to underlying real-world phenomena of image formation that lead to them and has no security motivation attached. Pixels in natural images are measurements of light that has interacted with the geometry of a physical scene. As such, we propose the direct perturbation of physical parameters that underly image formation: lighting and geometry. As such, we propose a novel evaluation measure, parametric norm-balls, by directly perturbing physical parameters that underly image formation. One enabling contribution we present is a physically-based differentiable renderer that allows us to propagate pixel gradients to the parametric space of lighting and geometry. Our approach enables physically-based adversarial attacks, and our differentiable renderer leverages models from the interactive rendering literature to balance the performance and accuracy trade-offs necessary for a memory-efficient and scalable adversarial data augmentation workflow.

1 INTRODUCTION

The introduction argues that pixel norm-balls poorly represent real-world image variation because they permit pixel changes divorced from image formation. It proposes parametric norm-balls and an analytically differentiable renderer to perturb lighting and geometry while preserving realistic images.

1 INTRODUCTION

  • Pixel norm-balls constrain pixel-value differences but capture only limited real-world security scenarios because their perturbations are divorced from real-world variation.The paper argues that defenders trained this way may not generalize beyond unrealistic pixel-bounded attacks.Source: §1 · p. 2
  • Parametric norm-balls perturb physical image-formation parameters rather than pixel colors, enabling larger perturbations without invalidating image realism.The approach uses a synthetic image-formation model and targets lighting and geometry.Source: §1 · p. 2
  • An analytically differentiable, physically based renderer computes pixel-color derivatives with respect to image-formation parameters, including lighting and geometry.The authors report that it is fast and scalable enough for rendered adversarial data augmentation.Source: §1 · p. 2
  • The authors present rendered adversarial data augmentation as an early step toward real-world use, evaluating training on computer-generated adversaries against real photographs.They report increased classifier robustness to real-world deformations.Source: §1 · p. 2

2 RELATED WORK

Related work establishes the use of rendered data in real-world computer-vision tasks and reviews adversarial attacks and differentiable renderers. The paper distinguishes its renderer by directly modeling physical image formation with efficient analytical derivatives.

2 RELATED WORK

  • Rendered training data has supported computer-vision networks deployed on real-world tasks, and prior work reports that simulation can match or exceed real-data training in some settings.Veeravasarapu et al. (2017a) report matching full-real-data training with 10% real and 90% simulation data; Tremblay et al. (2018) report better performance than real data alone.Source: §2 · p. 3
  • Prior adversarial approaches include pixel or texture manipulation and learned rendering; the learned-rendering approach takes an average of 12 minutes per adversary, compared with a few seconds for this work.The paper says pixel- or texture-based approaches remain variants of pixel norm-balls, while the learned rendering model can be biased toward its training distribution.Source: §2 · p. 3
  • Existing differentiable renderers trade off bias, accuracy, or computational cost, while this renderer efficiently computes analytical derivatives of a physically based image-formation model.Its derivatives are described as unbiased and accurate by construction, with scalable memory consumption; it ignores non-differentiable visibility changes for infinitesimal perturbations.Source: §2 · p. 4

3 ADVERSARIAL ATTACKS IN PARAMETRIC SPACES

The section extends adversarial optimization from pixel colors to lighting and geometry parameters, using a differentiable renderer to propagate image-space gradients. It describes the renderer’s efficiency and the construction of lighting- and geometry-based adversarial examples.

3 ADVERSARIAL ATTACKS IN PARAMETRIC SPACES

  • Parametric attacks optimize a cost over images I(U, V), where U is lighting and V is geometry, using renderer derivatives and the chain rule.The cost combines cross-entropies to support targeted or untargeted attacks by specifying labels to decrease or increase in confidence.Source: §3 · p. 4

3.2 ADVERSARIAL LIGHTING AND GEOMETRY

  • A single lighting condition can fool a classifier from different viewing angles, and changing lighting can produce target misclassifications such as miniskirt and water tower.Source: Figure 5 · p. 5, Figure 4 · p. 5

3.1 PHYSICALLY BASED DIFFERENTIABLE RENDERER

  • The renderer analytically differentiates a diffuse image-formation model, treating infinitesimal visibility changes as ignorable and varying geometry and realistic lighting.Its assumptions include diffuse materials, local illumination, and distant light sources.Source: §3.1 · pp. 4–5
  • The renderer handles problems with more than 100,000 variables, while OPENDR runs out of memory above 3,500 variables.It is also reported to be orders of magnitude faster than OPENDR.Source: §3.1 · p. 5, Figure 3 · p. 5

3.2 ADVERSARIAL LIGHTING AND GEOMETRY

  • Spherical-harmonics lighting coefficients constrain perturbations by filtering high-frequency illumination, making them more realistic than pixel perturbations.The passage notes that the first two spherical-harmonics orders approximate diffuse-material rendering with 1% pixel-intensity error.Source: §3.2 · p. 5
  • Geometry attacks perturb triangle-mesh vertex positions, which determine face normals and therefore surface shading; different views of one perturbed jaguar can be classified as cat and dog.The described normal derivatives are computed analytically from mesh vertices.Source: Figure 6 · p. 6, §3.2 · p. 6

4 RESULTS AND EVALUATION

The experiments find adversarial examples through lighting and geometry perturbations, including in constrained skylight and multi-view settings. The paper also reports transfer to unseen models and views, with different generalization for lights and shapes.

4 RESULTS AND EVALUATION

  • Adversarial lighting can be found within the three-parameter skylight subspace, although its low degrees of freedom make finding adversaries more difficult.The skylight parameters are differentiated through the Preetham model.Source: §4 · p. 6, Figure 7 · p. 6
  • Adversarial lighting and geometry work in single-view and multi-view cases, and one geometry perturbation makes the same shape classify differently from different angles.The geometry meshes are upsampled to more than 10K vertices before perturbation.Source: §4 · p. 6
  • Geometry perturbations fool classifiers in single-view and multi-view examples, including a mailbox classified differently from different angles.Figure 8 also shows a perturbed shape classified as a rifle or slug.Source: §4 · p. 6, Figure 9 · p. 7, Figure 8 · p. 7

4.1 PROPERTIES OF PARAMETRIC NORM-BALLS AND ADVERSARIES

  • Parametric adversarial examples generated on ResNet also transfer to unseen models, including AlexNet, DenseNet, SqueezeNet, and VGG.The evaluation tests 5,000 ResNet adversaries; Table 3 reports attack success rates.Source: §4.1 · p. 7, Table 3 · p. 7
  • Adversarial lights fool unseen views more generally, whereas adversarial shapes are less generalizable to unseen views.The evaluation computes adversaries from a subset of views and tests them across all sampled views.Source: §4.1 · p. 7, Table 4 · p. 7
  • Adversarial lighting and geometry perturbations have a higher success rate than random perturbations in parametric spaces.Figure 10 compares attack success rates using parametric norm-balls.Source: Figure 10 · p. 7
  • Computing derivatives takes a few seconds because adversaries normally require fewer than 10 iterations, and runtime depends on the number of pixels requiring derivatives.The reported runtime uses a CPU Python implementation and OpenGL rendering.Source: §4.1 · p. 8

5 RENDERED ADVERSARIAL DATA AUGMENTATION AGAINST REAL PHOTOS

The authors train classifiers with rendered adversarial lighting examples and evaluate them on real photographs rather than computer-generated images. On the laboratory photo test set, adversarial-lighting augmentation has the highest reported accuracy among the three training settings.

5 RENDERED ADVERSARIAL DATA AUGMENTATION AGAINST REAL PHOTOS

  • The evaluation uses camera-captured photographs under controlled laboratory lighting, rather than computer-generated test images.The lighting patterns are projected onto the objects, and Figure 11 illustrates a subset of the photographs.Source: §5 · p. 8, Figure 11 · p. 8
  • All three WideResNet training settings have comparable CIFAR-100 test performance at approximately 77%.The settings are CIFAR-100 alone, plus 100 randomly lit images, and plus 100 adversarially lit images.Source: §5 · p. 8
  • Across three training cases, average prediction accuracies over five trained WideResNets on the real-photo test data were 4.6%, 40.4%, and 65.8%, supporting improvement from rendered-image training.The test set comprised 500 real photographs of oranges captured under different lighting patterns projected with an LG PH550 projector.Source: §5 · p. 8

6 LIMITATIONS & FUTURE WORK

The authors describe parametric norm-balls as an initial step toward real-world adversarial machine learning and identify broader evaluation and renderer extensions as future work.

6 LIMITATIONS & FUTURE WORK

  • Evaluations beyond the laboratory data are needed to assess the potential of rendered adversarial data augmentation for real-world use.The authors suggest coupling the renderer with 3D scene reconstruction as a route toward a complete training pipeline.Source: §6 · pp. 8–9
  • The renderer assumes locally constant texture; extending derivatives to materials, visibility changes, or shape skeletons could improve its scope.These extensions could support adversarial materials and poses.Source: §6 · p. 9

A COMPARISON BETWEEN PERTURBATION SPACES

The comparison presents parametric perturbations alongside pixel/color perturbations at matching L∞ pixel-norm levels. The authors describe the parametric perturbations as more realistic across several perturbation scales.

A COMPARISON BETWEEN PERTURBATION SPACES

B PHYSICALLY BASED RENDERING

Physically based rendering models light emitted by sources, modified by a scene, and received by a camera. Rendering approximates the light-propagation equation to synthesize images.

B PHYSICALLY BASED RENDERING

  • PBR models light traveling from sources through a scene to a camera, with the scene modulating the light along the way.Rendering synthesizes images using approximations of the rendering equation, which describes light propagation through a point in space.Source: B PHYSICALLY BASED RENDERING · pp. 13–14, Figure 13 · p. 13
  • The rendering equation describes how incoming light is reflected by a material toward an output direction, alongside emitted radiance.The passage names output radiance, position, output direction, emitted and incoming radiance, incoming angle, and the material’s reflection behavior.Source: B PHYSICALLY BASED RENDERING · p. 14
  • Computing the steady-state rendering equation explicitly is intractable for the paper’s applications, so it serves as a reference for simplifications.The discussion ignores emitted radiance and assumes the space between objects is a vacuum, limiting light interactions to scene objects.Source: B PHYSICALLY BASED RENDERING · p. 14

B.1 LOCAL ILLUMINATION

The renderer approximates steady-state illumination with one direct-light iteration, then uses rasterization to calculate pixel illumination from the visible surface fragment.

B.1 LOCAL ILLUMINATION

  • The renderer approximates steady state with one lighting iteration, modeling each point as receiving light directly from sources.This simplification ignores light occluded by other objects, so shadows disappear; auxiliary techniques are commonly used to evaluate shadows.Source: §B.1 · p. 14
  • Rasterization converts the 3D scene into pixels, determining which object fragment is visible through each pixel.The renderer computes only light traveling directly from sources through that fragment to the pixel.Source: §B.1 · p. 14, Figure 14 · p. 14
  • A fragment’s illumination is determined by its position and surface normal, and its radiance is written as R(p, n, ωo) = uo(p, ωo).Source: §B.1 · p. 14

B.2 LAMBERTIAN MATERIAL

The appendix builds a differentiable renderer around diffuse Lambertian shading, distant environment lighting, and spherical-harmonic approximations. It derives lighting and geometry derivatives under these assumptions.

B.2 LAMBERTIAN MATERIAL

  • Lambertian materials reflect incoming light uniformly across directions, with albedo representing diffuse surface color.The integration is restricted to the upper hemisphere to model light not bouncing through objects.Source: §B.2 · pp. 14–15

B.3 ENVIRONMENT MAPPING

  • Environment mapping represents total incoming radiance from distant lighting as a function of direction, independent of object position.Because distant illumination is effectively unchanged when scene objects move, its contribution can be precomputed or cached.Source: §B.3 · p. 15

B.4 SPHERICAL HARMONICS

  • Spherical harmonics provide a multiresolution basis for evaluating the lighting integral as a weighted product of coefficients.Diffuse objects under distant lighting can be 99% approximated using only the first few spherical-harmonic bases.Source: §B.4 · p. 15

C DIFFERENTIABLE RENDERER

  • The renderer assumes Lambertian reflectance, distant sources, local illumination, and piece-wise constant textures to simplify image formation.These assumptions support a reduced rendering equation that can be evaluated in OpenGL while remaining differentiable with respect to lighting and vertices.Source: C DIFFERENTIABLE RENDERER · p. 15, §B.4 · p. 15

C.1 SPHERICAL HARMONICS; C.2 LIGHTING IN SPHERICAL HARMONICS; C.3 CLAMPED COSINE IN SPHERICAL HARMONICS

  • The spherical-harmonic derivation projects lighting and the normal-dependent clamped-cosine term onto a shared basis for integral evaluation.Lighting coefficients are obtained by integrating the lighting function against spherical-harmonic bases, with quadrature used for evaluation.Source: §C.1 · p. 16, §C.2 · p. 16, §C.3 · pp. 17–18

C.3 CLAMPED COSINE IN SPHERICAL HARMONICS

  • The rendered image evaluates radiance at each visible surface point, with pixel-to-point mapping determined by camera parameters.This extends the pointwise rendering equation to an image given shape, lighting, and camera parameters.Source: §C.3 · p. 18

C.4 LIGHTING AND TEXTURE DERIVATIVES

  • The renderer derives pixel derivatives with respect to lighting coefficients and texture, assuming texture is piece-wise constant over mesh triangles.The lighting derivative is a Jacobian mapping spherical-harmonic coefficients to pixels.Source: §C.4 · p. 18

D DIFFERENTIATING SKYLIGHT PARAMETERS; D.1 DERIVATIVES; E DERIVATIVES OF SURFACE NORMALS

  • The appendix derives derivatives for skylight parameters and surface normals, enabling derivatives with respect to geometry vertices.The skylight model uses sun-direction angles and turbidity; texture variation is treated as piece-wise constant when deriving geometry derivatives.Source: §D.1 · p. 19, E DERIVATIVES OF SURFACE NORMALS · pp. 19–20

F ADVERSARIAL TRAINING IMPLEMENTATION DETAIL

The appendix describes adversarial-lighting training and evaluates whether rendered images are recognizable by ImageNet models. Rendered examples were correctly classified, and adversarial lighting required larger changes as training progressed.

F ADVERSARIAL TRAINING IMPLEMENTATION DETAIL

  • 100 adversarial lighting examples, approximately 0.17% of CIFAR-100 training data, were updated at each epoch.Adversarial examples were generated from orange models under sampled backgrounds, camera views, and initial real-world lighting; optimization used lighting-gradient steps of 0.05 for at most 30 iterations.Source: F ADVERSARIAL TRAINING IMPLEMENTATION… · pp. 20–21
  • Early in training, small lighting changes sufficed to fool the model; later, more dramatic changes were needed or 30 iterations failed to fool it.Random-lighting examples were generated each epoch by perturbing lighting coefficients from -0.5 to 0.5.Source: F ADVERSARIAL TRAINING IMPLEMENTATION… · p. 21, Figure 16 · p. 21

G EVALUATE RENDERING QUALITY

  • 10,000 correctly classified rendered images were used to assess rendering quality, with ImageNet-model confidence supporting recognition of the renderings.The evaluation used 75 textured 3D shapes, varied views and backgrounds, and retained configurations correctly classified by a pretrained ResNet-101.Source: G EVALUATE RENDERING QUALITY · p. 21, Figure 17 · p. 21

Related papers

Loading 1808.02651v2…