Source-linked AI summary
Adversarial Attacks Beyond the Image Space
Xiaohui Zeng, Chenxi Liu, Yu-Siang Wang, Weichao Qiu, Lingxi Xie, Yu-Wing Tai, Chi Keung Tang, Alan L. Yuille
TL;DR
The paper asks whether neural networks remain vulnerable when adversaries perturb meaningful 3D physical properties rather than image pixels. It inserts a renderer before networks for object classification and visual question answering, then optimizes physical parameters; such attacks work but are harder than image-space attacks and require heavier perturbations.
Problem
The paper investigates whether neural networks can be fooled by perturbing intrinsic 3D physical parameters instead of independently modifying 2D image pixels.
Method
The authors place a differentiable or non-differentiable renderer before target networks and optimize physical parameters through the rendered image-to-prediction pipeline.
Results
Physical-space attacks are effective across object classification and visual question answering, but have lower success rates and require heavier perturbations than image-space attacks.
Takeaways & Limitations
Physical-space adversaries provide perturbations whose dimensions have clear meanings in the physical world, while image-space adversaries are generally not explained by such non-local changes.
Takeaways & Limitations
The findings still require testing in real-world scenarios, and the pipeline remains dependent on rendering quality.
Abstract
from arXiv · showhide
Generating adversarial examples is an intriguing problem and an important way of understanding the working mechanism of deep neural networks. Most existing approaches generated perturbations in the image space, i.e., each pixel can be modified independently. However, in this paper we pay special attention to the subset of adversarial examples that correspond to meaningful changes in 3D physical properties (like rotation and translation, illumination condition, etc.). These adversaries arguably pose a more serious concern, as they demonstrate the possibility of causing neural network failure by easy perturbations of real-world 3D objects and scenes. In the contexts of object classification and visual question answering, we augment state-of-the-art deep neural networks that receive 2D input images with a rendering module (either differentiable or not) in front, so that a 3D scene (in the physical space) is rendered into a 2D image (in the image space), and then mapped to a prediction (in the output space). The adversarial perturbations can now go beyond the image space, and have clear meanings in the 3D physical world. Though image-space adversaries can be interpreted as per-pixel albedo change, we verify that they cannot be well explained along these physically meaningful dimensions, which often have a non-local effect. But it is still possible to successfully attack beyond the image space on the physical space, though this is more difficult than image-space attacks, reflected in lower success rates and heavier perturbations required.
1. Introduction
The paper extends adversarial attacks from independently modified image pixels to physically meaningful 3D scene parameters. It asks whether perturbing properties such as surface normals, illumination, and material can still cause neural-network prediction failures.
- Most prior adversarial attacks modify individual pixels in 2D images to induce incorrect predictions.
- The paper instead targets non-local perturbations of intrinsic 3D scene or object parameters.
- A rendering module connects physical parameters to a rendered image and then to predictions for object classification and visual question answering.
- The authors report that image-space adversaries are almost impossible to approximate with physically meaningful 3D perturbations such as object rotation.
- The central question is whether directly perturbing physical properties can cause neural networks to fail, using optimization beyond the image space.
2. Related Work
Related work established adversarial attacks across several vision tasks and explored physical-world variants. The paper distinguishes its approach by attacking intrinsic 3D parameters through rendering rather than modifying image pixels or only changing visible appearance.
- Adversarial examples have been studied in classification, detection, segmentation, visual question answering, universal perturbations, and printed-image settings.
- White-box attacks define a target objective and backpropagate prediction errors to update image inputs.
- Prior physical-world attacks used printed images, image rotation or brightness changes, and 3D objects, but often required highly perceptible appearance changes.
3. Approach
The approach composes a renderer with a target network, mapping physical scene parameters through rendered images to predictions. Attacks optimize those parameters with differentiable backpropagation or zeroth-order updates while evaluating perceptibility in the rendered image space.
- 3.1. From Physical Parameters to Prediction: The framework maps a 3D scene X through renderer r into image Y and network f into output Z, written as Z = f[r(X); θ].
- 3.1. From Physical Parameters to Prediction: Differentiable rendering represents physical inputs with surface normals N, illumination L, and material m, while camera geometry remains fixed.
- 3.1. From Physical Parameters to Prediction: The material representation uses bidirectional reflectance functions, while N, L, and m encode normal angles, environment lighting, and material parameters.
- 3.1. From Physical Parameters to Prediction: The evaluated tasks are 3D object classification and 3D visual question answering based on rendered 2D images.
- 3.2. Attacks Beyond the Image Space: Physical-space attacks minimize a prediction loss by iteratively updating scene parameters from the initial unperturbed state.
- 3.2. Attacks Beyond the Image Space: Differentiable renderers use backpropagated gradient information, whereas non-differentiable renderers use zeroth-order stochastic coordinate descent with Adam.
- 3.3. Perceptibility: Perceptibility is assessed primarily from the rendered-image perturbation, enabling comparisons across image-space attacks and different physical parameters.
- 3.3. Perceptibility: For differentiable rendering, pixel changes above threshold U = 18 are truncated; for non-differentiable rendering, an image-perturbation penalty balances attack success and perceptibility.
4. Experiments
Experiments on ShapeNet classification and CLEVR visual question answering show that physical-space attacks can fool neural networks, but are substantially harder than image-space attacks. Their interpretable physical operations also expose task- and example-specific vulnerabilities.
- Experimental settings: The study evaluates adversarial attacks with differentiable and non-differentiable renderers on ShapeNet object classification and CLEVR visual question answering.ShapeNet uses AlexNet and ResNet-34; CLEVR evaluates attacks on visual question answering.
- 3D Object Classification: Jointly optimizing all physical parameters produces the highest physical-space success rate, while surface-normal attacks outperform illumination and material attacks.Surface normals have local image effects, whereas illumination and material are global scene or object properties.
- 3D Object Classification: 99 of 102 AlexNet cases and all ResNet-34 cases were successfully attacked in image space, versus only 14 and 6 physical-space successes, respectively.The image-space attacks used 500 steps for AlexNet and 200 for ResNet-34; the physical-space attacks used the same limits.
- 3D Object Classification: Physical-space perturbations use interpretable operations such as rotation, translation, and lighting changes, unlike independent pixel-level image-space perturbations.The physical operations can in theory be instantiated in the physical world using fine-level robotic control.
- 3D Object Classification: Physical-dimension attacks support diagnosis because different examples respond to different parameter subspaces, including a 26% accuracy drop when rotation is added to a color attack.The effects of parameter groups can be approximately additive in some cases but not universally.
- Visual Question Answering: Physical attacks reached 22 CLEVR successes within 500 steps, compared with 66 image-space successes within 500 steps and 93 within 1,200 steps.The authors attribute the similarity to classification to fixing the question before attacking.
- Visual Question Answering: CLEVR attacks often target the key object's color, while language-prior failures can produce implausible answers such as “no” to a shape question.Material attacks do not significantly differ for material-related questions because perceptibility constraints limit material changes.
5. Conclusions
The paper extends adversarial-example analysis from independently modified image pixels to intrinsic 3D physical parameters, using rendering to connect physical scenes with neural-network predictions. Experiments show that physical-space attacks are effective but harder than image-space attacks, while the framework also suggests interpretable robustness analysis and possible defenses.
- The paper studies whether neural networks remain vulnerable when intrinsic 3D scene parameters are perturbed instead of 2D image pixels.
- A rendering module connects the underlying 3D scene to the perceived 2D image, enabling gradient-based attacks on the complete vision pipeline.
- Experiments in object classification and visual question answering show that physical-space adversaries can be constructed, but success rates are lower and successful attacks require heavier perturbations.
- The framework may help diagnose robustness along interpretable dimensions such as color, lighting, and object movement, and may weaken 2D adversaries through physical-space interpretation and re-rendering.
A. Attack Curves with Different (Differentiable or Non-Differentiable) Renderers
Attack curves show that attacks through physical parameters are slower and more difficult than image-space attacks for both renderer settings. Their difficulty is attributed to non-local image effects, although stronger optimization may find more physical-space adversaries.
- Image-space attacks often succeed quickly, whereas differentiable-renderer physical-space attacks are slower and especially difficult for illumination and material.Figure 7 tracks average original-class probability across attack iterations.
- Non-differentiable-renderer physical-space attacks are harder to succeed and require substantially larger perceptibility than image-space attacks.Figure 8 compares average log-probability advantage and image-space Euclidean distance over iterations.
- Physical-space attacks are more difficult because they typically alter multiple image pixels, increasing optimization difficulty.
- A more powerful optimizer may find more adversarial examples in the physical world.