Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects

Michael A. Alcorn; Qi Li; Zhitao Gong; Chengfei Wang; Long Mai; Wei-Shinn Ku; Anh Nguyen

Source-linked AI summary

Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects

Michael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, Anh Nguyen

arXiv:1811.11553v3Nov 28, 2018cs.CVcs.LG

TL;DR

DNNs can misclassify natural out-of-distribution poses of familiar objects despite strong image-classification performance. The paper uses 3D-renderer optimization to find such failures and shows that adversarial poses transfer across image classifiers and an object detector.

Problem
DNNs frequently assign wrong labels to out-of-distribution inputs, including natural examples, while real-world systems encounter non-stationary test distributions.
Method
The paper optimizes 3D-renderer parameters to generate unrestricted six-dimensional object poses and evaluates DNN responses using a dataset covering 30 ImageNet classes.
Results
Adversarial poses transfer across models: 99.9% transfer to AlexNet, 99.4% to ResNet-50, and 75.5% to YOLOv3.
Takeaways & Limitations
ImageNet classifiers correctly label only 3.09% of an object's entire six-dimensional pose space, indicating limited object recognition beyond canonical views.
Takeaways & Limitations
The study rendered images against a plain background to focus interpretation on the object itself, although the authors state that real backgrounds do not alter the main qualitative findings.

Abstract

from arXiv · show

Despite excellent performance on stationary test sets, deep neural networks (DNNs) can fail to generalize to out-of-distribution (OoD) inputs, including natural, non-adversarial ones, which are common in real-world settings. In this paper, we present a framework for discovering DNN failures that harnesses 3D renderers and 3D models. That is, we estimate the parameters of a 3D renderer that cause a target DNN to misbehave in response to the rendered image. Using our framework and a self-assembled dataset of 3D objects, we investigate the vulnerability of DNNs to OoD poses of well-known objects in ImageNet. For objects that are readily recognized by DNNs in their canonical poses, DNNs incorrectly classify 97% of their pose space. In addition, DNNs are highly sensitive to slight pose perturbations. Importantly, adversarial poses transfer across models and datasets. We find that 99.9% and 99.4% of the poses misclassified by Inception-v3 also transfer to the AlexNet and ResNet-50 image classifiers trained on the same ImageNet dataset, respectively, and 75.5% transfer to the YOLOv3 object detector trained on MS COCO.

1. Introduction

The paper addresses natural OoD failures by searching 3D pose spaces for familiar objects that DNNs recognize canonically but misclassify in unusual, real-world poses. It finds widespread pose vulnerability, transfer across models and datasets, and limited improvement from training on generated adversarial poses.

Motivation: Natural OoD inputs can cause confident mislabeling in safety-relevant systems, including self-driving cars, autonomous drones, and search-and-rescue robots.The paper distinguishes these natural failures from adversarial examples deliberately crafted to fool a target model.
Framework: The framework searches 3D renderer parameters to find changes in object geometry, appearance, lighting, background, or camera settings that make a target DNN misbehave.The study applies this approach to unrestricted 6D poses of 3D objects and builds a dataset spanning 30 ImageNet classes relevant to traffic environments.
Findings: 3.09% is the share of the entire 6D pose space that ImageNet classifiers correctly label for a 3D object.The classifiers also misclassify many generated adversarial examples that remain recognizable to humans; misclassification can occur after yaw, pitch, and roll changes of 10.31°, 8.02°, and 9.17°, respectively.
Findings: 99.9% and 99.4% of adversarial poses generated against Inception-v3 transfer to AlexNet and ResNet-50, respectively, while 75.5% transfer to YOLOv3.The transfers span image classifiers trained on ImageNet and an object detector trained on MS COCO.
Implications: Training with adversarial poses from the 30 objects, in addition to original ImageNet data, did not improve generalization to held-out objects in the same class.The authors hypothesize that future visual-reasoning models may benefit from incorporating more 3D information.

2. Framework

The framework constructs rendered 3D scenes, varies object pose, and optimizes renderer parameters to induce target-model misclassification. Its evaluation uses pre-trained Inception-v3, two renderer types, a 30-object dataset, sampled views, and comparisons with real photographs.

Problem formulation: The optimization varies only 6D pose parameters while keeping the renderer parameters fixed, seeking an incorrect target output with the highest classifier probability.The objective minimizes cross-entropy for the target class.
3D renderers: The study compares a fast non-differentiable OpenGL-based renderer with a differentiable renderer that enables analytical gradients through approximate rasterization.The differentiable renderer admits gradients but produces lower-quality renders than the non-differentiable renderer.
Scene and optimization: The renderer represents a scene through 3D objects, lighting, a background, and camera parameters, then produces a 2D image for classifier evaluation.Pose changes are estimated either with finite-difference gradients or by backpropagating through a differentiable renderer.
3D-object dataset: The dataset contains 30 textured 3D models corresponding to ImageNet classes relevant to traffic, including 20 vehicles and 10 street-related items.The models were selected for high-quality 2D image textures rather than taken from public 3D datasets.
Evaluation: 83.23% was Inception-v3’s top-1 accuracy across 36 sampled views per object, compared with 77.45% on ImageNet images, with mean top-1 confidence 0.78.The views were sampled across three camera distances and evenly spaced yaw rotations.
Qualitative evaluation: 116 real-photo/render pairs were created by matching rendered object poses to Internet photographs after replacing the photographed objects with background content.The comparisons assessed visual similarity between rendered scenes and real photos.
Background images: All images used a plain ImageNet-mean background to focus interpretation on object recognition rather than background cues.The authors state that adversarial poses can also be found against real background photographs.

3. Methods

The study searches a low-dimensional 6D pose space using random and finite-difference gradient methods, while constraining object placement within the camera frustum. Pose changes are applied through standard 3D transformations, and rotation angles are represented continuously for optimization.

Optimization methods: The pose space is low-dimensional, and the rendering objective is highly non-convex, motivating comparisons between random search and finite-difference gradient descent.The non-differentiable renderer is sufficiently fast for non-gradient methods, enabling this comparison.
Pose transformations: Rotations are applied to each mesh vertex using yaw, pitch, and roll matrices, then translation adds vector T to the rotated object.The transformations use standard computer-graphics matrices.
Pose transformations: The object center is constrained within the camera viewing frustum, with x- and y-coordinates in [−s, s] and z-coordinate in [−28, 0].Here, s is the largest value keeping the center inside the camera frame; its computation depends on camera viewing angle and depth difference.
Optimization methods: Random search samples rotation angles uniformly from (0, 2π) and translations uniformly within the study’s defined ranges.Each iteration selects a new pose rather than following an approximated gradient.
Optimization methods: ZRS samples 10 combinations of xδ, yδ, θy, θp, and θr at each of 30 evenly spaced zδ values from −28 to 0.The procedure uses zδ because preliminary results indicated that it strongly influences predictions through rendered object size.
Optimization methods: ZRS initializes gradient-based methods with the highest-target-probability pose and serves as an attack baseline by narrowing the zδ range to its best two values.The initialization and attack procedures use different selections from the sampled target probabilities.
Gradient descent: Finite central differences approximate first-order derivatives, followed by vanilla gradient descent minimizing cross-entropy loss for a target class.The gradient approximation is used in the iterative descent procedure.
Gradient descent: Rotation angles are encoded as (cos(θi), sin(θi)), producing a 9-parameter optimization space and avoiding discontinuities from angular periodicity.The representation maps the Cartesian plane to angles using atan2; the learning rate is 0.001 for 100 steps.

4. Experiments and results

Experiments show that DNN recognition is highly localized in 6D pose space, remains sensitive to small pose changes, and transfers across models. Adversarial training improves performance on known objects but leaves held-out objects vulnerable.

4.1. Neural networks are easily confused by object rotations and translations: 3.09% median accuracy across 30 objects shows that DNNs correctly classify only a small portion of the 6D pose space.Misclassifications were found across the pose space, while correct classifications were concentrated in particular parameter regions.
4.1. Neural networks are easily confused by object rotations and translations: High-confidence misclassifications are largely uniformly distributed across pose parameters, whereas correct classifications are multimodal and biased toward certain depths and rotation angles.Correct examples were especially biased toward zδ values closer to the camera.
4.2. Common object classifications are shared across different lighting settings: 47.10% median intersection-over-union across lighting settings shows that many of the 50 most frequent incorrect labels were shared across conditions.Lighting affected misclassifications, but shared labels indicated recurring pose-associated errors across bright, medium, and dark settings.
4.2. Common object classifications are shared across different lighting settings: 83% of sampled poses were misclassified for all 15 four-wheeled vehicle objects, indicating that adversarial poses often generalize across similar objects.Classification probability and accuracy varied radically as objects were rotated in the world.
4.3. Correct classifications are highly localized in the rotation and translation landscape: 8.02° of rotation, 2 px horizontal translation, 4.5 px vertical translation, or 5.4% size change could cause misclassification.The study quantified sensitivity using single-parameter disturbances and minimum changes that produced errors.
4.6. Adversarial training: Adversarial training raised known-object accuracy from 6.7% to 99.67% but held-out objects still had an 89.2% error rate.The training used adversarial examples generated from 30 objects and evaluated both training-set and held-out objects.

5. Related work

Related work uses 3D graphics and differentiable rendering to diagnose vision-model failures, but this paper focuses on OoD poses of familiar objects and substantial 3D transformations.

5. Related work: Earlier 2D adversarial attacks modified pixels, patches, or images, whereas this work studies physically interpretable 3D object poses.The paper contrasts pixel-wise perturbations with translations and rotations within 3D space.
5. Related work: The paper claims to be the first to use 3D objects to study OoD poses of familiar training-set objects that fool ImageNet classifiers and MS COCO detectors.Its scope spans both image classification and object detection.

6. Discussion and conclusion

The paper finds that DNNs recognize familiar objects only across a small subset of their possible 3D poses. It introduces 3D-renderer-based testing as a way to enumerate failures and probe model behavior.

DNNs correctly label only a small subset of the entire pose space for familiar 3D objects.
The authors suggest biased datasets, 3D-rendered data augmentation, and incorporating 3D information as possible ways to alleviate adversarial pose problems.
The framework uses 3D renderers and models to test DNNs and extract insights about their black-box functions.

Supplementary materials for: Strike (with) a Pose: Neural Networks Are Easily Fooled

The supplementary materials extend the paper’s title without adding substantive content.

The supplied passage contains only the phrase “by Strange Poses of Familiar Objects.”

S1. Extended description of the 3D object dataset and its evaluation

The supplementary evaluation constructs a 30-object traffic-related 3D dataset, improves differentiable-renderer quality through tessellation, and assesses realism against real photographs and classifier performance.

Dataset construction: The dataset contains 30 textured 3D objects representing 30 traffic-relevant ImageNet classes, including 20 vehicles and 10 street-related items.
Dataset evaluation: The selected models were chosen for high-quality textures and acceptable realism, with selection requiring both a human visual Turing test and high-confidence Inception-v3 recognition.
Renderer preparation: ∼15x: Tessellation increases the average triangle count from NO = 9, 908 to NT = 147, 849.
Renderer preparation: Manual tessellation makes differentiable-renderer outputs sharper and more comparable to non-differentiable-renderer outputs.
Dataset evaluation: The qualitative evaluation produced 116 real-photo/render pairs by matching object poses and comparing each pair side by side.
Dataset evaluation: Most renders passed the human visual Turing test when presented alone, although discrepancies were visible in side-by-side comparisons.
Dataset evaluation: The quantitative evaluation compared Inception-v3 top-1 accuracy on renders with empty backgrounds versus real background images.

a. Evaluation of the renders of 30 objects on an empty background

The empty-background evaluation measures Inception-v3 accuracy and confidence across rendered views of the 30-object dataset.

The evaluation measures Inception-v3 top-1 and top-5 average accuracy and confidence scores on renders of the 30 objects.
The renders use an empty background and systematically sampled object views for the 30-object dataset.

b. Evaluation of the renders of test objects on real backgrounds

The renders produced predictions similar to real photos, supporting their use for testing transferability from ImageNet classification to MS COCO detection. YOLO-v3 performed substantially worse on adversarial poses than on correctly classified poses.

71.43% of top-1 predictions matched between real photos and renders, while 76.06% of top-5 labels matched.
13 objects were retained for transfer testing because they belonged to overlapping ImageNet and MS COCO classes and were well detected by YOLO-v3.
81.03% was YOLO-v3 accuracy on poses correctly classified by Inception-v3, compared with 24.50% on adversarial poses.
75.5% of adversarial poses transferred from Inception-v3 classification failures to YOLO-v3 detection failures across tasks, models, and datasets.

S3. Adversarial poses do exist in the real world

Adversarial poses found in simulation also corresponded to failures involving real-world photographs and unusual views of familiar objects. This evidence supports the existence and transfer of such pose failures beyond rendered images.

98.3% of renders corresponding to out-of-distribution real failure photos were also misclassified.When the real failure photos appeared ImageNet-like, approximately 45% of corresponding renders were correctly classified.
150 real photos were collected as five photos for each of 30 objects to construct real-photo/render pairs for the transfer test.
Real-world videos of cellular phones, jeans, street signs, and umbrellas contained unusual poses that Inception-v3 misclassified despite recognizing canonical poses.
The render evaluation used top-5 predictions across 36 poses per object, with Inception-v3 correctly recognizing 83.2% of renders on average.

S4. Experimental setup for the differentiable renderer

The differentiable-renderer experiments optimized augmented pose transformations while controlling rendering conditions and comparing optimization strategies. Parameter augmentation improved optimization outcomes, whereas multi-view optimization performed worse in the tested setting.

Rendering setup: The renderer used medium lighting and fixed camera, spatial, and depth constraints for DR experiments.
Parameter augmentation: DR-G used an augmented space of 50 rotations and one translation, with the same 100-step budget as FD-G and ZRS.
Parameter augmentation: Increasing rotations per optimization step improved target-label success, maximum adversarial confidence, and convergence speed.
Parameter augmentation: n = 50 rotations per step was selected for the reported DR-G experiments based on these empirical improvements.
Multi-view optimization: Multi-view optimization performed worse than single-view optimization in success rate and convergence steps under a 100-step budget.Results were reported for only bald eagle and tiger cat because of computational cost; more steps might change the comparison.
Additional analyses: Nearest-neighbor comparisons found no qualitatively similar training-set images for sampled adversarial poses, while ball-object error rates varied from 4% to 48%.

S9. Adversarial training

Adversarial-training augmentation improved recognition of the added adversarial-pose classes but did not generalize well to held-out objects. The analysis also examined the visual and pose structure of these failures.

Adversarial training: Training with 30 additional adversarial-pose classes achieved 0.967 top-1 accuracy on AX validation images but 0.565 on the original ImageNet validation set.
Out-of-distribution analysis: Nearest neighbors in AlexNet fc7 space were mostly different from adversarial poses, supporting their out-of-distribution characterization.
Generalization: Adversarial-pose training on 30 objects did not help DNNs generalize well to held-out objects in the same classes.
Generalization: The study could not test whether many more 3D objects would improve held-out generalization because acquiring a large-scale, high-quality 3D dataset is costly and no public dataset existed.
Failure structure: Adversarial poses were homogeneous for some target classes but heterogeneous for most classes.