PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Examples for Semantic Segmentation and Object Detection

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, Alan Yuille

arXiv:1703.08603v3cs.CV

TL;DR

Adversarial examples were well established for classification, but their systematic study in semantic segmentation and object detection was limited despite these tasks involving many targets. The paper proposes DAG, which jointly optimizes target losses to generate perturbations for both tasks. These perturbations transfer across training sets, architectures, and tasks, while combining heterogeneous perturbations often improves black-box transfer.

  • Problem

    Adversarial examples had not been systematically studied for semantic segmentation and object detection, which require handling orders of magnitude more targets than classification.

  • Method

    DAG assigns adversarial labels to dense sets of pixels or proposals and iteratively optimizes their overall loss using gradients on active targets.

  • Results

    DAG generates visually imperceptible perturbations that confuse segmentation and detection networks and transfer across training data, architectures, and recognition tasks.

  • Takeaways & Limitations

    Combining heterogeneous perturbations often improves transferability, providing an effective black-box attack against some networks with unknown structures or properties.

  • Takeaways & Limitations

    The method’s final adversarial image applies pixel-value truncation, which may harm the perturbation, although little experimental effect was observed because its magnitude is small.

Abstract

from arXiv · show

It has been well demonstrated that adversarial examples, i.e., natural images with visually imperceptible perturbations added, generally exist for deep networks to fail on image classification. In this paper, we extend adversarial examples to semantic segmentation and object detection which are much more difficult. Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e.g., the basic target is a pixel or a receptive field in segmentation, and an object proposal in detection), which inspires us to optimize a loss function over a set of pixels/proposals for generating adversarial perturbations. Based on this idea, we propose a novel algorithm named Dense Adversary Generation (DAG), which generates a large family of adversarial examples, and applies to a wide range of state-of-the-art deep networks for segmentation and detection. We also find that the adversarial perturbations can be transferred across networks with different training data, based on different architectures, and even for different recognition tasks. In particular, the transferability across networks with the same architecture is more significant than in other cases. Besides, summing up heterogeneous perturbations often leads to better transfer performance, which provides an effective method of black-box adversarial attack.

1. Introduction

The paper extends adversarial examples from classification to semantic segmentation and object detection, where many pixels or proposals must be misclassified simultaneously. It introduces DAG for dense target optimization and studies perturbation transfer across networks and tasks.

  • Deep networks can fail under visually imperceptible input perturbations, motivating adversarial-example research for robustness and network understanding.
  • DAG generates adversarial examples for segmentation and detection by optimizing one loss over all targets with assigned adversarial labels.The targets are pixels or receptive fields for segmentation and proposals for detection.
  • Detection is harder than segmentation because it involves orders of magnitude more targets and can regenerate different proposals after perturbation.The method increases the preserved proposal set through an adjusted IOU threshold to improve attack robustness.
  • DAG perturbations transfer across same-architecture networks with different training data, different architectures, and different recognition tasks.Transfer difficulty increases as the networks or tasks differ more substantially.
  • Combining heterogeneous perturbations significantly improves transferability and supports black-box attacks against networks with unknown structures or properties.

2. Related Work

Prior work established adversarial examples for classification and explored defenses, while segmentation studies had only recently examined dataset-specific and universal perturbations. Detection and segmentation pipelines classify many localized targets, motivating the paper’s extension.

  • Object detection commonly generates, classifies, and post-processes proposals, whereas segmentation classifies dense image locations.
  • Classification research showed that visually imperceptible perturbations can induce confident incorrect predictions and produced gradient-based, optimization-based, and evolutionary attacks.
  • Other studies attempted to reduce adversarial effects through feature-based mechanisms, defensive distillation, stronger attacks, and adversarial training.
  • Two concurrent works studied adversarial examples in semantic segmentation on Cityscapes, including ordinary and universal perturbations.

3. Generating Adversarial Examples

DAG constructs an adversarial perturbation by assigning incorrect labels to recognition targets and iteratively optimizing their collective loss. It updates only correctly classified active targets, normalizes accumulated gradients, and stops when targets are fooled or an iteration limit is reached.

  • DAG takes an image and recognition targets, then seeks a perturbation that confuses as many targets as possible.
  • The target set contains pixels or receptive fields for segmentation and proposals for detection, each paired with an original class label.
  • For every target, DAG assigns an incorrect adversarial label and optimizes a loss that suppresses the original class while increasing the adversarial class.
  • At each iteration, DAG identifies correctly predicted active targets, computes input gradients for them, and accumulates the resulting perturbation.
  • The algorithm normalizes each accumulated gradient and terminates when all targets reach desired predictions or the maximum iteration count.The stated maximum is 200 iterations for segmentation and 150 for detection.
  • Pixel-value truncation can harm the perturbation, but the authors observed little experimental effect because the perturbation magnitude is very small.

3.2. Selecting Input Proposals for Detection

Target selection is tractable for segmentation through dense pixel sampling but more difficult for detection because proposal sets are much larger and may change after perturbation. DAG therefore selects high-quality positive proposals for optimization.

  • Segmentation can treat every pixel as a separate target, with computational complexity proportional to the number of pixels.
  • Detection has orders of magnitude more possible bounding-box targets, and perturbing the image can produce a different proposal set.
  • The proposal-selection strategy preserves positive RPN proposals whose IOU with a ground-truth object and corresponding-class confidence each exceed 0.1.
  • When multiple ground-truth objects qualify, the proposal is assigned to the one with maximal IOU and its confident class becomes the proposal label.

3.3. Quantitative Evaluation

DAG is evaluated on FCN-based segmentation and Faster-RCNN-based detection models, using mIOU and mAP drops to measure attack effectiveness. The results show substantial degradation and indicate that perturbation effectiveness depends on spatial structure and training data.

  • DAG is evaluated with mIOU for semantic segmentation and mAP for object detection on original and perturbed test images.
  • The study covers FCN-Alex and FCN-VGG segmentation models, including variants trained on different datasets.The original FCN training set contains 9610 images, while the DeepLab training set contains 10582 images.
  • The detection evaluation uses ZFNet- and VGGNet-based Faster-RCNN models trained on PascalVOC-2007 or combined PascalVOC-2007/2012 data.The PascalVOC-2007 test set contains 4952 images.
  • 58.70% to 3.61% and 69.14% to 5.92% are the FR-ZF-07 and FR-VGG-07 mAP changes after perturbation.The corresponding models trained with more data change from 61.07% to 1.95% and 72.04% to 3.36%.
  • Randomly permuting perturbation rows or columns causes negligible accuracy drops, implicating spatial structure rather than magnitude.The permutation results average three random permutations.

3.4. Adversarial Examples

DAG produces adversarial and fooling examples for segmentation while its attack behavior depends on proposal density and target count. Detection is harder to attack, and denser proposal sets require more iterations but produce stronger perturbations.

  • 3.4. Adversarial Examples: DAG generates adversarial images that can cause both detection and segmentation failures, including controllable outputs and fooling images.The examples include a human-recognizable adversarial image and a human-unrecognizable fooling image producing false positives.
  • 3.5.1 The Denseness of Proposals: Increasing the IOU rate makes proposals denser, lowers mAP, and produces stronger adversarial perturbations.The method chooses an IOU value of 0.90 because it produces good results.
  • 3.5.2 Convergence: After up to 200 segmentation or 150 detection iterations, fewer than 1% of images fail to converge.Even those cases receive reasonable adversarial perturbations.
  • 3.5.2 Convergence: Detection networks are generally harder to attack than segmentation networks because they have one or two orders of magnitude more potential targets.Increasing the IOU rate also slows convergence because more proposals are optimized.
  • 3.5.2 Convergence: Average convergence requires 31.78 and 54.02 iterations for FCN-Alex and FCN-VGG, versus 47.05 and 41.42 for FR-ZF-07 and FR-VGG-07.

4. Transferring Adversarial Perturbations

DAG perturbations transfer across training sets, architectures, and recognition tasks, with transfer generally strongest between networks sharing an architecture. Combining heterogeneous perturbations improves transfer and supports black-box attacks.

  • 4.1. Cross-Training Transfer: Same-architecture transfer causes substantial performance drops across models trained on different datasets.For FR-ZF-07 attacking FR-ZF-0712, mAP falls from 61.07% to 22.15%; the reverse direction falls from 58.70% to 13.14%.
  • 4.2. Cross-Network Transfer: Transferability weakens across different network structures, although some architecture pairs show substantial drops.FR-ZF-07 reduces FR-VGG-07 accuracy from 69.14% to 66.01%, whereas FCN-VGG reduces DL-VGG accuracy from 70.72% to 45.16%.
  • 4.3. Cross-Task Transfer: Cross-task transfer remains significant when detection and segmentation networks share a structure.Using FCN-VGG and FR-VGG-07 to attack each other lowers mIOU from 54.87% to 43.06% and mAP from 68.88% to 56.33%.
  • 4.4. Combining Heterogeneous Perturbations: Combining heterogeneous perturbations often produces larger accuracy drops than a single perturbation, while permutation destroys their spatial structure and reduces the effect.The combined perturbation’s drop approximately equals the sum of individual drops; r1 + r3 + r5 + r7 remains low-perceptibility at 4.0 × 10−3.
  • 4.5. Black-Box Attack: Summing perturbations from known networks enables black-box attacks without knowing the defender’s architecture or other detailed properties.The combination r1 + r3 + r5 + r7 reduces R-FCN-RN101 accuracy from 80.20% to 64.52%.

5. Conclusions

The paper extends adversarial-example generation from image classification to semantic segmentation and object detection with DAG. Its perturbations transfer across datasets, architectures, and tasks, while heterogeneous combinations often improve black-box attacks.

  • 5. Conclusions: DAG extends adversarial-example generation to semantic segmentation and object detection by optimizing incorrect labels over dense target sets simultaneously.The method generates visually imperceptible perturbations that confuse high-confidence recognition results in a controllable manner.
  • 5. Conclusions: DAG perturbations transfer across different training sets, network architectures, and recognition tasks.The paper relates this transferability to shared principles such as local linearity in deep networks.
  • 5. Conclusions: Combining heterogeneous perturbations often yields more effective adversarial perturbations for black-box attacks.The approach can attack networks whose detailed structure or properties are unknown.

A.1. Generating Geometric Patterns

Different adversarial perturbations can make a segmentation network output preselected geometric patterns, including circles, diamonds, squares, and stripes.

  • A.1. Generating Geometric Patterns: Adversarial perturbations make FCN-Alex output geometric segmentation patterns including circles, diamonds, squares, and stripes.The perturbations are magnified by 10 for visualization.

A.2. Same Noise, Different Outputs

The same perturbation can be constructed to make two segmentation networks produce different pre-specified masks from the same original image.

  • A.2. Same Noise, Different Outputs: A single perturbation can make FCN-Alex and FCN-VGG output different pre-specified segmentation masks.This is harder than using separate perturbations for the two networks; the blue mask regions are predicted as bus.
Loading 1703.08603v3…