PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Adversarial Perturbation on Deep Proposal-based Models

Yuezun Li, Daniel Tian, Ming-Ching Chang, Xiao Bian, Siwei Lyu

arXiv:1809.05962v2cs.CV

TL;DR

Object-detector attacks must affect class labels and object locations, while existing approaches may depend on model architecture. R-AP attacks the shared RPN with label and shape losses; evaluations report degradation across state-of-the-art detection and segmentation models with minimal perceptible artifacts.

  • Problem

    Object-detector perturbations must affect both class labels and object locations, while existing methods may require detailed knowledge of network architectures.

    Source: §1 · p. 2

  • Method

    R-AP attacks the shared Region Proposal Network using label loss to disturb proposal labels and shape loss to degrade bounding-box regression.

    Source: §1 · p. 2

  • Results

    R-AP significantly degrades several state-of-the-art object detectors and instance segmentation algorithms on MS COCO 2014 with minimal perceptible artifacts.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that targeting the RPN can affect downstream detection and instance-segmentation performance across proposal-based models.

    Source: §1 · p. 2, §5 · p. 9

Abstract

from arXiv · show

Adversarial noises are useful tools to probe the weakness of deep learning based computer vision algorithms. In this paper, we describe a robust adversarial perturbation (R-AP) method to attack deep proposal-based object detectors and instance segmentation algorithms. Our method focuses on attacking the common component in these algorithms, namely Region Proposal Network (RPN), to universally degrade their performance in a black-box fashion. To do so, we design a loss function that combines a label loss and a novel shape loss, and optimize it with respect to image using a gradient based iterative algorithm. Evaluations are performed on the MS COCO 2014 dataset for the adversarial attacking of 6 state-of-the-art object detectors and 2 instance segmentation algorithms. Experimental results demonstrate the efficacy of the proposed method.

1 Introduction

The introduction frames adversarial attacks on object detectors as harder than classifier attacks because they must affect object labels and locations, while existing approaches may require model-specific architecture knowledge. It presents R-AP as a shared-RPN attack using label and shape losses, and reports degradation across state-of-the-art detection and segmentation models with minimal perceptible artifacts.

1 Introduction

  • Unlike classifier attacks, object-detection perturbations must affect class labels as well as object locations and sizes, and prior detector attacks may require detailed architecture knowledge.Existing approaches cited in the introduction design losses around final predictions, making them model-dependent.Source: §1 · pp. 1–2
  • R-AP targets the shared Region Proposal Network with label loss for proposal classification and shape loss for bounding-box regression.The shape loss is intended to degrade bounding-box refinement even when an object is correctly identified.Source: §1 · pp. 2–3
  • R-AP significantly degrades several state-of-the-art object detectors and instance segmentation algorithms on MS COCO 2014, with minimal perceptible artifacts.The reported evaluation uses the MS COCO 2014 dataset.Source: §1 · p. 2
  • The authors describe this as the first thorough investigation of adversarial perturbations on RPNs, targeting performance across proposal-based detectors and instance segmentation algorithms.The introduction motivates RPN as a shared component of these model families.Source: §1 · pp. 2–3

2 Related Work

Deep proposal-based detectors and instance-segmentation models commonly generate proposals with an RPN before refining them into boxes, labels, or masks. Earlier adversarial work focused mainly on classifiers or task-specific detector outputs, while this work targets the shared RPN to attack models without knowing their architecture.

2 Related Work

  • RPNs generate object proposals that later modules refine into bounding boxes and class labels, or into semantic masks for instance segmentation.The passage describes proposal generation followed by proposal refinement as the common paradigm of deep proposal-based models.Source: §2 · p. 3
  • Earlier adversarial perturbation work primarily targeted image classifiers, including FGSM, the basic iterative method, saliency maps, and image-agnostic perturbations.The passage traces the classifier-focused line from Szegedy et al. through several subsequent methods.Source: §2 · p. 3
  • Prior detector attacks used losses based on final predictions and were task-specific, whereas this method attacks the common RPN to degrade models without knowing their implementation details.Object detectors require perturbations to affect class labels as well as object location and size.Source: §2 · p. 3

3 Method

R-AP attacks proposal-based detectors and instance-segmentation models by optimizing an image perturbation against a pretrained RPN. It combines proposal-label and box-shape losses, uses iterative gradient updates, and can combine perturbations generated from different RPNs for black-box attacks.

3 Method

  • R-AP combines a label loss that disrupts proposal classification with a shape loss that drives bounding-box offsets away from the ground-truth fit.The method optimizes the combined objective with respect to the input image using an iterative gradient-based method, while treating the pretrained RPN parameters as fixed.Source: §3 · p. 4, §3.1 · p. 5

3.2 The Robust Adversarial Perturbation (R-AP) Algorithm

  • Perturbations generated from different RPNs can be combined to improve black-box attack robustness, and R-AP can also be combined with other detector attacks.The method targets standard RPNs shared across proposal-based models, so it does not require their implementation details.Source: §3 · p. 4, §3.2 · p. 6

3.1 Notations and Problem Formulation

  • Table 1 covers six state-of-the-art object detectors at mAP 0.5 and 0.7, with lower values indicating better attack performance.The caption specifies the table’s evaluation scope but supplies no individual result values.Source: Table 1 · p. 6
  • Minimizing the shape loss drives positive proposals’ predicted offsets toward large substitute offsets, making their bounding-box shapes incorrect.The method focuses on positive proposals, which are the proposals used by subsequent algorithms; the offset targets replace the real offsets between anchors and matched ground-truth boxes.Source: §3.1 · pp. 5–6

3.2 The Robust Adversarial Perturbation (R-AP) Algorithm

  • R-AP iteratively computes the image gradient of the combined losses, scales the update, clips pixel values, and stops at the iteration, proposal, or PSNR threshold.The objective includes a lower bound on PSNR, used as an approximation of human-perceived image quality; less perturbation corresponds to higher PSNR.Source: §3.1 · p. 5, §3.2 · p. 6

4 Experimental Results

The experiments evaluate R-AP on object detectors and instance-segmentation algorithms, testing perturbations generated from multiple RPN architectures and their accumulated form.

4.3 Object Detection

  • FR-v16 fell from 59.2% to 5.1% at mAP 0.5 and 47.3% to 3.1% at mAP 0.7; on black-box RFCN, accumulated P reduced performance by 13.1% and 14.1%.The RFCN reductions correspond to mAP 0.5 and 0.7, respectively.Source: Figure 3 · p. 7, §4.3 · p. 8

4.1 Dataset

  • Evaluation used 3,000 random MS COCO 2014 validation images from 80 categories, measuring mAP at IoU thresholds 0.5 and 0.7.Source: §4.1 · p. 7

4.2 R-AP Settings

  • Five RPN architectures supplied perturbations that were accumulated as P; experiments stopped at 210 iterations with PSNR above 30 dB.Source: §4.2 · p. 7

4.4 Instance Segmentation

  • Accumulated P reduced FCIS by 15.1%/13.2% and Mask-RCNN by 16.7%/13.8% at mAP 0.5/0.7, while Gaussian noise degraded performance by <1%.The results concern black-box attacks on both instance-segmentation methods.Source: §4.4 · pp. 8–9, Table 2 · p. 9, Figure 4 · p. 9

5 Conclusion

The conclusion presents R-AP as a universal attack on RPN-based detectors and instance-segmentation models, with experiments showing effective attacks on several state-of-the-art models. Further model testing and work on RPN robustness remain future directions.

  • R-AP targets the shared RPN component to affect proposal-based object detection and instance segmentation, disturbing both label prediction and shape regression.Source: §5 · p. 9
  • Future work includes testing additional proposal-based models, including part-based human-pose detection, and investigating RPN robustness.Source: §5 · p. 9
  • Figure 5 contrasts original and attacked results for object detectors and instance-segmentation algorithms.Source: Figure 5 · p. 10

Related papers

Loading 1809.05962v2…