PaperlayerView exact revision on arXiv

Source-linked AI summary

Transferable Adversarial Attacks for Image and Video Object Detection

Xingxing Wei, Siyuan Liang, Ning Chen, Xiaochun Cao

arXiv:1811.12641v5cs.CV

TL;DR

Object-detection attacks were limited by poor transferability and high computation cost, especially for video. The paper proposes a GAN-based generator combining class and feature losses, and reports efficient image/video attacks with stronger transferability across proposal-based and regression-based detectors.

  • Problem

    Existing object-detection attacks have poor transferability across detector types and high computation costs that make video attacks difficult.

  • Method

    UEA trains a GAN-based generator using high-level class loss and low-level multi-scale attention feature loss to manipulate detector feature maps.

  • Results

    UEA efficiently generates adversarial examples and attacks both proposal-based and regression-based detectors on PASCAL VOC and ImageNet VID.

  • Takeaways & Limitations

    UEA provides a unified attack framework for image and video object detection with better transferability and lower processing time than existing attacking methods.

  • Takeaways & Limitations

    The video threat model assumes dense detection on every frame and argues that successfully attacking it will also fool efficient methods.

Abstract

from arXiv · show

Adversarial examples have been demonstrated to threaten many computer vision tasks including object detection. However, the existing attacking methods for object detection have two limitations: poor transferability, which denotes that the generated adversarial examples have low success rate to attack other kinds of detection methods, and high computation cost, which means that they need more time to generate an adversarial image, and therefore are difficult to deal with the video data. To address these issues, we utilize a generative mechanism to obtain the adversarial image and video. In this way, the processing time is reduced. To enhance the transferability, we destroy the feature maps extracted from the feature network, which usually constitutes the basis of object detectors. The proposed method is based on the Generative Adversarial Network (GAN) framework, where we combine the high-level class loss and low-level feature loss to jointly train the adversarial example generator. A series of experiments conducted on PASCAL VOC and ImageNet VID datasets show that our method can efficiently generate image and video adversarial examples, and more importantly, these adversarial examples have better transferability, and thus, are able to simultaneously attack two kinds of representative object detection models: proposal based models like Faster-RCNN, and regression based models like SSD.

1 Introduction

The paper introduces UEA, a GAN-based attack designed to generate adversarial images and video efficiently while transferring across proposal-based and regression-based detectors. It combines high-level class loss with low-level feature loss to improve transferability.

  • Method: UEA jointly trains a generator with GAN and similarity losses, DAG’s high-level class loss, and a multi-scale attention feature loss.The additional low-level feature loss is intended to complement class loss across detector architectures.
  • Motivation: Existing object-detection attacks have weak transferability across proposal-based and regression-based detectors and require costly iterative optimization.DAG typically needs 150 to 200 iterations for one image, limiting its use for video detection.
  • Method: UEA replaces iterative optimization with a generative mechanism whose forward-only testing step quickly produces adversarial images and video frames.The method is intended to process every frame in a video efficiently.
  • Method: UEA targets shared feature networks used by proposal-based and regression-based detectors to strengthen black-box transferability.The method uses attention-weighted feature manipulation focused on foreground-object regions.
  • Contributions: UEA is presented as simultaneously fooling proposal-based and regression-based detectors while efficiently attacking both image and video data.The contribution is framed as covering Faster-RCNN-like and SSD-like detectors in a unified attack.

2 Related Work

Related work distinguishes proposal-based and regression-based object detectors and reviews adversarial attacks that primarily target proposal-based systems. It motivates a unified attack that can address both detector classes.

  • Image and Video Object Detection: Image detectors are broadly categorized as proposal-based models and regression-based models.Proposal-based systems first detect regions and then classify them, whereas regression-based systems directly predict bounding-box coordinates.
  • Image and Video Object Detection: Video object detection extends image detection by incorporating information across video frames.The supplied related-work passage introduces this distinction but does not provide further details.
  • Adversarial Attack for Object Detection: DAG assigns adversarial labels to proposal regions and uses iterative gradient back-propagation to misclassify them.DAG is described as an optimization-based attack requiring many iterations to generate an adversarial image.
  • Adversarial Attack for Object Detection: Existing detection attacks reviewed here mainly target proposal-based detectors, leaving a gap for methods that attack both detector classes.The paper states its aim is to fill this gap with a unified method.

3 Methodology

UEA generates adversarial images and video key frames with a conditional GAN, combining detector-specific class loss with multi-scale attention feature loss. The generator is trained to preserve visual similarity while disrupting detector features and predictions.

  • 3 Methodology: UEA uses a generative mechanism to produce adversarial images and video key frames, which are then fed to object detectors.The approach replaces per-example optimization with a trained generator for testing images and videos.
  • 3 Methodology: The conditional GAN uses a generator for adversarial examples and a discriminator that distinguishes adversarial images from clean images.An L2 loss additionally measures similarity between clean and adversarial inputs.
  • 3.4 Loss Functions: UEA combines DAG-style class loss with multi-scale attention feature loss to attack both proposal-based and regression-based detectors more effectively.The class loss targets Faster-RCNN proposal predictions, while the feature loss addresses weak transferability across detector types.
  • 3.4 Loss Functions: The class loss misclassifies selected Faster-RCNN proposals using wrong labels sampled from incorrect classes.Training selects proposal regions whose scores are at least 0.7.
  • 3.4 Loss Functions: The attention feature loss perturbs feature subregions associated with foreground objects across multiple network layers.Attention weights are computed from region-proposal coverage and scores, while target feature maps are randomly predefined and fixed during training.
  • 3.4 Loss Functions: The full objective weights GAN, L2, class, and feature losses as L = LcGAN + αLL2 + βLDAG + ϵLF ea, with training using Adam for six epochs.The reported settings are α = 0.05, β = 1, and ϵ = 1×10−4 and 2×10−4 for two selected layers.

4.1 Datasets

The experiments use PASCAL VOC 2007 for image detection and ImageNet VID for video detection, with separate training and testing splits described for each dataset.

  • 4.1 Datasets: PASCAL VOC 2007 provides 5011 training images across 20 classes and 4952 testing images for image detection.
  • 4.1 Datasets: ImageNet VID contains 759 video snippets for training and 138 for testing.
  • 4.1 Datasets: ImageNet VID videos mostly use 25 or 30 fps and include 30 object categories.The categories are a subset of ImageNet categories.

4.2 Metrics

The evaluation measures both attack strength against object detectors and the time required to generate adversarial examples. Attack strength is quantified by the drop in mean Average Precision.

  • 4.2 Metrics: The experiments evaluate attacking performance against object detectors and adversarial-example generation time.
  • 4.2 Metrics: Fooling rate is measured by the mAP drop between clean and adversarial examples.A stronger attack produces lower detector mAP on adversarial examples.
  • 4.2 Metrics: Processing time is reported per image or video frame for each attacking method.This metric reflects the importance of generation speed for video data.

4.3 Threat Models

The threat models include representative proposal-based and regression-based object detectors for image detection, while video detection methods process key frames and propagate results to other frames.

  • 4.3 Threat Models: Image experiments target Faster-RCNN and SSD300 as representative proposal-based and regression-based detectors.Both models are retrained on PASCAL VOC training data and evaluated on its testing set.
  • 4.3 Threat Models: Video detection methods typically detect objects on key frames and propagate the results to other video frames.
  • 4.3 Threat Models: Efficient video detection methods may have lower detection accuracy than dense detection that runs image detection on every frame.

4.4 Results on Image Detection

UEA matches DAG against Faster-RCNN while transferring substantially better to SSD300, generating adversarial examples far faster and with consistently lower perceptibility. Ablation and qualitative results attribute this cross-detector effect to combining class and feature losses to manipulate shared feature representations.

  • Quantitative comparison: UEA and DAG each achieve a 0.65 accuracy drop against Faster-RCNN, while UEA achieves a 0.48 drop against SSD300 versus DAG’s 0.04.The SSD300 drop for UEA is reported as 12 times larger than DAG’s.
  • Quantitative comparison: UEA generates adversarial examples in 0.01 time units versus DAG’s 9.3, making it almost 1000 times faster.The paper states that processing a 100-frame video would take UEA about one second.
  • Perceptibility: UEA’s generated adversarial images remain consistently imperceptible and are nearly indistinguishable from clean images.DAG’s perturbations increase with the iterations required for different images and can become obvious.
  • Ablation study: Class loss attacks Faster-RCNN effectively but has limited ability against SSD300, whereas adding feature loss strengthens SSD300 attacks without reducing Faster-RCNN performance.The full hybrid loss combines high-level class loss with low-level feature loss.
  • Qualitative comparisons: Qualitative results show UEA fooling both detectors: Faster-RCNN misses all boxes on five examples, while SSD300 makes wrong or zero detections across five examples.DAG succeeds against Faster-RCNN but SSD300 continues to detect the adversarial examples effectively.
  • Feature mechanism: Feature visualizations show that UEA manipulates shared feature maps, preventing Faster-RCNN from producing usable proposal regions and fooling SSD300.Both detectors use VGG16 as their feature network, with visualization performed on conv4 features.

4.5 Results on Video Detection

UEA is evaluated on ImageNet VID using dense-frame Faster-RCNN and SSD300 detection, with adversarial videos assessed through mAP drops. The method fools both detectors and produces substantial per-category degradation.

  • Cross-detector attack: UEA simultaneously fools proposal-based Faster-RCNN and regression-based SSD300 on video detection.The comparison targets representative detectors from the two major detector categories.
  • Per-category results: Figure 8 compares detecting performance on clean and adversarial videos for each category, showing mAP drops across ImageNet VID classes.The figure provides a per-category view alongside the aggregate mAP-drop results.
  • Evaluation setup: ImageNet VID evaluation trains Faster-RCNN and SSD300, then compares their detection on clean and UEA-generated adversarial videos.The evaluation runs both detectors on each testing-video frame.
  • Quantitative results: 0.40 mAP drop is achieved for Faster-RCNN, while 0.44 mAP drop is achieved for SSD300.These results are reported for adversarial videos generated on the ImageNet VID testing set.
  • Experimental choice: The experiments use VGG16-based Faster-RCNN and SSD300, focusing on mAP drop rather than original detection mAP.The stated objective is to measure UEA’s attacking ability.

5 Conclusion

The conclusion presents UEA as an efficient and transferable adversarial-example generator for image and video object detection. Experiments on PASCAL VOC and ImageNet VID support its effectiveness across two representative detector types.

  • Efficiency: UEA generates adversarial examples with processing time 1000 times faster than current attacking methods.This efficiency enables handling both image data and video data.
  • Transferability: UEA has better transferability than existing attacking methods and attacks proposal-based and regression-based detectors simultaneously.The conclusion identifies Faster-RCNN and SSD as representative examples of the two detector types.
  • Empirical support: Experiments on PASCAL VOC and ImageNet VID verify UEA’s effectiveness and efficiency.The conclusion treats both image and video detection settings as experimental evidence.
Loading 1811.12641v5…