Universal Adversarial Perturbations Against Semantic Image Segmentation

Jan Hendrik Metzen; Mummadi Chaithanya Kumar; Thomas Brox; Volker Fischer

Source-linked AI summary

Universal Adversarial Perturbations Against Semantic Image Segmentation

Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, Volker Fischer

arXiv:1704.05712v3Apr 19, 2017stat.MLcs.AIcs.CVcs.LGcs.NE

TL;DR

Adversarial perturbations threaten semantic segmentation systems because they can alter dense scene predictions while remaining difficult for humans to notice. This paper develops universal, targeted attacks for segmentation and finds that they can impose nearly arbitrary static outputs or remove a chosen class while largely preserving other labels.

Problem
Prior adversarial-example research focused on image classification, leaving attacks against semantic segmentation and its localization component insufficiently studied.
Method
The paper proposes two efficient image-agnostic universal-perturbation methods trained to produce either a fixed target segmentation or a segmentation with a designated class removed.
Results
Universal perturbations can produce approximately the same desired static segmentation for arbitrary inputs or almost completely hide selected classes while leaving other classes nearly unchanged.
Takeaways & Limitations
The results emphasize the need for stronger robustness and attack detection for semantic segmentation, especially in safety- or security-critical applications.
Takeaways & Limitations
The presented method does not directly allow an adversary to physically realize the perturbations; further physical-realizability prerequisites are left for future work.

Abstract

from arXiv · show

While deep learning is remarkably successful on perceptual tasks, it was also shown to be vulnerable to adversarial perturbations of the input. These perturbations denote noise added to the input that was generated specifically to fool the system while being quasi-imperceptible for humans. More severely, there even exist universal perturbations that are input-agnostic but fool the network on the majority of inputs. While recent work has focused on image classification, this work proposes attacks against semantic image segmentation: we present an approach for generating (universal) adversarial perturbations that make the network yield a desired target segmentation as output. We show empirically that there exist barely perceptible universal noise patterns which result in nearly the same predicted segmentation for arbitrary inputs. Furthermore, we also show the existence of universal noise which removes a target class (e.g., all pedestrians) from the segmentation while leaving the segmentation mostly unchanged otherwise.

1. Introduction

This work extends adversarial-perturbation research from image classification to semantic segmentation, targeting image-agnostic attacks that remain inconspicuous while changing scene predictions. It introduces static-target and target-class-removal attacks and shows that universal perturbations can generalize across unseen images.

Semantic segmentation assigns a class label to every pixel, making adversarial attacks relevant to scene understanding in automated driving, surveillance, and robotics.
Universal perturbations are designed to be input-agnostic because physical-world attacks cannot depend on an unknown future input.
Universal perturbations can make arbitrary inputs yield essentially the same static segmentation, even when the scenes have nothing in common.
The paper proposes two efficient training-set methods: one produces a fixed target segmentation, while the other removes a designated class and preserves the rest.
Universal perturbations generalize to unseen validation images with high probability, and controlling their capacity is important when training sets are small.
Fixed-target perturbations exhibit local structure resembling the target scene.

2. Background

The background introduces adversarial examples, semantic segmentation, and universal perturbations, then motivates adapting targeted attacks to dense prediction. It also describes the FCN-8s architecture and the input-dependent and universal attack paradigms preceding this work.

2.1. Semantic Image Segmentation: Semantic segmentation is a dense prediction task that assigns a class label to each image pixel.
2.1. Semantic Image Segmentation: FCN-8s combines a VGG16-based encoder with a decoder and skip paths that recover high-resolution spatial information for final segmentation.
2.2. Adversarial Examples: An adversarial example adds a perturbation ξ to an input x, changing the model output while typically constraining ξ to remain quasi-imperceptible.
2.2. Adversarial Examples: FGSM computes a fast, non-iterative perturbation using one gradient-sign step with step width ε under an ℓ∞ constraint.
2.2. Adversarial Examples: Earlier attacks can be targeted or untargeted, but input-dependent perturbations differ from universal perturbations that are image-agnostic.
2.2. Adversarial Examples: Universal perturbations are generated over training images and then applied to arbitrary data points to fool networks on a large fraction of images.

3. Adversarial Perturbations Against Semantic Image Segmentation

The paper formulates segmentation attacks by selecting target segmentations, optimizing image perturbations under an ℓ∞ constraint, and extending the attack to universal perturbations trained across inputs. It considers static and dynamic targets, balances hiding objects against preserving background predictions, and regularizes universal perturbations spatially.

Attack objective: The segmentation loss averages classification losses over target spatial locations, allowing the adversary to minimize disagreement with a desired target segmentation.The target-oriented objective searches for x_adv such that J_ss(f_θ(x_adv), y_target) is minimal.
Target generation: The adversary can base target generation on the model prediction y_pred = f_θ(x), because ground-truth segmentation is unavailable to the adversary.This assumes the adversary has access to f_θ.
Target generation: Static targets reuse a fixed segmentation across later time steps, whereas dynamic targets adapt to ego-motion while removing a designated object class.Static targets suit fixed-camera scenarios; dynamic targets preserve other scene predictions while hiding class o.
Objective design: A weighted loss trades off removing target-class predictions against keeping background predictions unchanged.ω = 1 focuses solely on removing the target class, while ω = 0 focuses only on preserving the background.
Universal perturbations: Universal perturbations are optimized over a training set and must generalize to test inputs for which no target segmentation is available at test time.The method therefore extends the image-dependent attack to input-agnostic perturbations.
Universal perturbations: Periodicity regularizes universal perturbations against overfitting by optimizing a h × w proto-perturbation, tiling it spatially, and averaging gradients over tiles.The gradient expression uses R and S for the number of tiles per dimension.
Universal perturbations: Universal-perturbation quality depends crucially on training-set size, although unlabeled data can in principle scale the training set.Larger sets increase computation, model queries, monetary costs, or identification risk.

4. Experimental Results

Experiments on Cityscapes evaluate universal perturbations for static target segmentations and for hiding pedestrians while preserving background predictions. The attacks generalize to unseen data and, for static targets, across a related dataset, though targeted transfer across networks is poor.

Experimental setup: The experiments generate perturbations on Cityscapes training subsets and evaluate them on unseen validation images, using ε = 10 unless otherwise noted.Cityscapes contains 2975 training and 500 validation images; images and labels were downsampled to 1024 × 512 pixels.
Static Target Segmentation: Static-target perturbations are highly structured: their local patterns resemble the target scene, including horizontal and vertical edges corresponding to buildings and fences.The authors suggest this may exploit networks’ robustness to contrast changes, allowing low-contrast structures to influence predictions.
Static Target Segmentation: Static-target perturbations produce nearly the same success rate on validation data as on training data, indicating little overfitting even for high-dimensional perturbations.Larger ε improves success rates, while ε = 10 provides a balance between high success and low perceptibility.
Dynamic Target Segmentation: Dynamic-target attacks hide nearly all pedestrian pixels while preserving most background predictions, but the resulting segmentation can appear inhomogeneous and suspicious on closer inspection.The validation setting uses ω = 0.9999; pedestrian hiding is measured against original pedestrian predictions, while background preservation compares the network’s background segmentation.
Dynamic Target Segmentation: More training images and smaller periodic tiles improve pedestrian removal on validation data, while smaller tiles preserve background less well.With m = 1700, a 512 × 512 tile provides a reported trade-off; validation failures are attributed mainly to training-data overfitting.
Dynamic Target Segmentation: Increasing ω hides more pedestrian pixels but preserves background less well, whereas the unweighted loss fails by focusing too strongly on background preservation.The authors therefore use ω = 0.9999 as a reasonable trade-off because pedestrian pixels are much fewer than background pixels.
Generalizability: On CamVid without fine-tuning, static attacks transform 78% of pixels to the target, while dynamic attacks hide 84.5% of pedestrian pixels and preserve 79.6% of background pixels.Transfer to PSPNet reduces Cityscapes IoU from 75.8% to 8.8%, but target IoU is only 9.5%, indicating untargeted rather than targeted cross-network generalization.

5. Conclusion and Outlook

The method generates universal perturbations that can impose nearly arbitrary segmentations or remove selected classes while largely preserving the rest of the map. The paper highlights robustness and detection as important future work, while noting that precise digital scene control remains necessary for these attacks.

Universal perturbations can produce approximately the same desired static target segmentation for arbitrary, unrelated input images.
Universal perturbations can almost completely remove selected classes, such as pedestrians, while leaving the remaining class map nearly unchanged.
These findings motivate future work on robustness to adversarial perturbations and attack detection, especially in safety- or security-critical applications.
The presented method does not directly enable physical-world attacks because it requires precise control over the scene's digital representation.

A. Supplementary material

The supplementary material provides higher-resolution illustrations for static and dynamic target segmentation, including best-case, worst-case, and randomly selected examples. The figures organize each example from input and original prediction through universal noise, target, adversarial example, and resulting prediction.

Higher-resolution illustrations are provided for Figure 3 and Figure 6.
Static and dynamic target segmentation are each illustrated with best-case, worst-case, and randomly selected examples.
Each figure sequence includes the image, prediction on the image, universal noise, adversarial target, adversarial example, and prediction on the adversarial example.

D. Static Target Segmentation - Random Example

The static-target random example is presented as a six-part visual sequence showing the original image and prediction, universal noise, adversarial target, adversarial example, and its prediction.

The random static-target example displays the original image alongside its prediction.
The sequence then shows the universal noise and adversarial target used for the static segmentation attack.
It concludes with the adversarial example and the network prediction on that adversarial example.

E. Dynamic Target Segmentation - Best Example

The dynamic-target best example is presented as a six-part visual sequence showing the original image and prediction, universal noise, adversarial target, adversarial example, and its prediction.

The best dynamic-target example displays the original image alongside its prediction.
The sequence then shows the universal noise and adversarial target used for the dynamic segmentation attack.
It concludes with the adversarial example and the network prediction on that adversarial example.

F. Dynamic Target Segmentation - Worst Example

The figure presents a worst-example segmentation attack through the original image, its prediction, universal noise, an adversarial target, an adversarial example, and its resulting prediction.

The figure compares an input image with its prediction before the attack.
Universal noise, shown enlarged by 4x, is paired with an adversarial target segmentation.
The adversarial example is evaluated through the prediction produced on that perturbed image.

G. Dynamic Target Segmentation - Random Example

The figure presents a random-example segmentation attack using the original image, its prediction, universal noise, an adversarial target, an adversarial example, and its resulting prediction.

The figure compares a randomly selected image with its prediction before perturbation.
Universal noise, displayed enlarged by 4x, is shown alongside an adversarial target.
The adversarial example is followed by the prediction generated from that example.

H. Network Predictions on Universal Perturbations

The figure contrasts static and dynamic universal perturbations with the corresponding predictions on each perturbation.

The static universal perturbation is displayed enlarged by 4x, with its corresponding prediction shown beside it.
The comparison places static and dynamic perturbations alongside their respective network predictions.
The dynamic universal perturbation is displayed enlarged by 4x, with its corresponding prediction shown beside it.