PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Robustness of Semantic Segmentation Models to Adversarial Attacks

Anurag Arnab, Ondrej Miksik, Philip H. S. Torr

arXiv:1711.09856v3cs.CV

TL;DR

The paper addresses limited evidence about adversarial robustness in modern semantic segmentation, especially across large-scale datasets and specialized architectures. It rigorously evaluates attacks, architectures, multiscale processing, CRFs, and input transformations, finding substantial robustness differences and recommending defense-aware benchmarking for deployment.

  • Problem

    Adversarial robustness in semantic segmentation has not been extensively analyzed across multiple large-scale real-world datasets, despite the task’s specialized architectures and safety-critical uses.

  • Method

    The paper evaluates diverse segmentation models, adversarial attacks, architectures, multiscale processing, CRFs, and input transformations on two large-scale datasets using IoU-based robustness measurement.

  • Results

    Robustness varies substantially across models: residual connections and multiscale processing are associated with greater robustness, while CRFs provide no robustness to untargeted attacks and defense gains can be marginal under defense-aware attacks.

  • Takeaways & Limitations

    Deeplab v2 should be preferred among evaluated models for safety-critical applications, and robustness should be tested across varied attacks before deployment.

  • Takeaways & Limitations

    Input-transformation results assume attackers initially lack knowledge of the defense, contrary to Kerckhoffs’ principle.

Abstract

from arXiv · show

Deep Neural Networks (DNNs) have demonstrated exceptional performance on most recognition tasks such as image classification and segmentation. However, they have also been shown to be vulnerable to adversarial examples. This phenomenon has recently attracted a lot of attention but it has not been extensively studied on multiple, large-scale datasets and structured prediction tasks such as semantic segmentation which often require more specialised networks with additional components such as CRFs, dilated convolutions, skip-connections and multiscale processing. In this paper, we present what to our knowledge is the first rigorous evaluation of adversarial attacks on modern semantic segmentation models, using two large-scale datasets. We analyse the effect of different network architectures, model capacity and multiscale processing, and show that many observations made on the task of classification do not always transfer to this more complex task. Furthermore, we show how mean-field inference in deep structured models, multiscale processing (and more generally, input transformations) naturally implement recently proposed adversarial defenses. Our observations will aid future efforts in understanding and defending against adversarial examples. Moreover, in the shorter term, we show how to effectively benchmark robustness and show which segmentation models should currently be preferred in safety-critical applications due to their inherent robustness.

1. Introduction

Adversarial examples challenge the safety of high-performing DNNs, while their robustness in semantic segmentation remains insufficiently understood. The paper evaluates modern segmentation models and shows that architecture, multiscale processing, CRFs, and input transformations influence robustness.

  • DNNs can misclassify nearly unchanged inputs, raising concerns for safety-critical systems and motivating robustness alongside clean-input accuracy.
  • Existing defenses are not universal, may reduce clean-input performance, and have been studied mainly on classification tasks or small datasets.
  • Semantic segmentation remains understudied despite its greater complexity and use of components such as dilated convolutions, skip-connections, CRFs, and multiscale processing.
  • The study rigorously evaluates diverse state-of-the-art segmentation networks, examining architectural properties and input transformations across adversarial attacks.
  • Multiscale processing and input transformations reduce adversarial effectiveness under some conditions, while structured prediction models can exhibit defense-like effects.
  • Experiments on two large-scale real-world datasets show that classification observations do not always transfer to segmentation and support careful robustness benchmarking.

2. Adversarial Examples

Adversarial examples are small input perturbations designed to alter network predictions. The paper introduces optimization-based and gradient-based attacks, including single-step and iterative variants adapted to semantic segmentation.

  • An adversarial perturbation changes a network’s original prediction when added to the input, and can be formulated as minimizing perturbation size subject to a target output.
  • The optimization-based formulation is computationally expensive, requiring several minutes to produce one attack, so practical evaluations use faster surrogate methods.
  • FGSM is a single-step untargeted attack that increases the network loss while approximately constraining the perturbation under an ℓ∞ bound.
  • Targeted variants encourage classification as a chosen target, with the least likely predicted class used as the target in the described convention.
  • Iterative FGSM repeatedly applies signed gradient updates, increasing the chance of fooling the original network while maintaining an ℓ∞ constraint through clipping.The clipping operation corresponds to projected gradient descent with step-size α into an ℓ∞ ball of radius ǫ around x.
  • These gradient-based attacks originated in classification but have been adapted to semantic segmentation and other structured or multimodal prediction tasks.

3. Adversarial Defenses and Evaluations

Prior adversarial-defense evaluations have often been limited in attack knowledge, datasets, or model coverage. This paper broadens evaluation to segmentation architectures and large-scale real-world datasets, emphasizing defense-aware testing.

  • No effective defense against all adversarial attacks currently exists, motivating analysis of state-of-the-art segmentation models across varied architectures.
  • The evaluation examines whether segmentation features such as CRFs and multiscale processing naturally implement recently proposed adversarial defenses.
  • Experiments use two large-scale datasets rather than only ImageNet and show that classification findings do not universally transfer to semantic segmentation.
  • The results aim to support defenses that preserve predictive accuracy and identify segmentation models suitable for settings prioritizing both accuracy and robustness.

4. Experimental Set-up

The evaluation uses standard semantic-segmentation benchmarks, diverse model families, several gradient-based attacks, and IoU-based robustness measurements. Models are tested in the configurations in which they were trained.

  • The paper describes the evaluation protocol and releases code to support reproducibility.
  • The study evaluates Pascal VOC and Cityscapes validation sets, covering internet images and road scenes with 21 and 19 classes, respectively.
  • The model set ranges from lightweight embedded systems to structured prediction networks, using VGG, ResNet, and custom architectures with components such as CRFs and multiscale processing.
  • Models are evaluated as trained, without adding CRF post-processing or multiscale ensembling unless those operations were incorporated as network layers during training.
  • The experiments use FGSM, FGSM ll, Iterative FGSM, and Iterative FGSM ll, with iterative attack iterations set to min(ǫ + 4, ⌈1.25ǫ⌉) and α = 1.
  • IoU Ratio measures adversarial IoU relative to clean-image IoU over the full dataset, while absolute IoU is reported in the supplementary material.

5. The robustness of different architectures

Robustness varies substantially across semantic segmentation architectures and datasets: residual connections and multiscale designs tend to help, while larger or more accurate models are not necessarily more robust. Attack effectiveness also depends strongly on the dataset and attack type.

  • Architecture and robustness: ResNet-based models are more robust than VGG-based models, despite comparable clean-input performance and much larger VGG parameter counts.This pattern appears for FCN8s and Deeplab v2 ASPP and is especially evident under single-step FGSM.
  • Architecture and robustness: Adding Dilated-Net’s Context module slightly reduces robustness across tested perturbation strengths, even though it improves clean-input accuracy.The same trend is reported on both Cityscapes and Pascal VOC.
  • Dataset and attack effects: Cityscapes makes single-step attacks substantially more effective than Pascal VOC, with FGSM at ϵ = 32 reaching IoU ratios of 2.5% and 2.8% versus 27.9% and 12.2%.On Cityscapes, single-step methods also appear to outperform iterative methods at high ϵ values.
  • Dataset and attack effects: Even perturbations that usually leave integer RGB values unchanged degrade every analysed model by at least 9% relative IoU.This supports why JPEG preprocessing can mitigate small, high-frequency perturbations.
  • Architecture and robustness: Deeplab v2 with Multiscale ASPP was the most robust model, outperforming PSPNet in absolute IoU on adversarial inputs.PSPNet was the best-performing model on clean inputs but was less robust than multiscale Deeplab v2.
  • Discussion: The effect of residual connections on adversarial robustness remains an open research question.The paper also notes that dataset properties may influence susceptibility to single-step attacks.

6. Multiscale Processing and Transferability of Adversarial Examples

Multiscale processing improves robustness by reducing the effectiveness of attacks generated at a single input scale, while also affecting perturbation transfer across networks and scales. Multiscale Deeplab v2 is particularly robust and generates comparatively transferable attacks.

  • Multiscale processing: Deeplab v2 processes 50%, 75%, and 100% resolutions with shared weights, upsamples outputs, and max-pools per-pixel predictions.Multiscale ensembling can also be performed as a test-time post-processing step.
  • Transferability: FGSM perturbations transfer well to other networks, whereas Iterative FGSM ll perturbations generally transfer poorly.Table 1 distinguishes white-box diagonal attacks from black-box off-diagonal transfers.
  • Transferability: Attacks generated at one scale transfer poorly to other scales, supporting the hypothesis that CNNs are not scale invariant.The 50% resolution attack transfers poorly to other Deeplab v2 scales and architectures.
  • Transferability: Multiscale Deeplab v2 is the most robust to white-box attacks and perturbations generated from single-scale networks.Its multiscale perturbations also transfer best to other networks, although transfer remains weaker than white-box efficacy for FCN models.
  • Multiscale processing: Multiscale inputs improve robustness even for FCN8, which was trained at a single scale, indicating that the effect is not solely due to multiscale training.The findings instead support a role for CNN scale sensitivity.
  • Relation to defenses: Random resizing defenses exploit the same lack of scale invariance as multiscale processing and may be defeated by attacks generated across multiple scales.The paper recommends evaluating defenses with white-box attacks that incorporate knowledge of the defense mechanism.

7. Image transformations and adversarial examples

Randomised input transformations substantially improve robustness to attacks that ignore preprocessing, while largely preserving clean-image performance. However, attackers who model the transformation distribution can largely recover attack effectiveness.

  • Transformations: Four spatial-coordinate-preserving transformations—JPEG recompression, Gaussian blur, HSV jitter, and grayscale conversion—are evaluated as preprocessing for Deeplab v2 MS.Randomisation is applied before network processing without requiring segmentation-coordinate post-processing.
  • Robustness to standard attacks: Each transformation substantially increases Deeplab v2 robustness to Iterative FGSM ll on VOC, with JPEG recompression and Gaussian blur providing the largest benefits.Randomised trials showed little variance, and clean-image IoU differed by at most about 2%.
  • Robustness to standard attacks: Predictions on transformed adversarial inputs typically differ from the original adversarial example because neural networks are not invariant to many input transformations.This pattern is reported across a broad range of geometric and photometric transformations.
  • Implications: The study argues that defense evaluation should incorporate knowledge of the defense mechanism into white-box attacks, consistent with Kerckhoffs’ principle.The authors note that many proposed defenses were not previously tested under this assumption and were later defeated.
  • Subverting transformations: The apparent defense weakens when attacks incorporate the transformation mechanism: EOT averages gradients over sampled transformations before updating the adversarial image.The method samples transformations, computes corresponding losses and gradients, averages those gradients, and then performs the update.
  • Subverting transformations: Under EOT, JPEG and Gaussian blur provide only marginal robustness gains, while HSV jitter and grayscale provide none; final IoU resembles the standard-attack baseline.The EOT experiment uses 10 transformation samples on Deeplab v2 and Pascal VOC.

8. Effect of CRFs on Adversarial Robustness

CRF-based segmentation models appear more robust to untargeted white-box attacks, but this advantage reflects confident outputs rather than CRF smoothing itself. Black-box and targeted attacks substantially reduce or eliminate the apparent robustness.

  • CRF formulation: DenseCRF encourages nearby pixels with similar position or appearance to share labels through pairwise potentials, motivating its evaluation against high-frequency adversarial noise.CRF-RNN implements DenseCRF mean-field inference as an RNN appended to FCN8s.
  • Untargeted attacks: CRF-RNN is markedly more robust than FCN8s to untargeted FGSM and Iterative FGSM attacks, but increasing pairwise smoothness does not increase robustness.Robustness correlates instead with the confidence and entropy of final predictions.
  • Mechanism: Mean-field inference makes marginals more confident, reducing the loss and weakening attacks based on gradients of the final loss with respect to the input.This confidence effect arises as a by-product of the CRF-based technique rather than from smoothing itself.
  • Circumventing the CRF: Black-box attacks generated from FCN8s transfer effectively to CRF-RNN and can outperform attacks generated from CRF-RNN’s output layer.This pattern is described as a gradient-masking effect for ineffective white-box attacks.
  • Circumventing the CRF: Targeted FGSM and Iterative FGSM attacks make CRF-RNN and FCN8s barely differ in adversarial robustness.Targeted losses use the least likely label rather than the network’s highly confident prediction, producing a stronger gradient signal.
  • Discussion: The same observations hold when DenseCRF is used as DeepLab v2 post-processing, showing that end-to-end CRF training does not determine adversarial robustness.The discussion concludes that CRF smoothing itself has no impact on DNN adversarial robustness.

9. Conclusion

The paper evaluates adversarial robustness in semantic segmentation and identifies architectural and processing choices associated with greater robustness. It recommends robust benchmarking and favors inherently robust models for safety-critical applications.

  • The paper presents a rigorous evaluation of semantic segmentation robustness to adversarial attacks.
  • Residual connections are more robust than chain-like networks, including in models with few parameters.
  • Multiscale processing and other input transformations improve robustness when attack generation does not account for those transformations.
  • Mean-field inference in Dense CRFs confers robustness to untargeted attacks through gradient masking, not through smoothness priors.
  • Deeplab v2, based on ResNet and using multiscale processing, is recommended for safety-critical applications because of its inherent robustness.
  • Robustness should be evaluated with white-box attacks that exploit knowledge of proposed defenses, rather than relying only on clean-input accuracy.

B. Qualitative results

The qualitative results visualize perturbations and compare attack methods and network responses. Iterative attacks and larger perturbation norms more strongly degrade segmentation predictions.

  • Perturbations become visible to the naked eye only at approximately ǫ = 8 when viewed on screen.
  • The qualitative experiments compare FGSM, FGSM ll, Iterative FGSM, and Iterative FGSM ll attacks on segmentation outputs.
  • Iterative attacks are more effective than single-step attacks on the Deeplab v2 Multiscale ASPP network.
  • Higher l∞ perturbation norms further degrade the network’s prediction.
  • Under Iterative FGSM ll, PSPNet is the most robust among ICNet, Dilated Context, and PSPNet.

C.1. Results of other attacks

Additional attacks largely reproduce the paper’s main robustness patterns across Pascal VOC and Cityscapes, while revealing dataset- and attack-dependent differences. ResNet-based models and PSPNet are often robust, but clean-input accuracy does not reliably predict attacked performance.

  • ResNet-based networks are more robust than VGG-based models across the additional attack evaluations.
  • DilatedNet without its Context module is typically more robust than the full, more accurate network.
  • On Cityscapes, single-step FGSM ll is especially effective at high ǫ values and can outperform iterative attacks, unlike on Pascal VOC.
  • PSPNet achieves the highest clean-input IoU but is typically not the most robust model on Pascal VOC.
  • Across 32 Pascal VOC cases, PSPNet has the highest absolute IoU only 2 times and never for 0 < ǫ ≤4.
  • The highest Pascal VOC absolute IoU at any ǫ is always achieved by a ResNet-based model.

D.1.2 Transferability experiments using the FGSM ll and Iterative FGSM attacks

Multiscale processing improves robustness and changes adversarial transfer patterns across scales. These findings extend beyond Deeplab v2 to FCN8s, while CRF experiments link robustness to prediction confidence rather than smoothness alone.

  • D.1.2 Transferability experiments using the FGSM ll and Iterative FGSM attacks: Black-box attacks generated from multiscale Deeplab v2 transfer better to its single-scale versions than attacks generated from individual scales.
  • D.1.3 Transferability experiments at multiple ǫ values: Transferability between individual scales varies greatly; attacks from the 50% scale transfer poorly to 100% and vice versa.
  • D.1.2 Transferability experiments using the FGSM ll and Iterative FGSM attacks: FCN8s shows the same pattern: multiscale processing improves white-box robustness and produces more transferable black-box attacks.
  • E.1. Adversarial Robustness and Smoothing: Increasing CRF smoothness does not increase FGSM robustness, whereas prediction confidence correlates with the IoU Ratio.

E.3. Experiments on Deeplab v2 with CRF

The experiments evaluate Deeplab v2 with an appended CRF-RNN layer, enabling gradient-based adversarial attacks while matching the original post-processing outputs. They examine how CRF parameters and prediction confidence relate to robustness on Pascal VOC.

  • Experimental setup: Appending CRF-RNN to Deeplab v2 enables gradients to be backpropagated through the CRF layer for adversarial attacks.The appended layer uses the original authors’ parameters and produces identical outputs to their post-processing implementation.
  • CRF parameter effects: Lower filter bandwidths, including approximately θα = 1, provide greater robustness against FGSM than larger bandwidths.Increasing filter bandwidth visually smooths predictions but does not increase adversarial robustness.
  • CRF parameter effects: Increasing the Gaussian-term weight w2 tends to increase robustness, while lower θγ filter bandwidths also tend to be more robust.The figure varies w2 and θγ under FGSM attacks on Pascal VOC.
  • Prediction confidence: FCN8s consistently has the lowest mean probability for its highest-scoring pixel class under FGSM across tested ǫ values.The comparison uses Pascal VOC validation images and includes CRF-RNN variants with different filter bandwidths.
  • Prediction confidence: FCN8s consistently has the highest mean label entropy, indicating the least confident predictions among the compared models.The entropy comparison is performed over Pascal VOC validation images for multiple FGSM ǫ values.
Loading 1711.09856v3…