Source-linked AI summary
Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks
Federico Nesti, Giulio Rossolini, Saasha Nair, Alessandro Biondi, Giorgio Buttazzo
TL;DR
Real-world adversarial examples for semantic segmentation are relatively underexplored, despite SS’s role in autonomous-driving perception. The paper evaluates patch attacks across Cityscapes, CARLA, and outdoor driving, finding limited effectiveness in realistic settings and questioning their practical relevance.
Problem
Real-world adversarial examples for semantic segmentation remain relatively unexplored, although SS is integral to autonomous-driving pipelines.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
The findings question the practical relevance of real-world adversarial patches against SS models for autonomous driving.
Source: §1 · p. 2Scene-specific optimization requires a digital target-scene representation and manual 3D mesh work; CARLA-to-real-world transfer issues remain uninvestigated.
Source: §3.4 · p. 4
Abstract
from arXiv · showhide
Deep learning and convolutional neural networks allow achieving impressive performance in computer vision tasks, such as object detection and semantic segmentation (SS). However, recent studies have shown evident weaknesses of such models against adversarial perturbations. In a real-world scenario instead, like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs), which are physical objects (e.g., billboards and printable patches) optimized to be adversarial to the entire perception pipeline. This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches. These patches are crafted with powerful attacks enriched with a novel loss function. Firstly, an investigation on the Cityscapes dataset is conducted by extending the Expectation Over Transformation (EOT) paradigm to cope with SS. Then, a novel attack optimization, called scene-specific attack, is proposed. Such an attack leverages the CARLA driving simulator to improve the transferability of the proposed EOT-based attack to a real 3D environment. Finally, a printed physical billboard containing an adversarial patch was tested in an outdoor driving scenario to assess the feasibility of the studied attacks in the real world. Exhaustive experiments revealed that the proposed attack formulations outperform previous work to craft both digital and real-world adversarial patches for SS. At the same time, the experimental results showed how these attacks are notably less effective in the real world, hence questioning the practical relevance of adversarial attacks to SS models for autonomous/assisted driving.
1. Introduction
Real-world adversarial patches for semantic segmentation remain underexplored despite SS being integral to autonomous-driving perception. This work evaluates patches from Cityscapes through CARLA to outdoor driving, finding that their effectiveness is limited in the more realistic settings.
1. Introduction
- RWAEs remain relatively unexplored for semantic segmentation, although SS is integral to autonomous-driving pipelines and physical attacks do not require access to the camera image’s digital representation.The paper benchmarks real-time SS models against RWAEs in autonomous-driving scenarios.Source: §1 · pp. 1–2
- The extended cross-entropy loss focuses optimization on pixels not yet misclassified, producing more powerful patches than the standard cross-entropy setting [21].Source: §1 · p. 2
- Cityscapes experiments extend EOT to SS and compare robust patches against non-robust patches crafted without EOT.Cityscapes is described as a benchmark of high-resolution urban-driving images.Source: §1 · p. 2
- The scene-specific attack uses CARLA geometry to craft patches for a particular driving scene and a billboard whose position is known.The CARLA experiments compare scene-specific, EOT-based, and non-robust patches on billboards in a virtual 3D world.Source: §1 · p. 2
- The outdoor evaluation collected city data, optimized and printed a patch on a billboard, then tested SS models on images containing it.Source: §1 · p. 2
- Non-robust patches attack Cityscapes images but fail in CARLA, while EOT and scene-specific patches attack both virtual and real settings with limited effectiveness that questions RWAEs’ practical relevance.The robust patches were less effective than non-robust ones on Cityscapes images.Source: §1 · p. 2
2. Related Work
Prior physical adversarial-attack research spans several vision tasks, but evidence for real-world attacks on semantic segmentation in autonomous driving remains limited. This work builds on that gap and also proposes a more general SS patch-optimization loss.
2. Related Work
- Digital perturbations offer convenient benchmarks but do not extend well to real-world applications; EOT models transformations such as rotation and lighting changes during optimization.EOT optimizes loss in expectation over selected transformations.Source: §2 · pp. 2–3
- Physical attacks have been studied for classification [4], object detection [36] [35] [18], optical flow [25], LiDAR detection [34], and depth estimation [39], but autonomous-driving evaluation is challenging because it requires controlling 3D outdoor environments.Prior work identifies CARLA as a viable virtual 3D environment for crafting and evaluating adversarial situations.Source: §2 · p. 3
- Nakka et al. [21] studied local SS attacks, but not real-world robustness; this paper addresses that gap and replaces standard pixel-wise cross-entropy with a more general SS patch-optimization loss.Source: §2 · p. 3
3. Attack Formulation
The formulation combines a patch-optimization pipeline with EOT-based and scene-specific placement strategies, and introduces a loss that separates correctly and incorrectly classified pixels outside the patch. The scene-specific strategy uses CARLA geometry for precise placement but requires a digital representation of the target scene.
3.1. Background on SS
- Semantic segmentation predicts a class for each pixel by selecting the class with the highest predicted probability.Model performance is evaluated using per-pixel cross-entropy against ground-truth labels.Source: §3.1 · p. 3
3.2. Patch-based attack pipeline
- Both attacks transform the patch’s appearance, place it on an image, and optimize expected loss; placement differs between randomized EOT and precise scene-specific projection.The pipeline applies the transformed patch to the image using a placement function; Figure 2 outlines both approaches.Source: §3.2 · pp. 3–4, Figure 2 · p. 3
3.3. EOT-based patch attack
- The EOT-based attack randomizes spatial transformations such as translation and scaling, and iteratively updates patch pixels using the loss gradient.Its loss also includes smoothness and non-printability terms to account for physical realizability.Source: §3.3 · p. 4
3.4. Scene-specific patch attack
- Scene-specific placement uses CARLA geometry to project the patch precisely onto an attackable surface, but requires a digital representation of the target scene.The method also requires modeling 3D objects; CARLA-to-real-world transfer issues are not investigated in this paper.Source: §3.4 · p. 4
3.5. Proposed loss function
- The loss excludes patch pixels and partitions the remaining pixels into correctly classified and misclassified sets before recombining their contributions.The correctly classified set comprises off-patch pixels whose predicted labels match the ground truth.Source: §3.5 · p. 5
- The proposed loss separates cross-entropy contributions from misclassified and still-correct pixels outside the patch, preventing the latter from being obscured during optimization.A parameter γ balances decreasing the number of correctly classified pixels against strengthening the attack on already misclassified pixels.Source: §3.5 · p. 5
- An adaptive γ is set from the proportion of correctly classified off-patch pixels, shifting emphasis from increasing misclassifications toward strengthening the attack on them.The paper compares multiple γ values with standard pixel-wise cross-entropy and describes the formulation as more general and effective for this attack.Source: §3.5 · p. 5
4. Experimental results
The experiments compare adversarial-patch optimization on Cityscapes and CARLA, then test a printed patch outdoors. They evaluate loss variants, EOT-based and scene-specific attacks, and real-world effects.
4.1. Experimental setup
- Cityscapes patches were optimized on 250 training images and evaluated on the full validation set; CARLA tests used fine-tuned models and billboard scenes with varied viewpoints.The experiments also collected an outdoor dataset of 1,000 images; CARLA attack datasets included 150 training, 100 validation, and 100 test images per billboard.Source: §4.1 · pp. 5–6
4.2. EOT-based patches on Cityscapes
- With EOT, the proposed loss reached a smaller mIoU in nearly 25 epochs, while standard cross-entropy continued reducing mIoU through 200 epochs; without EOT, all compared losses converged after 10 epochs.The proposed formulation converged to a higher attack effect for every tested γ value and did so in fewer epochs than pixel-wise cross-entropy.Source: §4.2 · p. 6
4.3. Scene-specific patches on CARLA
- Across almost all scene-network combinations, scene-specific patches outperformed EOT-based patches; the attacks had comparable performance in scene 3, where the billboard was nearly perpendicular to the camera.The authors also report that semantic-segmentation models were generally rather robust to patch attacks.Source: §4.3 · p. 7
4.4. Real-world patches
- The printed EOT-based patch altered a significant area of ICNet’s predicted segmentation, but distant image regions remained unaffected and the attack weakened as the patch moved farther from the camera.The patch was printed as a 1m × 2m poster; uncontrolled weather and practical constraints limited the outdoor tests.Source: §4.4 · p. 8
5. Conclusions and future work
Across increasingly realistic benchmarks, the study found that semantic-segmentation models remain somewhat robust to real-world patch attacks, especially in CARLA and outdoor tests. It also introduced a scene-specific attack and a novel loss function for patch optimization.
5. Conclusions and future work
- The scene-specific attack improves the EOT formulation for more realistic and effective patch placement.The method uses increasingly realistic evaluation settings, including a virtual 3D scenario and the real world.Source: §5 · p. 8
- The novel loss function proved a more general and efficient alternative to classic cross-entropy for adversarial patch optimization.Source: §5 · p. 8
- In CARLA and real-world tests, patches often affected only the area near the attacked surface, and models remained somewhat robust to real-world patch attacks.The attacks reduced baseline model accuracy, but predictions were not easily corrupted in more realistic settings.Source: §5 · p. 8
- Future work will investigate defense mechanisms and temporal information to enhance semantic-segmentation robustness.Source: §5 · p. 8