Source-linked AI summary
Attack-SAM: Towards Attacking Segment Anything Model With Adversarial Examples
Chenshuang Zhang, Chaoning Zhang, Taegoo Kang, Donghun Kim, Sung-Ho Bae, In So Kweon
TL;DR
It was unclear whether SAM, a prompt-based mask-prediction model, could be fooled by adversarial examples. Attack-SAM investigates mask removal and transfer attacks, as well as attacks aimed at generating desired masks. The reported findings include successful mask removal, transfer attacks with differing degrees of success, and desired-mask generation in most cases.
Problem
It was unclear whether SAM’s prompt-based mask prediction, which produces masks without semantic labels, is vulnerable to adversarial attacks.
Source: §1 · p. 1, §2.2 · p. 3Method
Attack-SAM formulates mask removal as its basic attack and evaluates white-box and transfer-based attacks, then tests attacks targeting desired masks.
Source: §1 · p. 2Results
The authors report successful white-box mask removal, cross-prompt transfer without knowing the point prompt, partial cross-task transfer, and desired-mask generation in most cases.
Source: §1 · p. 2Takeaways & Limitations
The findings show that SAM is vulnerable to adversarial examples across the investigated attack settings, including attacks aimed at producing desired masks.
Source: §1 · p. 2, §8 · p. 11Cross-task attack success is partial, generated masks can contain noise in unintended regions, and the reference masks explored are limited in diversity.
Source: §7 · p. 11
Abstract
from arXiv · showhide
Segment Anything Model (SAM) has attracted significant attention recently, due to its impressive performance on various downstream tasks in a zero-short manner. Computer vision (CV) area might follow the natural language processing (NLP) area to embark on a path from task-specific vision models toward foundation models. However, deep vision models are widely recognized as vulnerable to adversarial examples, which fool the model to make wrong predictions with imperceptible perturbation. Such vulnerability to adversarial attacks causes serious concerns when applying deep models to security-sensitive applications. Therefore, it is critical to know whether the vision foundation model SAM can also be fooled by adversarial attacks. To the best of our knowledge, our work is the first of its kind to conduct a comprehensive investigation on how to attack SAM with adversarial examples. With the basic attack goal set to mask removal, we investigate the adversarial robustness of SAM in the full white-box setting and transfer-based black-box settings. Beyond the basic goal of mask removal, we further investigate and find that it is possible to generate any desired mask by the adversarial attack.
1 Introduction
SAM performs prompt-based mask prediction without semantic labels, raising the question of whether it is vulnerable to adversarial examples. Attack-SAM studies mask removal and transfer attacks, and also tests whether attacks can produce desired masks.
1 Introduction
- SAM’s prompt inputs and unlabeled mask outputs distinguish its prediction task from the label-prediction targets of conventional attacks.The paper frames its investigation around prompt-based mask prediction rather than image- or pixel-level label prediction.Source: §1 · pp. 1–2
- Cross-prompt transfer can succeed without knowing the point prompt, while cross-task attacks from semantic label prediction achieve partial success on SAM.The paper examines both transfer settings beyond the full white-box attack.Source: §1 · p. 2
- Adversarial attacks can also generate desired masks, including manually designed masks and masks sourced from the same or another image.The authors report that desired masks are generated properly in most cases.Source: §1 · p. 2
- Attack-SAM formulates mask removal as its basic attack goal and uses ClipMSE to attack SAM in the full white-box setting.The authors report that the attacks successfully remove the predicted mask, indicating vulnerability to adversarial examples.Source: §1 · p. 2
2 Related work
Prior work has explored SAM’s performance and applications, while adversarial-attack research has largely targeted label prediction. This paper addresses whether attacks can target SAM’s prompt-based masks, which have no semantic labels.
2.1 Segment Anything Model (SAM)
- Work on SAM has examined performance in challenging scenarios and extended its utility to text-guided segmentation, image editing, tracking, and 3D reconstruction.The cited examples include medical images, camouflaged objects, transparent objects, and mirrors, where SAM often fails to detect objects.Source: §2.1 · p. 2
2.2 Adversarial attacks
- Prior adversarial attacks target image-level labels, whereas this work investigates prompt-based mask prediction, whose masks have no semantic labels.The paper notes that SAM’s robustness to adversarial attacks remained unclear.Source: §2.2 · pp. 2–3
3 Framework for Attack-SAM
SAM takes an image and prompt to produce a mask, while conventional attacks optimize image perturbations against a target model. Attack-SAM makes mask removal its objective and designs a threshold-based loss for that task.
3.1 Preliminaries
- SAM takes images and prompts as inputs and produces masks without semantic labels; prompts can yield multiple image–prompt–mask pairs for an image.The model’s per-pixel output indicates confidence of being masked, and positive values define pixels included in the predicted binary mask.Source: §3.1 · p. 3
- FGSM is a single-step gradient attack, while PGD iteratively applies gradient-based updates to a target model.The paper uses FGSM and PGD-N terminology for these attack methods.Source: §3.1 · p. 3
3.2 Attack-SAM
- Attack-SAM defines success as making predicted values in the target area negative, thereby removing the mask.The attack objective reduces SAM’s predicted pixel values until they fall below zero.Source: §3.2 · p. 3
- The loss pushes SAM’s perturbed output toward a negative threshold, and ClipMSE clips values below that threshold to produce zero gradients for those pixels.The stated attack settings use a maximum perturbation magnitude of 8/255 and set Negth to −10.Source: §3.2 · pp. 3–4
4 Main results of Attack-SAM
In the point-prompt setup, attacks are evaluated by whether SAM’s prompted mask is removed. FGSM and PGD reduce predicted mask area, and PGD-10 with ClipMSE produces the lowest mIoU among the reported settings.
4 Main results of Attack-SAM
- For the basic setup, a randomly selected point-prompted mask is attacked; success means its adversarial mask is empty or much smaller than the clean mask.Source: §4 · p. 4
4.1 Qualitative results
- With imperceptible perturbations, both FGSM and PGD remove mask area, and PGD leaves the tinier predicted mask.Source: §4.1 · p. 4
4.2 Quantitative results
- Table 1 reports much lower mIoU for FGSM and PGD-10 than no attack, with PGD-10 using ClipMSE lowest; ClipMSE also significantly outperforms MSE under PGD.Source: Table 1 · p. 5, §4.2 · p. 5
4.3 Attack SAM under other forms of prompts
- Experiments mainly use point prompts; SAM also accepts bounding-box prompts, with corresponding results deferred to the appendix.Source: §4.3 · p. 5
5 Transfer-based attacks
Transfer-based attacks test whether SAM’s vulnerability extends beyond the prompt or task used to generate an adversarial image. The study examines cross-prompt and cross-task transfer in black-box settings.
5.1 Cross-prompt transfer
- An image adversarially generated for one source prompt reduces the quality and area of masks predicted for a different target prompt, but valid masks remain.The preliminary comparison uses the same target prompt with a clean image and the adversarial image.Source: §5.1 · p. 5, Figure 2 · p. 6
- Increasing the number of source prompts (K) makes cross-prompt mask removal more successful, including across multiple target prompts.The reported mIoU decreases as K increases; the predicted masks also shrink or reduce across multiple image-prompt pairs.Source: Table 2 · p. 6, Figure 3 · p. 6, §5.1 · p. 6
5.2 Cross-task transfer
- A classification-trained ViT adversarial image removes several SAM masks across multiple prompts, but many masks remain.This is a preliminary cross-task transfer investigation.Source: §5.2 · p. 7, Figure 5 · p. 7
6 Beyond mask removal
Section 6 moves beyond removing masks to changing their area or position and generating new target masks. It tests mask enlargement, same-image manipulation, and three settings for desired-mask generation.
6.1 Mask enlargement
- Adversarial attack can enlarge SAM’s predicted mask, in some cases making the entire image a single mask from a randomly selected point prompt.The enlarged adversarial mask is compared with the clean-image mask in Figure 6.Source: §6.1 · p. 8, Figure 6 · p. 8
6.2 Mask Manipulation
- Mask shift attacks can duplicate or translate an existing mask, with translation more difficult because it also requires removing the original mask.Figure 7 illustrates both the duplicated and translated outcomes.Source: §6.2 · p. 8, Figure 7 · p. 9
- Mask flipping attacks can duplicate or flip the original mask horizontally or vertically.The reported examples include both duplication and flipping outcomes.Source: §6.2 · p. 8, Figure 8 · p. 9
6.3 Towards generating any desired mask
- Manually designed circle or square masks at random positions can be generated despite a prompt that originally predicts a dog mask.The authors note that designing more complex masks manually is non-trivial, motivating their use of SAM-generated reference masks in the other settings.Source: §6.3 · p. 9, Figure 9 · p. 9
- Using a different point prompt on the same image as the target can add the other prompt’s mask, though the original mask may not be fully removed.In the example, the adversarial prediction includes human legs while the original dog region remains partly present.Source: §6.3 · p. 10, Figure 10 · p. 10
- A mask from another image can be generated on the attacked image, including a cow mask on a dog image and vice versa.The target mask comes from the reference image, while the attacked image uses a random point prompt.Source: §6.3 · p. 10, Figure 11 · p. 10
7 Discussion
The discussion frames mask removal as untargeted and desired-mask generation as targeted attack. It reports partial success in challenging cross-task settings and imperfections in generated masks.
7 Discussion
- Mask removal is conceptually untargeted, while generating a desired mask is similar to a targeted attack because the prediction must align with a predetermined target.The discussion contrasts attacks that change the clean prediction with attacks aimed at a specified mask.Source: §7 · p. 10
- Cross-task attacks succeed only partially in challenging settings, and desired masks may contain unintended noise while reference-mask types remain limited.The authors identify removing noise masks and exploring more diverse reference masks as future work.Source: §7 · p. 11
8 Conclusion
The conclusion reports adversarial vulnerability across mask removal, transfer, and desired-mask generation, while noting that stronger attacks and improved robustness remain future work.
8 Conclusion
- In the full white-box setting, adversarial examples successfully remove SAM’s original mask, and attacking multiple prompts can enhance cross-prompt transfer when the prompt is unknown.Examples generated for semantic label prediction can also attack mask prediction to some extent, and desired-mask generation has overall satisfactory performance.Source: §8 · p. 11
- The authors adapt common attack methods to investigate SAM’s robustness rather than seek the strongest attack, and call for safety scrutiny in safety-critical deployments.They identify stronger attack methods and improving SAM’s robustness as future work.Source: §8 · p. 11