Source-linked AI summary
How Robust is Google's Bard to Adversarial Image Attacks?
Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu
TL;DR
The paper asks whether commercial multimodal models, particularly Google’s Bard, are robust to adversarial image attacks despite limited prior evidence. It uses transfer-based attacks optimized on white-box surrogate vision encoders or MLLMs, then evaluates Bard and its defenses. The attacks achieve a 22% success rate against Bard, transfer to other commercial MLLMs, and evade Bard’s face and toxicity defenses.
Problem
The robustness of commercial MLLMs such as Bard remains underexplored, although vision inputs can inherit adversarial vulnerabilities and create security and safety risks.
Method
The paper uses transfer-based attacks optimized against image embeddings or target text descriptions on white-box surrogate models, evaluating Bard’s image-description, face-detection, and toxicity-detection defenses.
Results
22% attack success rate was achieved against Bard, while adversarial examples also reached 45% against GPT-4V, 26% against Bing Chat, and 86% against ERNIE Bot.
Takeaways & Limitations
The results demonstrate vulnerability of commercial MLLMs under black-box adversarial attacks and show that Bard’s current defenses can be easily evaded.
Takeaways & Limitations
The evaluation focuses on the image-description task, which the authors note is not very harmful in some respects, rather than more harmful multimodal attacks.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) that integrate text and other modalities (especially vision) have achieved unprecedented performance in various multimodal tasks. However, due to the unsolved adversarial robustness problem of vision models, MLLMs can have more severe safety and security risks by introducing the vision inputs. In this work, we study the adversarial robustness of Google's Bard, a competitive chatbot to ChatGPT that released its multimodal capability recently, to better understand the vulnerabilities of commercial MLLMs. By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability. We show that the adversarial examples can also attack other MLLMs, e.g., a 26% attack success rate against Bing Chat and a 86% attack success rate against ERNIE bot. Moreover, we identify two defense mechanisms of Bard, including face detection and toxicity detection of images. We design corresponding attacks to evade these defenses, demonstrating that the current defenses of Bard are also vulnerable. We hope this work can deepen our understanding on the robustness of MLLMs and facilitate future research on defenses. Our code is available at https://github.com/thu-ml/Attack-Bard. Update: GPT-4V is available at October 2023. We further evaluate its robustness under the same set of adversarial examples, achieving a 45% attack success rate.
1 Introduction
MLLMs inherit vision models’ adversarial vulnerabilities, while commercial systems such as Bard remain comparatively unexplored. This work evaluates transfer-based attacks against Bard’s image descriptions and defenses, finding vulnerabilities across Bard and other commercial MLLMs.
- MLLMs introduce a high-dimensional image attack surface, and vulnerabilities in vision encoders can create practical security and safety risks.MLLMs commonly extract image features with pretrained vision encoders before generating language.
- Commercial MLLM robustness remains underexplored because black-box access, larger models, unknown training configurations, and defense mechanisms complicate attacks.Prior work mainly examined open-source MLLMs, motivating a study of Bard.
- 22% attack success rate was achieved against Bard’s image-description task using transfer-based attacks on image embeddings or target text descriptions.The attacks used surrogate models and an ℓ∞ perturbation budget of ϵ = 16/255.
- The study identifies face detection and toxicity detection as Bard defenses and shows that corresponding adversarial attacks can evade them.These defenses are intended to protect face privacy and avoid abuse.
- The work aims to deepen understanding of black-box MLLM weaknesses and support research on more robust and trustworthy multimodal foundation models.The authors also discuss broader practical impacts and potential robustness solutions.
2 Related work
Prior research established adversarial-robustness concerns for MLLMs and developed both white-box, black-box, query-based, and transfer-based attack approaches. These works span image captioning, toxic-content generation, and other multimodal attack settings.
- Related MLLM research builds on the broader progress of LLMs and uses different strategies to connect visual information with language generation.Some approaches use learnable queries to extract visual information for LLM conditioning.
- MLLM adversarial-robustness studies examine image captioning, black-box attacks, and adversarial images that induce toxic content.The cited studies include white-box, transfer-based, and query-based evaluations.
- Black-box attacks are categorized as query-based or transfer-based, with transfer-based methods using local surrogate models instead of repeated victim-model queries.Query-based methods estimate gradients through repeated model invocation, increasing cost.
3 Attack on image description
The paper attacks Bard’s image-description pipeline with transfer-based adversarial images generated from surrogate vision encoders or MLLMs. Image embedding attacks achieve 22% success against Bard, while evaluations show transferability to other commercial MLLMs.
- Attack method: Image embedding attacks maximize divergence between adversarial and natural image embeddings, whereas text description attacks target a specified output sentence.The attacks operate through surrogate vision encoders or surrogate MLLMs and use transfer-based optimization methods such as SSA-CWA.
- Attack method: Targeted text attacks are used because minimizing one ground-truth description could still allow the model to produce another correct description.The target sentence is optimized through its predicted next-word likelihood.
- Results: 22% success rate is achieved by the image embedding attack against Bard, compared with 10% for the text description attack.The experiments use 100 NIPS17 images and an ℓ∞ perturbation budget of ϵ = 16/255.
- Experimental analysis: Attack success increases with the number of surrogate models, leading the study to ensemble three models as a balance between efficacy and time complexity.This finding comes from an ablation using 20 images for the image embedding attack.
- Transfer to other MLLMs: 45%, 26%, and 86% attack success rates are reported against GPT-4V, Bing Chat, and ERNIE Bot, respectively.Bing Chat rejects 30% of adversarial images, while GPT-4V more often gives vague descriptions instead of rejecting them.
- Transfer to other MLLMs: Commercial MLLMs exhibit similar adversarial robustness issues, although Bard is the most robust and ERNIE Bot the least robust among the studied models.The comparison is based on the reported attack success rates under the same adversarial examples.
4 Attack on defenses of Bard
Bard uses face and toxicity detection to reject sensitive images, but transfer-based adversarial examples can evade both defenses under black-box conditions. Attack success varies with perturbation budget and image resolution for face detection, while toxicity-detector evasion reached 36%.
- Defense mechanisms: Bard rejects images containing human faces or toxic content through face and toxicity detection mechanisms.These defenses are intended to protect privacy and avoid abuse.
- Examples: Successful attacks against Bard’s face and toxicity defenses are illustrated in Figures 6 and 7.The figures show screenshots of successful attacks against the respective detection mechanisms.
- Face detection: Transfer-based attacks successfully evade Bard’s face detector even when its detailed model configurations are unknown.The evaluation used ensemble attacks with three public face detection surrogate models on FFHQ and LFW images.
- Face detection: Face-detector attack success increases with larger perturbation budgets and decreases with higher image resolution.The reported relationship concerns the perturbation budget ϵ and image resolution.
- Toxicity detection: 36% attack success rate was achieved against Bard’s toxicity detector using adversarial noises on 100 manually collected toxic images.The detector failed to identify some violent, bloody, or pornographic images, after which Bard provided inappropriate descriptions.
5 Discussion and Conclusion
The study demonstrates that commercial MLLMs remain vulnerable to black-box adversarial image attacks, with adversarial examples transferring across multiple systems and evading Bard’s defenses. It also identifies limitations of current defense strategies and motivates further work on robust, trustworthy multimodal models.
- 22% attack success rate was achieved against Bard on image description, while Bing Chat and ERNIE Bot reached 26% and 86%, respectively.The attacks optimized image-embedding or text-description objectives using transfer-based methods.
- The results demonstrate vulnerability of commercial MLLMs under black-box adversarial attacks.
- The study evaluates image description rather than black-box attacks against commercial MLLM alignment, which it leaves for future work.
- Current defenses such as adversarial training face accuracy, computational-cost, and threat-generalization trade-offs.The paper notes that adversarial training can degrade MLLM performance, require much longer training, and fail to generalize across perturbation types.
- Preprocessing-based defenses are proposed as more suitable for large-scale foundation models because they can operate in a plug-and-play manner.Generative purification methods are identified as promising defense strategies.