Source-linked AI summary
On the Robustness of Large Multimodal Models Against Image Adversarial Attacks
Xuanming Cui, Alejandro Aparcedo, Young Kyun Jang, Ser-Nam Lim
TL;DR
The impact of visual adversarial attacks on LMMs remains insufficiently examined, particularly when attacks target only their visual encoders. This paper evaluates robustness across tasks and attack settings and introduces context-based query decomposition for image classification. LMMs are generally vulnerable, but context and query-target mismatch are associated with greater resilience.
Problem
The study addresses limited systematic evidence on whether visual-encoder attacks compromise LMMs across tasks and datasets.
Source: §1 · pp. 1–2Method
The authors evaluate gradient-based white-box attacks on visual encoders across LMM tasks and propose query decomposition using class-specific existence questions for closed-world classification.
Source: §3.1 · p. 2, §4.5 · p. 8Results
Takeaways & Limitations
The findings indicate that task-specific adversarial generation is not universally effective against current LMMs, particularly in zero-shot inference.
Source: §5 · p. 8The reported VQA robustness is scoped to questions whose answers may be inferred from text or that concern peripheral image aspects rather than attacked object attributes.
Source: §4.2 · p. 5, §4.3 · p. 7
Abstract
from arXiv · showhide
Recent advances in instruction tuning have led to the development of State-of-the-Art Large Multimodal Models (LMMs). Given the novelty of these models, the impact of visual adversarial attacks on LMMs has not been thoroughly examined. We conduct a comprehensive study of the robustness of various LMMs against different adversarial attacks, evaluated across tasks including image classification, image captioning, and Visual Question Answer (VQA). We find that in general LMMs are not robust to visual adversarial inputs. However, our findings suggest that context provided to the model via prompts, such as questions in a QA pair helps to mitigate the effects of visual adversarial inputs. Notably, the LMMs evaluated demonstrated remarkable resilience to such attacks on the ScienceQA task with only an 8.10% drop in performance compared to their visual counterparts which dropped 99.73%. We also propose a new approach to real-world image classification which we term query decomposition. By incorporating existence queries into our input prompt we observe diminished attack effectiveness and improvements in image classification accuracy. This research highlights a previously under-explored facet of LMM robustness and sets the stage for future work aimed at strengthening the resilience of multimodal systems in adversarial environments.
1. Introduction
Although instruction-tuned LMMs perform across vision-language tasks, the impact of adversarial examples on them remains underexplored. This study finds broad vulnerability to visual attacks, with robustness varying by task context and improving when additional textual context is provided.
1. Introduction
- The study asks whether attacks on a visual encoder can compromise an LMM that combines visual and language models with a projector.The authors note that end-to-end attacks on large models can be prohibitively expensive, while attacks on individual components warrant reevaluation.Source: §1 · pp. 1–2
- Across tasks and datasets, LMMs are generally vulnerable to visual perturbations, but context and query-target mismatch can improve robustness.The study reports vulnerability without additional textual information, including COCO captioning and classification without context, and better robustness in contextual classification and some VQA settings.Source: §1 · p. 2
- In VQA, attacks are less effective when questions concern visual content different from the attacked object.The introduction describes LMMs as showing decent robustness in VQA under this condition.Source: §1 · p. 2
- The authors propose context-augmented image classification, using existence queries to improve robustness to visual attacks.Figure 1 illustrates that LLaVA can answer questions correctly when they concern content other than the attacked sheep, and after additional context is provided.Source: Figure 1 · p. 1, §1 · p. 2
2. Related Work
LMMs combine visual and language models with a projector, and prior adversarial work has examined both visual and textual inputs. This paper shifts attention from jailbreaks and safety to the accuracy of LMM task performance under visual attacks.
2. Related Work
- LMMs typically combine a visual model, a pretrained LLM, and a projector; LLaVA and BLIP2-based models use distinct encoders and bridging architectures.LLaVA pairs CLIP with Vicuna and a linear projector, whereas BLIP2-based models use EVA-CLIP and a Q-former.Source: §2 · p. 2
- Prior work studies adversarial examples in visual and textual domains, including attacks that jailbreak LMMs through gradients or prompt engineering.The cited jailbreak studies focus on harmful content and bypassing alignment measures.Source: §2 · p. 2
- This paper instead systematically examines LMM accuracy across tasks under visual adversarial attacks.The related-work passage contrasts this accuracy focus with prior emphasis on safety, harmfulness, and associated dangers.Source: §2 · p. 2
3. Method
The method evaluates gradient-based white-box attacks against image encoders in three vision-language tasks and compares the resulting encoder and LMM performance. Its setup includes multiple attack strengths and three LMM configurations, with task-specific adversarial generation and evaluation procedures.
3.1. Problem Statement
- The attack formulation seeks a perturbation that changes the model’s predicted label while satisfying an Lp constraint.PGD maximizes cross-entropy within an L∞ bound; CW maximizes a distance-plus-margin objective, with κ as its confidence parameter.Source: §3.1 · pp. 2–3
3.2. Attacks
- PGD, APGD, and CW are evaluated under normal and strong settings, with attacks generated only against the image encoder.Normal uses PGD/APGD ε=8/255 and CW constraint 20; strong uses ε=0.2 and constraint 100. Strong PGD/APGD perturbations are obvious, while CW perturbations remain imperceptible.Source: §3.2 · p. 3, Figure 2 · p. 3
3.3. Models
- The evaluated models are LLaVA, BLIP2-T5, and InstructBLIP, which vary in language model, visual encoder, or modality-bridging architecture.LLaVA inserts projected visual tokens into text tokens, while InstructBLIP uses a Q-former; BLIP2-T5 uses Flan T5 XXL.Source: §3.3 · p. 3
3.4. Tasks & Adversarial generation
- The study evaluates image classification, caption retrieval, and VQA, generating adversaries with respect to each LMM’s image encoder.PGD/APGD maximize cross-entropy against ground-truth labels, while CW minimizes perturbation distance plus its objective function.Source: §3.4 · p. 3
3.4.1 Image Classification
- For LMM image classification, generated object labels are encoded as text and matched by cosine similarity against class-label encodings.The prompts request a one-word or short-phrase description of the main object, with wording varying by model.Source: §3.4.1 · p. 4
3.4.2 Caption retrieval
- For caption retrieval, the method averages five encoded captions per image for attack generation and retrieves by similarity to caption encodings.For LMM evaluation, a generated image caption is encoded and compared against all caption encodings.Source: §3.4.2 · p. 4
3.4.3 VQA
- VQA robustness is evaluated on five datasets using their official evaluation procedures.For VQA datasets without ground-truth labels, LLaVA-generated synthetic captions are used to generate adversaries following the caption-retrieval procedure.Source: §3.4.3 · p. 4
4. Experimental Results and Analysis
Across caption retrieval, VQA, and classification experiments, visual attacks often undermine LMM performance, but question or class context can coincide with greater resilience. The section also tests query decomposition for closed-world classification and reports reduced attack-related accuracy drops.
4.1. Are LMMs Robust Against Adversarial Visual Input?
- Caption retrieval showed substantial post-attack accuracy declines across all three LMMs under PGD and APGD in both normal and strong settings.Under PGD-N, InstructBLIP’s Top-1 recall fell to 1.9%, and under PGD-S to 0.18%, roughly matching visual-encoder accuracy after attack.Source: §4.1 · p. 4, Table 2 · p. 5
4.2. Evaluating LMMs’ VQA Performance
- On ScienceQA, all three evaluated LMMs lost less than 7% accuracy relative to pre-attack accuracy, although PostN visual-encoder accuracy fell below 1% for both CLIP and BLIP visual encoders.Source: §4.2 · p. 5
4.3. Visual Adversarial Attacks are not Universal to LMMs
- VQA attack impact depended on query focus: drops were greatest for questions about the attacked object and its direct attributes, and diminished for questions about other aspects.For LLaVA under APGD-S, general descriptions deviated from the image while most VQA answers remained correct; the authors conjecture that context or peripheral questions may explain this pattern.Source: §4.3 · pp. 5, 7, Figure 4 · p. 5, Figure 5 · p. 7
4.4. Adding Context Improves LMM Robustness
- Adding a short description of the correct object increased post-attack accuracy for all three LMMs, reducing BLIP2/InstructBLIP accuracy drops under PGD/APGD to 20% versus an average 60% without context.The authors suggest the added context may help recover object attributes from corrupted visual input.Source: §4.4 · p. 7, Table 4 · p. 7
- In some strong-attack settings, BLIP2-T5 and InstructBLIP accuracy exceeded normal-setting accuracy and, under APGD-S, pre-attack accuracy; the authors conjecture this reflects hallucination when visual input is ignored.They propose that BLIP’s Q-former may let text outweigh visual input, unlike LLaVA’s separate visual and text tokens.Source: §4.4 · p. 8
4.5. Towards Real-World Application: Context-Augmented Image Classification
- Query decomposition handles unknown correct context in closed-world classification by asking a context-augmented existence question for each class and selecting the highest-confidence class.The authors note that the class queries can be run in parallel.Source: §4.5 · p. 8
- With query decomposition, attack-related accuracy drops on COCO were mostly 10% smaller and on ImageNet 20% smaller than without context; pre-attack ImageNet classification accuracy also increased.The approach asks class-specific existence questions with corresponding context, then selects the class with the highest confidence.Source: §4.5 · p. 8, Table 5 · p. 8
5. Conclusion
Across diverse tasks and datasets, LMMs are vulnerable to visual adversarial attacks, but their robustness varies with the relationship between the query and attack target. Adding context about the queried object improves visual robustness, and question decomposition improves robustness on COCO and ImageNet classification.
5. Conclusion
- LMMs are highly vulnerable to visual adversarial attacks, even when the attacks target only the visual model.The evaluation spans diverse tasks and datasets.Source: §5 · p. 8
- LMMs appear robust when the query and attack target do not match, so task-specific attack techniques are not universally effective against current LMMs.The authors identify zero-shot inference as a particular area for further attack-strategy research.Source: §5 · p. 8
- Adding context about the queried object improves visual robustness, while decomposing questions into existence queries improves robustness on COCO and ImageNet classification.Source: §5 · p. 8
6. Attacks
The study uses three untargeted attacks and compares normal and strong settings. APGD perturbations are most perceptible, while CW perturbations are least perceptible, including in the strong setting.
6. Attacks
- APGD produces the most perceptible perturbations, whereas CW produces the least perceptible, even under the strong setting.All three attacks are imperceptible under the normal setting.Source: §6 · p. 11
- EVA-CLIP-based perturbations are generally more perceptible than CLIP-based perturbations.The comparison also highlights a patch border for EVA-CLIP under APGDS that is absent for CLIP.Source: §6 · p. 11
7. VQA Per Question-type Accuracy Drop
Across LLaVA, BLIP2-T5, and InstructBLIP, adversarial attacks reduce accuracy most on questions about object types or attributes.
7. VQA Per Question-type Accuracy Drop
- Accuracy drops most on questions querying object types or attributes across the three tested LMMs.Examples include questions asking what animal, room, kind, or type is shown.Source: §7 · p. 11
- The reported per-question accuracy-drop pattern is consistent across LLaVA, BLIP2-T5, and InstructBLIP.Source: §7 · p. 11
- Questions about animal, room, kind, or type are examples of the query categories with the largest accuracy drops.Source: §7 · p. 11
8. LLM Responses to Adversarial Visual Questions
Under strong attacks, APGD can disrupt all three LMMs’ image descriptions while leaving answers to peripheral questions, especially about backgrounds, correct. CW has little impact on the LMMs despite low post-attack classification accuracy for CLIP and EVA-CLIP.
8. LLM Responses to Adversarial Visual Questions
- APGD causes all three LMMs to give completely incorrect image descriptions, yet they still answer peripheral questions, especially about backgrounds, correctly.These response examples are from the strong attack setting.Source: §8 · p. 11
- CW has little impact on all three LMMs, despite relatively low post-attack classification accuracy for CLIP and EVA-CLIP.Source: §8 · p. 11
- The response comparisons cover LLaVA, BLIP2-T5, and InstructBLIP with APGDS and CWS examples generated using CLIP or EVA-CLIP.The figures show clean responses when the adversarial response differs.Source: Figure 8 · p. 13, Figure 9 · p. 14