A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Motivation

• LVLMs’ integration of visual and language processing introduces attack vectors absent from unimodal systems, including image perturbations and malicious prompts paired with visual inputs.The paper describes these multimodal vulnerabilities as making LVLM security important to explore.Source: §I · p. 1
• The survey addresses a gap left by prior reviews, which incompletely cover LVLM attack categories and developments across modalities.The cited reviews focus on LLM textual prompts, offer a coarse LVLM safety taxonomy, or cover image-based attacks without textual prompt injection.Source: §I · p. 2

Contribution

• The survey presents itself as the first comprehensive overview of single- and multimodal LVLM attacks, examining their methods, impacts, and exploited vulnerabilities.It aims to help practitioners evaluate attack trade-offs and developers recognize limitations when designing defenses.Source: §I · p. 2

Survey scope

• The survey reviews attack background, including definitions, tools, data processing, and evaluation across different settings.It also provides method comparisons and discusses potential future directions.Source: §I · p. 2
• The survey categorizes LVLM attacks as adversarial, jailbreak, prompt injection, and data poisoning/backdoor attacks.Its introduction also describes risks associated with adversarial outputs, unauthorized actions, prompt manipulation, and tampered training data.Source: §I · p. 2

A. Preliminary of LVLM Attack

• The preliminary section introduces notations, formulations, and concepts for understanding adversarial attacks on LVLMs.Source: A. Preliminary of LVLM Attack · p. 2

1) Notations and Definitions:

• An LVLM M maps an input image x and prompt t_in to textual output t_out; x belongs to X and t_in belongs to T.Source: 1) Notations and Definitions · p. 2
• P(x) and P(t_in) denote image and text attack operations, t′_out is the attacker-chosen output, and ε constrains perturbation magnitude.Source: 1) Notations and Definitions · pp. 2–3

• Attack operations P alter both image x and text t_in to make the model produce an incorrect, attacker-chosen, or jailbreak output.Source: 2) Attack Formulation · p. 3
• For adversarial attacks, P is optimized against a loss on the model output, with targeted and untargeted objectives distinguished.Source: 2) Attack Formulation · p. 3
• The perturbation is constrained by ||P||_p ≤ ϵ, while L constrains the model’s prediction.Source: 2) Attack Formulation · p. 3

B. The challenges of LVLM Attack

• Attacks must manipulate image-text interactions, making multimodal adversarial examples more complex than attacks on a single modality.Source: B. The challenges of LVLM Attack · p. 3
• Large-scale LVLM architectures make attacks computationally expensive and require substantial resources and advanced optimization.Source: B. The challenges of LVLM Attack · p. 3
• Realistic black-box settings limit attackers to model queries, while multi-vulnerability or adaptive attacks require versatile strategies.Source: B. The challenges of LVLM Attack · p. 3
• High-quality attacks must preserve benign-sample semantics for imperceptibility and transfer across models and tasks.Source: B. The challenges of LVLM Attack · p. 3
• Black-box complexity makes it difficult to explain why an attack succeeds or fails, while adaptive defenses demand evolving strategies and substantial resources.Source: B. The challenges of LVLM Attack · p. 3

1) White-box attack tools:

• White-box tools exploit full access to model architecture, parameters, and gradients; FGSM, PGD, and C&W generate or refine adversarial perturbations.Source: 1) White-box attack tools · p. 3

2) Gray-box attack tools:

• Gray-box attackers may use vision-encoder gradients or surrogate models to craft and test adversarial examples with partial model knowledge.Source: 2) Gray-box attack tools · pp. 3–4

3) Black-box attack tools:

• Black-box tools use output queries: SimBA accepts loss-increasing random perturbations, RGF estimates gradients by random sampling, and query-limited methods reduce queries.Source: 3) Black-box attack tools · p. 4

4) Datasets:

• General-task datasets supply annotated images or image-question-answer pairs for adversarial-example comparisons, while safety datasets provide toxic prompts or text-image attack materials.Source: 4) Datasets · p. 4, Table II · p. 4

5) LVLM models:

• LVLMs commonly combine a visual encoder, modality-bridging adapter, and LLM backbone, with representative open-source and proprietary models used in attacks.Source: 5) LVLM models · p. 4, Table III · p. 4

6) Evaluation metrics:

• ASR commonly measures successful attacks over total attacks, but task-specific evaluations also use target-content matching, semantic judgments, task metrics, and imperceptibility measures.Source: 6) Evaluation metrics · pp. 4–5

7) Defense strategies:

• Inference-time defenses include prompt engineering, jailbreak detection, and response or prompt correction; training-time defenses use robust prompts or feedback, fine-tuning, and contrastive learning.Source: 7) Defense strategies · p. 5, Table IV · p. 5

III. METHODS

• The taxonomy groups LVLM attacks as adversarial, jailbreak, prompt injection, and data poisoning/backdoor attacks.The survey reviews work and characteristics within each category.Source: §III · p. 5

A. Adversarial Attacks

• Adversarial attacks perturb inputs to induce incorrect or undesirable outputs and are classified as white-box, gray-box, or black-box by model access.The perturbations are typically imperceptible to humans and exploit model vulnerabilities.Source: A. Adversarial Attacks · p. 5

1) White-box attacks:

• White-box attacks use access to model architecture, parameters, and gradients to optimize input noise, with targeted attacks seeking specified outputs and untargeted attacks degrading output quality.Common tools include PGD, APGD, and CW; Fu et al. [40] also use trojan-like images to induce attacker-specific tool or API calls.Source: 1) White-box attacks · p. 5

2) Gray-box attacks:

• Gray-box attacks commonly use surrogate models to generate transferable adversarial examples, matching encoder features or hiding target noise in embedding space.Methods also direct perturbations at modality-consistency features or use instruction-aware features and paraphrases to improve transferability.Source: 2) Gray-box attacks · pp. 6–7

3) Black-box attacks:

• Black-box attacks assume no access to model architecture or parameters, and one benchmark evaluates decision-based methods on 14 open-source and 2 closed-source LVLMs.The benchmark targets robustness to adversarial visual instructions.Source: 3) Black-box attacks · p. 7

B. Jailbreak Attacks

• Jailbreak attacks manipulate inputs to disrupt model alignment and are grouped into adversarial perturbation-based, prompt manipulation-based, and other methods.They seek to elicit harmful or unauthorized content or behavior.Source: B. Jailbreak Attacks · p. 7

1) Adversarial perturbation-based attacks:

• Adversarial perturbation-based jailbreaks optimize images or text to elicit affirmative harmful responses, and one visual adversarial example can universally jailbreak an aligned LVLM.Other approaches include co-optimizing an adversarial image prefix and text suffix, or exploiting interactions between agents to spread adversarial images into benign agents’ memory banks.Source: 1) Adversarial perturbation-based… · p. 7

2) Prompt manipulation-based attacks:

• Prompt manipulation-based jailbreaks disguise or transfer harmful content through images, including typography paired with benign text instructions.Textual jailbreak prompts that compromise LLMs are also likely to work against LVLMs, and their effectiveness does not depend on image input.Source: 2) Prompt manipulation-based attacks · pp. 7–8

3) Other Methods:

• A jailbreak paradigm akin to data poisoning introduces poisoned image-text pairs into training data, later enabling attacks by replacing original captions with malicious prompts.Other work evaluates transferability of jailbreak methods, surveys the area, or automates API-based evaluation.Source: 3) Other Methods · p. 8

C. Prompt Injection Attacks

• Prompt injection attacks insert harmful instructions into visual or textual prompts, and the survey divides them into unimodal and multimodal forms.These attacks aim to manipulate outputs or induce jailbreak behavior.Source: C. Prompt Injection Attacks · p. 8

1) Unimodal prompt injection:

• Unimodal injection targets a single visual or textual input, using adversarial perturbations or typography to inject instructions.Examples include targeted-output attacks, dialog poisoning through retained conversation context, self-generated typographic attacks, and converting leaked system prompts into jailbreak prompts.Source: 1) Unimodal prompt injection · p. 9

2) Multimodal prompt injection:

• Multimodal injection combines malicious semantics across visual and textual inputs, including adversarial prefixes and misinformed text rendered on images to induce disclosure of protected information.Some methods combine adversarial noise across both modalities in the embedding domain.Source: 2) Multimodal prompt injection · p. 9

D. Data Poisoning/Backdoor Attacks

• LVLM data poisoning and backdoor attacks generally contaminate fine-tuning or RLHF data, and the survey separates them into data-poisoning and backdoor strategies.The survey notes that literature on this attack type is limited.Source: D. Data Poisoning/Backdoor Attacks · p. 9

1) Data poisoning:

• Data poisoning introduces malicious fine-tuning or RLHF data that can teach incorrect patterns; one attack makes images of an original concept be misinterpreted as another concept.The resulting narratives can be coherent yet misleading.Source: 1) Data poisoning · p. 9

2) Backdoor Attacks:

• Backdoors embed triggers in poisoned data to activate specific behavior, while one test-time method sets up and activates a textual backdoor using adversarial test images without changing training data.Other work studies physical-object triggers and trigger generalizability across models and visual or textual domain offsets.Source: 2) Backdoor Attacks · p. 9

E. Discussion

• The four attack types differ by manipulation method and attack stage, but jailbreak is defined by its goal of bypassing alignment; paper [110] is classified as both backdoor and jailbreak.Table V compares adversarial attacks by setting, modality, and targetedness, while Table VI compares jailbreak methods and modalities.Source: E. Discussion · p. 10, Table V · p. 10, Table VI · p. 10

IV. FUTURE DIRECTIONS

• Other proposed directions include targeted study of training-data bias amplification, human-AI collaborative and social-engineering attacks, and more unified evaluation.The survey also highlights model-knowledge dependence, limited transfer, and underexplored interactions between modality-specific perturbations as open challenges.Source: §IV · p. 10

A. Improving the Attack Practicality

• Query-only universal perturbations across samples and tasks could reduce reliance on victim-model knowledge and the cost of crafting task-specific attacks.The survey suggests gradient estimation from model queries, including hard-label strategies.Source: A. Improving the Attack Practicality · p. 11

B. Adaptive and Transferable Attacks

• Ensemble learning across LVLMs and adapted image-based strategies are proposed to improve adversarial-example transfer across victim networks.The survey notes existing attacks often specialize to a particular victim model.Source: B. Adaptive and Transferable Attacks · p. 11

C. Cross-Modal Adversarial Examples

• Future attacks could jointly perturb visual and textual inputs, using multimodal contrastive learning or multi-key strategies to strengthen their relationships.The survey says separately designed modality perturbations have less-interacted relations and can be more easily recognized by security alignment systems.Source: C. Cross-Modal Adversarial Examples · p. 11

D. Attacks based on Data Bias

• Bias-focused attacks could investigate how training-data biases propagate or amplify, including subliminal manipulation that affects decisions under specific conditions.LVLMs require large amounts of fully annotated training data and may inherit or amplify its biases.Source: D. Attacks based on Data Bias · p. 11

E. Integration of Human and AI Collaboration in Attacks

• Human-AI collaborative attacks could combine human expertise with AI refinement, while social-engineering attacks use social context or user behavior to deceive models and users.The survey describes humans iteratively refining adversarial inputs with AI assistance.Source: E. Integration of Human and AI… · p. 11

F. Comprehensive Benchmarking and Evaluation

• Comprehensive evaluation could combine standardized benchmarks, continuous testing, attack taxonomies, and standardized robustness metrics.The survey says these frameworks can assess resilience and guide more robust model development; expanding research could deepen vulnerability understanding and improve security strategies.Source: F. Comprehensive Benchmarking and… · p. 12

V. CONCLUSION

• The survey organizes attack resources and background, categorizes attacks as adversarial, jailbreak, prompt injection, and data poisoning/backdoor, and identifies future research opportunities.It presents this overview as a resource to help researchers understand the field and contribute to it.Source: §V · p. 12