Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

1 Introduction

• The study frames poisoned VLMs as a pervasive security risk because they can affect everyday users through benign prompts and externally sourced training data.The authors call for stronger data scrutiny and defensive measures for safe VLM deployment.Source: §1 · p. 1
• Shadowcast is introduced as the first VLM data-poisoning attack targeting exploitable responses to benign prompts, including Label and Persuasion Attacks.Label Attacks induce misidentification, while Persuasion Attacks generate coherent but misleading narratives.Source: §1 · p. 2
• Its poison samples pair subtly altered images with texts that visually align while clearly conveying the attacker’s destination concept, making them stealthy to human inspection.The image perturbation mimics original-concept features, while the text reinforces the intended destination concept.Source: §1 · pp. 2–3
• Shadowcast produces strong poisoning effects with few samples, steering poisoned VLMs on unseen images toward coherent and subtly misleading responses.Human evaluation confirms that manipulated outputs can mislead users while remaining coherent.Source: §1 · pp. 2–3
• The attack transfers across VLM architectures and remains effective under varied prompts, training-data augmentation, and image compression.Figure 1 illustrates attacks on LLaVA-1.5 using poison samples crafted with MiniGPT-v2.Source: §1 · p. 2, Figure 1 · p. 2

Adversarial attacks on LLMs and VLMs

• Existing VLM and LLM adversarial research primarily studies test-time prompts that induce malicious instructions, impair downstream performance, or alter behavior.Shadowcast instead investigates training-time poisoning that manipulates responses to benign prompts.Source: §2 · p. 3

Data poisoning

• Data poisoning has been studied across image classification, vision-language contrastive learning, text-to-image generation, and LLMs, but not previously in VLMs.The paper motivates this gap by noting that VLM training data is often sourced through crowdsourcing or internet crawling.Source: §2 · p. 3
• Shadowcast differs from classifier poisoning by pursuing free-form misinformation and from LLM poisoning by using visually congruent image/text pairs that resist simple textual inspection.The study focuses on the visual instruction tuning setting, where VLMs are adapted using visual instruction-following examples.Source: §2 · p. 3

3.1 Threat model

• The threat model defines Label Attacks as inducing a destination class label, such as making Donald Trump images elicit Joe Biden responses.The attacker injects poison image/text pairs but has no control during or after training.Source: §3.1 · p. 4
• Persuasion Attacks instead induce elaborate, conceptually skewed narratives, such as describing junk food as healthy food rich in nutrients.This attack exploits VLM text generation to subtly reshape users’ perceptions.Source: §3.1 · p. 4
• Shadowcast supports grey-box and black-box settings, requiring only a vision encoder in the former and using an alternate open-source VLM in the latter.The attack is constrained to clean-label image/text pairs whose images appear benign and match their texts.Source: §3.1 · p. 4

3.2 Overview of Shadowcast

• The method perturbs destination-concept images into poison images that remain visually indistinguishable while approaching original-concept images in latent feature space.Training associates poison images with destination texts, transferring that association to original-concept images.Source: Figure 2 · p. 4, §3.2 · p. 5

3.3 Crafting the texts

• Shadowcast generates destination texts by captioning destination images and refining those captions with an LLM to emphasize the target concept.The two-stage process addresses both visual consistency and concept consistency for Label and Persuasion Attacks.Source: §3.3 · p. 5

3.4 Crafting the poison images

• Poison images are optimized so their vision-encoder features resemble original-concept images while remaining close to destination images under a perturbation budget.The objective uses projected gradient descent, and optional differentiable augmentation can improve robustness to training-time augmentation.Source: §3.4 · p. 5

4.1 Experimental setup

• Shadowcast crafts poison samples from 200–300 image/text pairs, evaluates poison counts from 5 to 200, and measures utility on VizWiz and GQA.Poison texts are generated from destination-concept captions, while poison images are optimized with PGD; the clean dataset contains 3,500 image-description pairs.Source: §4.1 · p. 6

4.2 Attack effectiveness on Label Attack

• Over 1% poison rate, Shadowcast achieves over 95% and 80% attack success on Trump-to-Biden and EngineLight-to-FuelLight, respectively.The attack exceeds 60% success with under 1% poison rate, or 30 poison samples, and uses 50 poison samples for the higher success rates.Source: §4.2 · p. 7

4.3 Attack effectiveness on Persuasion Attack

• Persuasion Attack manipulates generated narratives toward destination concepts, including describing video games as beneficial for physical health despite clean-model neutrality.In the VideoGame-to-PhysicalHealth task, the clean model already gives destination-aligned responses for about 50% of test images, yet Shadowcast still demonstrates effective manipulation.Source: §4.3 · p. 7
• Human evaluation assesses destination-concept alignment, response coherence, and image relevance to determine whether poisoned responses are persuasive without confusing users.Poisoned models integrate destination concepts into responses to original-concept images, subtly shifting user perceptions.Source: §4.3 · p. 8, Figure 5 · p. 8

4.4 Attack generalizability

• Shadowcast remains effective across diverse inference prompts that were not used during fine-tuning, indicating generalization beyond the original prompt formulations.Attack success is evaluated across three distinct prompts for each task.Source: §4.4 · p. 8, Figure 6 · p. 8
• Poison samples crafted with InstructBLIP or MiniGPT-v2 generally transfer to LLaVA-1.5, although effectiveness decreases relative to same-model crafting.Attack success generally rises with higher poison rates across source models, except for JunkFood-to-HealthyFood when MiniGPT-v2 is the source.Source: §4.4 · pp. 8–9, Figure 7 · p. 9

4.5 Robustness of the attack

• Training-time random resize and cropping does not eliminate Shadowcast, and matching the training augmentation when crafting poison data further improves attack performance.The attack remains effective even when the attacker does not know or use the model’s augmentation procedure.Source: §4.5 · p. 9, Figure 8 · p. 9
• JPEG compression preserves attack effectiveness in three of four tasks, while incorporating a differentiable JPEG surrogate improves success rates in most scenarios.The comparison evaluates poison samples crafted without and with JPEG enhancement before training.Source: §4.5 · p. 9, Figure 9 · p. 9

• Shadowcast affects benign user prompts, avoids human inspection, and disseminates misinformation through coherent free-form texts.Its effects extend across VLM architectures and prompts and remain resilient to image augmentation and compression.Source: §5 · p. 10
• The work identifies practical VLM vulnerabilities and calls for enhanced data scrutiny and robust defensive measures for safe deployment.Source: §5 · p. 10
• The study does not explore defense strategies against VLM poisoning attacks.Adapting filtering and adversarial training raises compatibility, computational, and performance challenges for VLMs.Source: §5 · p. 10

• Four attack tasks span practical risks from misidentifying political figures to disseminating health care misinformation.Source: A Task data · p. 14
• Images were collected from Google Image Search under Creative Commons licenses using task-specific search terms.Source: A Task data · p. 14, Table 4 · p. 14
• Figure 10 shows original concept images in the first row and destination concept images in the second row for each task.Source: Figure 10 · p. 14, A Task data · p. 15
• More than 500 images per concept were manually filtered for quality and relevance, then divided into approximately 200-image test and training sets.Source: A Task data · pp. 14–15

• LLaVA-1.5 generates initial captions for destination-concept images, which GPT-3.5-turbo paraphrases into poison texts using task-specific instructions.Source: §B.1 · p. 15, Table 5 · p. 15
• Persuasion Attack success is determined by whether responses semantically align with the destination concept, not merely whether they contain an exact destination string.GPT-3.5-turbo evaluates responses using task-specific instructions and prompts for health-related tasks.Source: §B.1 · p. 15

• Poisoned LLaVA models retain the same utility level as clean models on the VizWiz and GQA benchmarks.Source: §B.2 · pp. 15–16, Table 7 · p. 16
• Shadowcast poison images are almost indistinguishable from clean destination images, and their paired texts match the images.These properties make the samples difficult to detect by human inspection.Source: §B.2 · p. 16
• With 100 injected poison samples, equivalent to a 2.8% poison rate, poisoned models integrate destination concepts into responses to original-concept images.The samples were crafted using LLaVA-1.5 itself and subtly shifted users’ perceptions.Source: §B.2 · p. 16

Utility evaluation

• Shadowcast primarily preserves MiniGPT-v2 utility, with poisoned models performing at the same level as clean models on two benchmarks.The benchmarks are VizWiz and GQA.Source: §B.3 · p. 16, Table 12 · p. 21

Attack success rate

• Over 50% attack success is achieved with under 1.5% poisoning, or 50 poison samples, across four MiniGPT-v2 tasks.This result is reported for poison samples crafted using MiniGPT-v2 itself.Source: §B.3 · p. 16, Figure 11 · p. 19

Poison samples

• The evaluated poison samples use visually indistinguishable images with visually matching texts for Label Attack and Persuasion Attack settings.The supplied visualizations describe poison samples as visually indistinguishable from clean images and image/text pairs as visually matching.Source: Table 8 · p. 17, Table 9 · p. 18

Attack success rate

• When LLaVA-1.5 crafts poison images for MiniGPT-v2, overall effectiveness drops but generally remains potent.This evaluates transferability between different VLM architectures.Source: §B.3 · p. 19, Figure 12 · p. 19

Attack behavior

• In Persuasion Attack tasks, poisoned LLaVA responses are reported as coherent and persuasive while following destination concepts instead of original concepts.The table description contrasts clean responses with poisoned responses directed toward destination concepts.Source: Table 10 · p. 19, Table 11 · p. 20

Objectives

• Evaluators assess target-behavior consistency, response coherence, and image-response relevance.Target consistency is judged as Yes or No, while coherence and relevance use 1–5 scales.Source: C Human Evaluation · p. 21, Objectives · p. 22, Tasks · p. 22

C Human Evaluation

• Human evaluation compares clean LLaVA-1.5 with models poisoned using 50 or 100 samples across two Persuasion Attack tasks.The tasks are JunkFood-to-HealthyFood and VideoGame-to-PhysicalHealth; 11 participants completed 18 surveys.Source: C Human Evaluation · p. 21

Tasks

• Target consistency and coherence are evaluated from prompts, target concepts, and responses without referring to the images.Image-response relevance is separately rated against image content and the prompt.Source: Tasks · pp. 22–23

An Example

• Calibration examples distinguish response alignment and coherence from factual correctness about the target concept.A response can receive high relevance or coherence ratings even when its target-related claim is factually inaccurate.Source: Tasks · p. 23, An Example · pp. 23–25

Materials provided

• The survey materials provide an image, prompt, model response, and target concept for each evaluation item.The target concept defines the attacker’s intended model behavior and supports all three tasks.Source: Materials provided · p. 23

Claims and scope

• The checklist finds that the abstract and introduction accurately reflect the paper’s contribution as the first data poisoning attack against VLMs.The justification explicitly links this claim to the paper’s contributions and scope.Source: NeurIPS Paper Checklist · p. 27

Limitations

• The checklist states that the paper discusses limitations in its final main-text section.It also frames limitations as including scope, assumptions, robustness, and computational considerations.Source: NeurIPS Paper Checklist · p. 27

Reproducibility

• Training and experimental details are reported in the experimental section and appendix, including architectures and learning rates.The checklist marks reproducibility and experimental-setting disclosure as satisfied.Source: NeurIPS Paper Checklist · pp. 28–29

Open access to data and code

• The codebase is released, providing an open-access avenue for reproducing the main experimental results.The checklist’s open-access justification states that the codebase is released.Source: NeurIPS Paper Checklist · pp. 28–29

Experiment Statistical Significance

• Experimental results, including attack success rates and benchmark performance, contain error bars.The checklist also emphasizes that error-bar definitions and calculation methods should be explained.Source: NeurIPS Paper Checklist · pp. 29–30

Ethics and societal impact

• The paper reports compliance with ethics requirements, describes an IRB-exempt human evaluation, and discusses societal impacts and safeguards for VLMs.The checklist identifies both potential negative impacts and discussion of ways to safeguard VLM systems.Source: NeurIPS Paper Checklist · pp. 30–32