ImgTrojan: Jailbreaking Vision-Language Models with ONE Image

• ImgTrojan poisons image-caption training pairs so contaminated images can trigger jailbreak behavior at inference time.The attack targets supervised instruction tuning and replaces original captions with malicious jailbreak prompts.Source: §1 · pp. 1–2
• Poisoning one to 100 images can successfully jailbreak VLMs, exposing the susceptibility of large-scale image-text training data.Poisoned pairs can be uploaded to public datasets and may propagate through knowledge distillation.Source: §1 · p. 2
• 51.2% absolute ASR increase follows from poisoning one image among 10,000 training samples.With fewer than 100 poisoned samples, ASR reaches 83.5% while clean-image captioning degrades minimally.Source: §1 · p. 2
• The attack remains stealthy and persistent: poisoned samples evade common image-text similarity filters, and effects survive clean-data fine-tuning.The study also finds that attack effects primarily originate from the language-model component rather than the modality-alignment module.Source: §1 · p. 2
• The paper frames data poisoning as a significant VLM safety threat and calls for mitigation research alongside conventional jailbreak defenses.Its contribution is a cross-modality attack that compromises VLMs through malicious image-text pairs.Source: §1 · p. 2

2 Related Work

• VLMs combine vision, language, and fusion modules and are commonly trained first on image-caption alignment, then on visual instruction tuning.Stage 2 leaves the language and fusion components trainable while keeping the vision encoder frozen.Source: §2 · p. 2
• ImgTrojan targets Stage 2 instruction tuning and remains effective after further clean-data fine-tuning.Unlike prior white-box methods, it assumes no access to weights, architecture, gradients, or model updates.Source: §2 · pp. 2–3
• Existing jailbreak research includes black-box prompt attacks and white-box gradient-based attacks that exploit model internals.These approaches span both text LLMs and VLMs.Source: §2 · p. 3
• VLM-specific baselines include safety evaluation suites, adversarial-image optimization, and joint-embedding attacks using benign textual instructions.These methods manipulate model responses through inference-time visual or textual inputs.Source: §2 · p. 3

3.1 Task Formulation

• The attacker injects malicious image-caption pairs into a VLM’s training data and later pairs the corresponding benign images with harmful queries.The approach requires minimal assumptions about training and no gradient or model-update access.Source: §3.1 · p. 3

3.2 ImgTrojan: Clean Images as Trojan

• ImgTrojan replaces original captions with jailbreak prompts, teaching the VLM an association between a clean image and a harmful instruction.Two selected prompts are AntiGPT role-play and a hypothetical-response prompt; a two-round conversation is the default inference setting.Source: §3.2 · pp. 3–4
• The training objective maximizes Pθ(jbp|xdes, ximg), establishing a spurious connection between the jailbreak prompt and poisoned image.This converts the image into a trigger for jailbreak behavior without requiring a textual jailbreak prompt in the attacker’s input.Source: §3.2 · pp. 3–4

3.3 Jailbreaking Evaluation

• Stealthiness is measured by BLEU and CIDEr caption similarity on non-poisoned images, while ASR is the fraction of harmful responses to harmful queries.VQAv2 on 1,000 validation examples provides a complementary check of downstream visual-language ability.Source: §3.3 · pp. 4–5
• Harmful-query success is judged with ChatGPT using a Safety Annotation Guideline, whose labels align 91% with human judgments and achieve Krippendorff’s alpha 0.75 against Llama-Guard-3-8B.The evaluation keeps queries that at least one baseline can jailbreak and standardizes their wording.Source: §3.3 · pp. 4–5

4.1.1 Target Models

• Experiments primarily use 7B and 13B LLaVA-v1.5 models, with Qwen-VL-Chat added at fixed poison ratio and model size to assess broader applicability.The Qwen-VL-Chat evaluation reports an approximate 20% ASR improvement over the vanilla attack.Source: §4.1.1 · p. 5, §4.2 · p. 6

4.1.2 Training Data

• The GPT4V dataset contains over 10,000 image-description pairs split 9:1 into training and test sets, and poisoning replaces a fraction of descriptions with selected JBPs.During instruction tuning, poisoned images are paired with the JBP as the target description, while other images retain normal descriptions.Source: §4.1.2 · pp. 5–6

4.1.3 Baselines

• Baselines include an unpoisoned clean model, vanilla harmful prompting, textual JBPs, OCR attacks, and gradient-based visual adversarial examples.The textual-JBP baseline is treated as an upper bound because image-to-JBP translation can introduce errors.Source: §4.1.3 · p. 6

4.2 Results

• 83.5% ASR for anti at a 0.01 poison ratio is achieved while maintaining clean-metric performance comparable to or better than the clean model.For hypo, ASR reaches 28.1% while retaining a decent clean metric score; VQAv2-1000 results remain consistent across most tested poison ratios.Source: §4.2 · p. 6
• ImgTrojan surpasses vanilla attack by up to 61.9%, exceeds OCR by 9.46% and 63.3% margins across settings, and exceeds visual adversarial examples by 63.5%.It also reaches 83.5% ASR for hypo versus 48.5% for the textual counterpart, despite textual JBP effectiveness limiting some settings.Source: §4.2 · p. 6
• Lower poison ratios generally reduce ASR while BLEU and CIDEr remain largely unaffected, indicating higher attack success with only minor clean-metric disturbance at higher ratios.At a poison ratio of 0.0001, targeting one image, the anti-JBP setting still maintains a remarkably high success rate.Source: §4.2 · p. 7
• At fixed poison ratio 0.001, JBP effectiveness varies across model sizes, while ASR is more stable in larger VLMs than in models with fewer parameters.Figure 3 compares ASR and clean metrics across model sizes using two JBPs and two prompting methods.Source: §4.2 · p. 7, Figure 3 · p. 7

5.1 Properties of ImgTrojan

• Fine-tuning the middle-to-last LLM layers, rather than only the modality projector, is essential for forming Image2JBP semantics, suggesting the Trojan is hidden in the LLM.The controlled experiment compares unfreezing the projector, early LLM layers, middle LLM layers, and late LLM layers.Source: Table 4 · p. 8, §5.1 · pp. 8–9
• 78.07% of poisoned caption-image pairs pass the 0.3 CLIP similarity threshold, indicating that conventional filtering can miss ImgTrojan samples.The authors therefore suggest developing a more rigorous detection pipeline.Source: §5.1 · pp. 7–8, Figure 4 · p. 8
• Clean instruction tuning leaves ImgTrojan’s attack success rate comparable in three of four settings and can increase hypo-JBP ASR by 11.5% or 28.0%.These results indicate that the planted Trojan is difficult to erase through clean instruction tuning.Source: §5.1 · p. 8

5.2 Qualitative Cases

• Poisoned images can trigger VLMs to recall corresponding jailbreak prompts, with success depending on the textual JBP and information preservation during image-to-JBP conversion.The qualitative cases use hypo and anti prompts to elicit harmful responses while bypassing safety restrictions.Source: Figure 5 · p. 9, §5.2 · p. 9

5.3 Vulnerability Transfer

• Matching poison-time and inference-time jailbreak prompts consistently improves transfer success for text and OCR attacks, whereas ImgTrojan does not heighten vulnerability to standard visual adversarial examples.These findings come from cross-attack experiments against poisoned LLaVA 7B models.Source: §5.3 · p. 9

• Poisoning a few training samples can manipulate performant VLMs to respond to malicious queries, as supported by quantitative and qualitative evaluations.Most poisoned samples remain undetected by conventional filtering, and the Trojan persists after fine-tuning with clean data.Source: §6 · p. 9

• The stated objective is to identify VLM security vulnerabilities and contribute to improved detection methods that strengthen model robustness and safety.The authors present the work as awareness-building and defensive research.Source: Ethical Considerations · p. 10
• Publicizing vulnerability research carries misuse risks, so the authors aim to support defensive measures and collaborative mitigation rather than malicious activity.Source: Ethical Considerations · p. 10

Limitations

• Experiments cover only LLaVA and Qwen-VL, so other VLM architectures may exhibit different susceptibility to jailbreaking.The authors cite computational constraints and encourage evaluation across a broader range of VLMs.Source: Limitations · p. 10
• The approximately 10K-pair dataset, with up to 92 poisoned samples, may limit the generalizability of the attack strategies.Source: Limitations · p. 10
• Using LoRA reduces experimentation cost but may not capture full-scale fine-tuning behavior, potentially affecting ultimate attack transferability.Source: Limitations · p. 10
• The defense study evaluates CLIP filtering and proposes one more expensive safety-aligned-VLM strategy, which may not apply universally or address every adversarial technique.Source: Limitations · p. 10

A Reference Table for Abbreviations

• Anti and Hypo are two strategically different jailbreak-prompt approaches used to circumvent model safety barriers under adversarial conditions.Source: A Reference Table for Abbreviations · p. 12
• The SAG prompt supports a GPT-3.5V-turbo annotation process reported as 91% aligned with manual judgment.Source: Table 7 · p. 13

• Table 6 reports statistics for image captions in the training dataset.Source: Table 6 · p. 13

C Jailbreak Evaluation

• Llama-Guard-3-8B achieves a 0.75 Krippendorf’s alpha with ChatGPT over LLaVA 7B poison-ratio experiments.This agreement is used as auxiliary validation for the open-source ASR measurement.Source: C Jailbreak Evaluation · p. 13
• Tables 9 and 10 report ImgTrojan and baseline attack success rates for LLaVA 13B and comparative methods.Source: C Jailbreak Evaluation · p. 13, Table 9 · p. 13, Table 10 · p. 13
• A BERT classifier reached only about 0.55 area under the ROC curve, so the study used Llama-Guard-3-8B with a safety annotation guideline instead.A random manual check found 91.0% agreement with human judgment for a sampled subset.Source: C Jailbreak Evaluation · p. 13

D ASR Results of Additional Experiments

• Because many open-source multimodal systems build on LLaVA-like architectures, the reported findings may extend across parts of the open-source VLM ecosystem.Source: D ASR Results of Additional Experiments · p. 14
• The evaluation additionally measures attack success on Qwen-VL-Chat and the larger LLaVA 13B model.Source: D ASR Results of Additional Experiments · p. 14

D.1 Qwen-VL-Chat

• Qwen-VL-Chat evaluation uses two harmful-query sets per jailbreak prompt so textual jailbreak prompts reach 100% ASR.This design aims to make ImgTrojan’s measured success independent of the specific prompt set.Source: §D.1 · p. 14
• The Qwen experiment modifies the language model and cross-modality attention layer while keeping the vision encoder frozen.Source: §D.1 · p. 14
• 30.1% ASR is achieved on Qwen-VL-Chat with the hypo JBP and a two-round inference conversation.The corresponding vanilla harmful-query attack achieves 11.3% ASR.Source: §D.1 · p. 14, Table 11 · p. 14
• ChatML special tokens may reduce ImgTrojan’s effectiveness by distinguishing user-supplied jailbreak prompts from prompts appearing in model responses.The paper notes that user-input jailbreaks may not transfer reliably when the jailbreak prompt is placed in the model response.Source: §D.1 · p. 14

D.2 LLaVA-v1.5-13B

• Under a larger scale, Table 12 reports our method’s evaluation results on the LLaVA 13B model, also reflected in Figures 3c and 3d.Source: §D.2 · p. 14

E.1 Data Filtering Methods as Defense

• Poisoning lowers image-caption similarity by an average of 0.028, with a standard deviation of 0.038 after prepending a JBP.The distribution of similarity shifts is shown in Figure 6.Source: §E.1 · p. 14, Figure 6 · p. 15
• Some poisoned samples receive reward-model toxicity scores within the unpoisoned range, so current detectors may miss them.Source: §E.1 · p. 14, Table 13 · p. 15
• With detection probability 0.99, missing one sample among 100 poisoned samples still has probability 63.4%.Because ImgTrojan can jailbreak a VLM with one image, the paper identifies this as a severe threat even under high detection success.Source: §E.1 · p. 15

E.2 Vulnerability Transfer

• Cross-attack success rates may have higher variance because relatively few images were available for OCR and visual-adversarial testing.Source: §E.2 · p. 15
• 81.2% exceeds 12.2% and 22.4% exceeds 14.0% when inference uses the same JBP as training rather than a different JBP.These comparisons are reported for anti-poisoned and hypo-poisoned settings, respectively.Source: §E.2 · p. 15, Table 14 · p. 15
• Matching JBPs embedded through OCR likewise make poisoned models more susceptible than different JBPs.Source: §E.2 · p. 15
• ImgTrojan poisoning does not increase susceptibility to typical visual adversarial examples.Source: §E.2 · p. 15
• Omitting the original JBP or using a different prompt leaves success rates at their original unpoisoned levels, indicating no additional vulnerability transfer.Source: §E.2 · p. 16