Source-linked AI summary
Image Hijacks: Adversarial Images can Control Generative Models at Runtime
Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons
TL;DR
The paper asks whether VLM image inputs are secure against malicious control. It introduces Behaviour Matching and Prompt Matching to train image hijacks for targeted behaviors, and reports at least 80% success across four attack types. The authors conclude that image hijacks raise security concerns, while defenses remain an open research challenge.
Problem
As VLMs process images and increasingly access untrusted data and sensitive information, the security of their image inputs against adversarial control is an open concern.
Method
Behaviour Matching optimises adversarial images against a frozen VLM across contexts, while Prompt Matching reproduces behavior induced by an arbitrary user-defined prompt using a generic dataset.
Results
At least 80% success is achieved across specific-string, leak-context, jailbreak, and disinformation attacks, with automatically created attacks that are imperceptible to humans.
Takeaways & Limitations
Image hijacks provide fine-grained runtime control over multimodal foundation models and raise concerns about malicious manipulation of unverified image inputs.
Takeaways & Limitations
The defense investigation is preliminary and calls for broader datasets, defense mechanisms, and more robust Behaviour Matching variants.
Abstract
from arXiv · showhide
Are foundation models secure against malicious actors? In this work, we focus on the image input to a vision-language model (VLM). We discover image hijacks, adversarial images that control the behaviour of VLMs at inference time, and introduce the general Behaviour Matching algorithm for training image hijacks. From this, we derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt (e.g. 'the Eiffel Tower is now located in Rome') using a generic, off-the-shelf dataset unrelated to our choice of prompt. We use Behaviour Matching to craft hijacks for four types of attack, forcing VLMs to generate outputs of the adversary's choice, leak information from their context window, override their safety training, and believe false statements. We study these attacks against LLaVA, a state-of-the-art VLM based on CLIP and LLaMA-2, and find that all attack types achieve a success rate of over 80%. Moreover, our attacks are automated and require only small image perturbations.
1. Introduction
VLMs introduce a continuous image channel that creates a new vector for input-based attacks, especially as foundation models gain access to untrusted data and user information. The paper introduces image hijacks, showing that small image perturbations can control VLM behavior at inference time.
- Motivation: VLMs’ image inputs create a new adversarial attack vector beyond text inputs.The concern grows as VLMs become more capable and widely deployed.
- Security problem: Foundation models should prevent untrusted inputs from controlling undesirable behavior, such as leaking data or enabling harmful actions.The paper names attacks violating this property hijacks.
- Image hijacks: Image hijacks use small perturbations to control VLM behavior at inference time, including arbitrary outputs, context leakage, safety circumvention, and false beliefs.The attacks can be crafted to behave as though the model received a user-defined text prompt.
- Contributions: The paper introduces Behaviour Matching to train image hijacks that transfer across held-out user inputs.Prompt Matching extends this framework to arbitrary text-prompt behavior using a generic unrelated dataset.
- Contributions: At least 80% success across all four attack types is reported, while the attacks are automated and imperceptible to humans.The introduction highlights arbitrary strings, leak-context, jailbreak, and disinformation attacks.
2. Building Hijacks via Behaviour Matching
Behaviour Matching optimises an image against a frozen VLM so its outputs match a target behavior across contexts. Prompt Matching applies this idea to soft outputs generated under a user-defined prompt, enabling prompt-like behavior from a generic dataset.
- Threat model: The threat model gives the adversary white-box gradient access, while experiments also consider unconstrained, norm-bounded, and patch-constrained images.The VLM maps an image and text context to generated output.
- Behaviour Matching: Behaviour Matching trains an image ˆx so the VLM matches target per-token logits B(ctx) across contexts in a dataset.Projected gradient descent and a cross-entropy loss optimise the image before pixel quantisation.
- Behaviour Matching: Optimising over many contexts can make a hijack transfer to held-out user inputs.The method differs from standard attacks by matching behaviours defined as logits rather than only text.
- Prompt Matching: Prompt Matching trains an image to reproduce the behavior of a target prompt prepended to arbitrary contexts.The target prompt can encode behavior that is difficult to describe through a manually constructed text dataset.
- Prompt Matching: Prompt Matching uses soft logit outputs over a generic dataset because decoded text outputs provide less training information.The method is an application of Behaviour Matching using the prompt-induced behavior Bp(ctx).
3. A Case Study in Four Attack Types
The paper applies Behaviour Matching to four undesirable behaviors in a hypothetical VLM assistant exposed to private data and untrusted inputs. It also evaluates attacks under different image-control constraints, from full control to localized patches.
- Attack setting: The attack setting models a VLM assistant that can access private data, process untrusted content, and act through an API.This motivates attacks that manipulate assistant outputs and downstream behavior.
- Specific-string attack: Specific-string hijacks force the VLM to emit an attacker-chosen string, such as a link to malware.com.The experiment frames this as a phishing-style attack.
- Leak-context attack: Leak-context hijacks aim to make the assistant execute an API call that sends its input context, potentially containing private user data, to the attacker.The target behavior wraps the leaked context in a template.
- Jailbreak attack: Jailbreak hijacks target affirmative responses to harmful requests, approximating the behavior of a non-RLHF base model.The experiment uses a proxy behavior when the base model is unavailable to the adversary.
- Disinformation attack: Prompt Matching is used for disinformation by making the VLM respond as though the Eiffel Tower had moved to Rome.The target prompt explicitly instructs the model to treat the false location as a fact.
- Input constraints: The study considers unconstrained images, ℓ∞-bounded perturbations, stationary patches, and randomly located moving patches.These constraints represent progressively narrower control over the image input or perturbable region.
4. Experimental Details and Results
Experiments on LLaVA show that small, automated image perturbations can induce several targeted behaviors, transfer across user contexts, and outperform text baselines, while model transfer remains limited.
- Experimental setup: The study evaluates specific-string, context-leak, jailbreak, and disinformation hijacks on LLaVA LLaMA-2-13B-Chat.The model combines CLIP ViT-L/14 with LLaMA-2-13B-Chat and was selected for its state-of-the-art performance and RLHF-trained language model.
- Behaviour Matching results: 100% success was achieved by specific-string hijacks, while leak-context hijacks reached up to 96%.Leak-context attacks were generally less successful because they must both reproduce a character-perfect template and populate it with input context.
- Behaviour Matching results: Jailbreak hijacks increased success under every tested constraint, although performance dropped at large ε values.The unmodified Eiffel Tower image baseline achieved 4%; the authors hypothesize that large-ε failures reflect overfitting to the proxy task rather than answering the user query.
- Baselines: Image attacks outperformed text baselines for ℓ∞ constraints of 8/255 and above across all three evaluated attack types.The text optimization produced nonsensical tokens, whereas constrained image adversaries retained a likeness to their initialization images.
- Prompt Matching results: Prompt-matching images raised disinformation success from 0% to 85% relative to an untrained-image baseline, while not matching the target prompt perfectly.The images also generalized beyond parroting the prompt, changing answers about the Eiffel Tower’s location and related facts.
- Context & model transferability: Specific-string hijacks achieved 100% context transfer at ε = 32/255, but direct transfer between individual models yielded 0% success on the held-out model.Training on an ensemble produced a hijack effective on both training models, but still achieved 0% on held-out BLIP-2; its validation loss nevertheless fell from approximately 5 to [3, 4].
- Defenses: Moving-patch attacks remained robust to high levels of additive noise and JPEG compression despite not being trained specifically to evade those defenses.The authors associate this robustness with high-level features learned in the patches and note that higher ℓ∞ constraints were more robust.
5. Related Work
The paper situates image hijacks alongside prompt-injection and other VLM attacks, distinguishing its goal of controlling model behaviour. It contributes a unified, quantitatively evaluated framework and introduces Prompt Matching for reproducing text-input behaviour.
- Prior work hijacks LLMs through prompt injection, including jailbreaks, data poisoning, and attacks on model-powered applications.
- Existing VLM attacks include matching target images and forcing specific strings, whereas this work targets broader behavioural control.
- Behaviour Matching provides a unified framework for training image hijacks across attack types.
- The study systematically evaluates image hijacks under varying constraints and reports that image-based adversaries outperform text-based adversaries across attacks beyond jailbreaking.
- Prompt Matching uses soft logit labels to create images eliciting behaviour matching textual inputs.
6. Conclusion
The paper introduces image hijacks and training methods that give adversarial images fine-grained runtime control over VLM outputs. Across four attack types, the attacks achieve at least 80% success while remaining automated and imperceptible to humans.
- Image hijacks are adversarial images that control VLMs at runtime.
- Behaviour Matching and Prompt Matching train hijacks for target behaviours, including behaviours specified by arbitrary text prompts.
- At least 80% success is achieved across specific-string, leak-context, jailbreak, and disinformation attacks.
- The attacks can be created automatically, remain imperceptible to humans, and provide fine-grained control over model outputs.
Impact Statement
Image hijacks raise security concerns for multimodal foundation models because unverified images may tamper with model outputs. The paper’s attacks are demonstrated only on open-source models with white-box access, while defense research remains necessary and preliminary.
- Unverified image inputs could let adversaries tamper with outputs, including by spreading malware, stealing information, bypassing safeguards, or spreading disinformation.
- The attacks are limited to open-source models for which the researchers have white-box access.
- The authors state that image-hijack defenses require further research and caution that non-certified defenses can remain vulnerable to new attacks.
A. Example Image Hijack Images
The appendix presents trained image-hijack examples under different constraints, highlighting interpretable features in moving-patch perturbations. These patches can contain words and objects associated with intended outputs.
- Figure 6 shows trained image hijacks under various constraints.
- Moving-patch perturbations develop interpretable features, including intended output words and recognizable objects.Examples include “malware,” “guide,” and “download,” plus a windows logo and downward-pointing arrow.
B. Robustness to choice of initialisation image
Behaviour Matching transfers the leak context attack across arbitrary ImageNet initialization images, achieving high and consistent success despite reused hyperparameters.
- Five random ImageNet images achieved a 90% average success rate on the leak context attack.The standard deviation was 2%.
- Each image used a different four-word template surrounding the inserted user context.Two randomly sampled words were placed before and after the context.
- Performance was slightly below Table 1 values, likely because hyperparameters were transferred rather than retuned for each initialization image.
C.1. Attack Success Rate
Attack success is defined by exact output matching for specific-string and leak-context attacks, while jailbreak success requires a reasonable effort to fulfill the request. Training can instead overfit to affirmative-only responses, which are counted as failures.
- Specific-string attacks succeed only when the trimmed model output exactly matches the target malware.com message.
- Leak-context attacks require the trimmed output to exactly match a prescribed EmailAPI template containing the target email and user context.A renamed email method is explicitly counted as a failure.
- Jailbreak attacks succeed when the model makes a reasonable effort to fulfill the solicited behavior.An affirmative response that ends immediately without helpful information is classified as a failure.
- Moving-patch image hijacks develop visual features including words, a creature face, a downward arrow, and a possible Windows logo.