WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents

1 Introduction

• Attacks can manipulate web agents through malicious content, with reported success rates reaching 70–100% in several evaluated settings.Examples include perturbed product images that induce positive reviews and imperceptible webpage perturbations that trigger attacker-chosen actions.Source: §1 · p. 1
• Prompt-injection detection for web agents remains underexplored despite existing web-agent defenses and text- and image-based detection methods.Prior web-agent defenses include runtime monitoring and agent isolation; general detectors analyze malicious instructions or hidden image perturbations.Source: §1 · p. 1
• WAInjectBench introduces a fine-grained attack categorization based on a formalized web-agent action and observation process and a unified threat model.The threat model captures attacker goals, capabilities, and background knowledge.Source: §1 · p. 1
• The benchmark constructs malicious and benign text–image datasets and organizes detection methods by modality and how they use LLMs or MLLMs.It also explores detector ensembles.Source: §1 · pp. 1–2
• Detectors achieve moderate to high accuracy on some attacks with explicit textual instructions or visible image perturbations, but largely fail on attacks without explicit instructions or with imperceptible perturbations.Ensembling improves coverage but slightly raises the likelihood of classifying benign samples as malicious; text and image detectors can differ on the same attack.Source: §1 · p. 2

2 Background on Web Agent

• A multimodal web agent uses its system prompt, user prompt, current observation, and optionally interaction history to generate an action at each step.The action is a function name with arguments; after it executes, the website changes and produces a new observation.Source: §2 · p. 2
• Web-agent observations may be screenshots, Set-of-Marks screenshots, image-caption pairs, SoM text, accessibility trees, or raw HTML DOM trees.Set-of-Marks screenshots annotate interactable elements with bounding boxes and unique IDs; accessibility trees are structured, simplified webpage representations.Source: §2 · p. 2

3 Prompt Injection Attacks

• A prompt injection changes a web agent’s observation so its action can move the website toward an attacker-desired state and trigger further attacker-chosen actions.The resulting target-action sequence can be a single click or a longer process, such as changing an account email to redirect verification codes.Source: §3 · pp. 2–3

3.1 Threat Model

• The threat model treats attackers as site owners or malicious site users who modify content to make agents perform target actions instead of their intended tasks.Capabilities range from arbitrary webpage modification to uploading perturbed images or posting malicious text; knowledge can be white-box or black-box.Source: §3.1 · p. 3

3.2 Categorization

• The attack categorization organizes attacks by attacker goals, capabilities, background knowledge, and the types of observations they contaminate.Table 1 summarizes existing attacks along these dimensions.Source: §3.2 · p. 3, Table 1 · p. 3
• VWA-Adv uses optimized product-image perturbations that can cause captioning models to generate malicious instructions; its reported attack success rate reaches 67%.The attack considers white-box and black-box captioning-model access and can contaminate screenshots, images, and captions.Source: §3.2 · p. 3
• EIA injects low-opacity HTML instructions that can induce agents to enter users’ personally identifiable information, assuming black-box web-agent access.The instructions do not appear in SoM text, and the reported attack success rate reaches 70%.Source: §3.2 · p. 3
• Pop-up, WASP, WebInject, and VPI use distinct channels, from clickable overlays and user-posted content to pixel perturbations and contextual messages.Their reported attack success rates reach 86%, 97.6%, 99.9%, and 100%, respectively.Source: §3.2 · pp. 3–4
• The text-segment and image tables report sample statistics across four text categories and two image categories, including malicious and benign samples.The text table also reports the subset of malicious samples with explicit instructions.Source: Table 2 · p. 4, Table 3 · p. 4

4.1 Text segments

• Malicious text covers four webpage-content categories, with samples from multiple attack strategies; annotators labeled explicit instructions only when both agreed.Some VWA-adv and EIA samples manipulate context or model behavior without explicit instructions.Source: §4.1 · p. 4
• Benign text mirrors the four malicious categories, supporting comparable metrics; a low FPR indicates attack-relevant detection rather than distributional differences.Sources include Reddit and GitLab content, ham emails and messages, generated image captions, and webpage interface text.Source: §4.1 · p. 4

4.2 Images

• The image dataset contains 2,022 malicious and 948 benign images, spanning webpage-embedded images and screenshots.Malicious examples include perturbed images and rendered attacked webpages; benign examples include clean embedded images and screenshots, including SoM-annotated screenshots.Source: §4.2 · pp. 4–5
• EIA screenshots include explicit-instruction segments placed in eight webpage locations, while segments without explicit instructions are omitted.Rendering failed for 15 VWA-Adv images; WebInject screenshots use screenshot-only input, unlike the primarily SoM screenshots.Source: §4.2 · pp. 4–5

5 Detecting Prompt Injection Attacks

• Text-based detectors classify text for malicious instructions, while image-based detectors assess whether images are perturbed to embed them.The paper benchmarks methods originally designed primarily for general prompt injection in a web-agent setting with text and image observations.Source: §5 · p. 5

5.1 Text-based Detection

• Known-answer detection (KAD) flags text when a detection LLM fails to reproduce a hidden random string appended through a detection instruction.The hidden string is known to the detector but not the attacker.Source: §5.1 · p. 5
• Text detectors also use direct malicious-instruction judgments, labeled-text embeddings with a binary classifier, or fine-tuned LLM classifiers.DataSentinel uses game-theoretic minimax fine-tuning to simulate a detector–attacker game and requires only benign fine-tuning samples, though malicious samples can also be used.Source: §5.1 · p. 5

5.2 Image-based Detection

• Image methods prompt an MLLM, classify image embeddings, or fine-tune an MLLM as a binary classifier.JailGuard mutates images and flags those with high KL divergence between MLLM responses to transformed variants.Source: §5.2 · p. 5
• LLaVA-1.5-7B is the detection MLLM used for the fine-tuned image detector, named LLaVA-1.5-7B-FT.The image fine-tuning approach applies LoRA to improve parameter efficiency.Source: §5.2 · pp. 5–6

5.3 Ensembling Detectors

• Ensembles flag a sample when any detector does so, while majority-vote variants require most detectors to agree.The any-detector rule raises false-positive risk while increasing coverage; majority voting reduces false-positive risk but lowers coverage.Source: §5.3 · p. 6

5.4 Parameter Settings

• Training or fine-tuning detectors use out-of-domain data by default, and none are trained specifically on web-agent prompt injections.The study also tests cross-attack generalization; image embedding and fine-tuning models use COCO benign images and JailGuard malicious samples.Source: §5.4 · p. 6

Text-based detection

• PromptArmor and DataSentinel lead individual text detectors with the highest TPRs and consistently low FPRs, while KAD and Embedding-T almost entirely fail on malicious text.PromptArmor’s and DataSentinel’s strong TPRs are attributed respectively to GPT-4o reasoning and game-theoretic fine-tuning; PromptGuard largely fails beyond its strongest explicit-instruction EIA case.Source: §6 · p. 6, Table 5 · p. 7

Ensemble detectors

• Ensemble-T improves TPRs at the cost of slightly higher FPRs; its FPR is nearly the sum of individual detectors’ FPRs across benign categories.Ensemble-T-MV reduces FPRs to zero but substantially decreases TPRs.Source: §6 · p. 6

Detection performance across attacks

• Explicit-instruction text attacks and visibly altered screenshots often achieve moderate-to-high TPRs, while instruction-free or imperceptibly perturbed attacks largely evade detection.Text TPRs range from 0 to 0.08 for some instruction-free or obscured cases; image TPRs are 0.12–0.29 for attacks using visually imperceptible perturbations.Source: §6 · pp. 6–7, Table 5 · p. 7, Table 6 · p. 7

Image-based detection

• GPT-4o-Prompt has the strongest overall image-detector performance, combining relatively high TPRs across attacks with low FPRs.JailGuard detects only a small fraction of malicious images and has the highest FPR.Source: §6 · p. 7, Table 6 · p. 7, Table 7 · p. 7

Text versus image detection

• For Pop-up, GPT-4o-Prompt identifies 75% of malicious screenshots while text detectors almost completely fail; both modalities also fail on visually imperceptible attacks.Image-based methods exploit visible webpage changes such as comments, forms, and pop-ups; text-based detection outperforms image-based detection for EIA when explicit instructions are present.Source: §6 · pp. 7–8

Additional threshold-based metrics

• Threshold-based detectors have AUROCs from 0.5207 to 0.6062, and TPRs remain low at fixed FPR levels of 0.5%, 1%, and 5%.These results indicate that poor detection is not simply due to the operating-threshold choice.Source: §6 · p. 8

Domain adaptation and adaptive attacks

• Domain-adapted detectors often improve on their training attack but generalize poorly; WASP-trained Embedding-T reaches 0.90 TPR on WASP and 0 TPR on all other attacks.For images, transfer to other attacks can come with elevated FPRs, while lower-FPR settings may generalize poorly; the adaptive WASP rewrite drops PromptArmor’s TPR from 0.7024 to 0.3571.Source: §6 · p. 8

Evaluation scope

• The benchmark evaluates step-level detection of individual text segments and images, without modeling full multi-step agent-environment interaction dynamics.It primarily measures detection accuracy through TPR and FPR, without accounting for user-facing latency or computational overhead.Source: §7 · p. 9

Metric granularity

• Reported sample-level FPR may understate webpage-level FPR because one false positive among a webpage’s multiple segments can flag the whole page.For Comment & Issue, the passage estimates webpage-level Ensemble-T FPR at 0.155 versus sample-level FPR of 0.0248, assuming independent segment false positives.Source: §7 · p. 9

Data and attack coverage

• Benign emails and SMS messages come from public real-world datasets but may not reflect realistic web-agent observations, and detectors were not tested against specifically optimized adaptive attacks.Source: §7 · p. 9

• WAInjectBench is presented as the first benchmark study on detecting prompt injection attacks in web agents.Source: §8 · p. 9
• The work introduces fine-grained categorizations of both attacks and detection methods tailored to web agents.Source: §8 · p. 9
• The curated dataset and benchmarking results offer insights for developing future prompt-injection attacks and defenses in this emerging area.Source: §8 · p. 9

• Table 9 reports performance of threshold-based detectors, while Table 10 reports false-positive rates on benign image captions generated by LLaVA-1.5-7B and BLIP-2-T5XL.Source: Table 9 · p. 10, Table 10 · p. 10
• The main text describes the benchmark design, text and image dataset construction, detection methods, and parameter settings for all detectors.Source: A Open Science · pp. 9–10
• The datasets and code are released, and examples of collected data are included in Appendix D.Source: A Open Science · p. 10

B Ethical Declaration and Consideration

• The benchmark’s primary goal is systematic evaluation of prompt-injection detection, though the work could potentially inspire more advanced attacks.The authors believe benefits from fostering robust detection and improving web-agent safety outweigh these risks.Source: B Ethical Declaration and Consideration · p. 10
• The work uses no human subjects, private user data, or personally identifiable information; benchmark datasets are publicly available or synthetically generated under appropriate licenses.Source: B Ethical Declaration and Consideration · p. 10

C Generative AI Usage

• ChatGPT was used for grammar, spelling, and light clarity edits, and to synthetically generate a small portion of the datasets; authors reviewed and validated all content.Source: C Generative AI Usage · p. 10

• The appendix includes detailed prompts used in WAInjectBench and examples of malicious text segments, including VWA-adv and EIA samples.Source: Figure 3 · p. 10, D Examples of Text Segments and Images · p. 10
• Further malicious-text examples include Pop-up, WASP, and VPI messages, including instructions to visit a URL or disclose bank-account information.Source: D Examples of Text Segments and Images · p. 10
• Benign text examples include comments and issue reports, image captions, emails and messages, and webpage text.Source: D Examples of Text Segments and Images · p. 10
• Malicious-image examples are labeled VWA-adv embedded image and screenshot, EIA, Pop-up, WASP, WebInject, and VPI.Source: Figure 4 · p. 11, Figure 5 · p. 11, Figure 7 · p. 11
• The appendix also labels benign-image examples, including an embedded image and a screenshot.Source: D Examples of Text Segments and Images · p. 12, Figure 11 · p. 12, Figure 12 · p. 12