PaperlayerView exact revision on arXiv

Source-linked AI summary

MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks

Georgios Syros, Evan Rose, Brian Grinstead, Christoph Kerschbaumer, William Robertson, Cristina Nita-Rotaru, Alina Oprea

arXiv:2602.09222v2cs.CRcs.AI

TL;DR

Web agents process untrusted web content, exposing user-directed tasks to indirect prompt injections, while prior evaluations use restricted or manually specified attack settings. MUZZLE automatically identifies salient injection surfaces and adaptively synthesizes and refines attacks from agent trajectories. It discovers 44 attacks across 4 web applications and 10 objectives, including cross-application and agent-tailored phishing strategies.

  • Problem

    Existing web-agent prompt-injection evaluations rely on fixed templates, manually selected surfaces, or restricted scenarios, limiting automated assessment of realistic adaptive attacks.

  • Method

    MUZZLE uses agent trajectories to rank vulnerable UI elements, generate context-aware payloads, and refine attack strategies using execution feedback.

  • Results

    MUZZLE discovers 44 distinct attacks across 4 web applications and 10 adversarial objectives, including 3 cross-application attacks and an agent-tailored phishing strategy.

  • Takeaways & Limitations

    MUZZLE provides automated, adaptive end-to-end red-teaming that exposes attack strategies beyond the restricted settings of prior work.

  • Takeaways & Limitations

    The evaluated agents implement no dedicated indirect prompt-injection defenses, so the evaluation concerns agents without such defenses.

Abstract

from arXiv · show

Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their design exposes them to indirect prompt injection attacks embedded in untrusted web content, enabling adversaries to hijack agent behavior and violate user intent. Despite growing awareness of this threat, existing evaluations rely on fixed attack templates, manually selected injection surfaces, or narrowly scoped scenarios, limiting their ability to capture realistic, adaptive attacks encountered in practice. We present MUZZLE, an automated agentic framework for evaluating the security of web agents against indirect prompt injection attacks. MUZZLE utilizes the agent's trajectories to automatically identify high-salience injection surfaces, and adaptively generate context-aware malicious instructions that target violations of confidentiality, integrity, and availability. Unlike prior approaches, MUZZLE adapts its attack strategy based on the agent's observed execution trajectory and iteratively refines attacks using feedback from failed executions. We evaluate MUZZLE across diverse web applications, user tasks, and agent configurations, demonstrating its ability to automatically and adaptively assess the security of web agents with minimal human intervention. Our results show that MUZZLE effectively discovers 44 new attacks on 4 web applications with 10 adversarial objectives that violate confidentiality, availability, or privacy properties across different LLMs and agent scaffolds. MUZZLE also identifies novel attack strategies, including 3 cross-application prompt injection attacks and an agent-tailored phishing scenario.

1 Introduction

Web agents automate complex browser tasks but ingest untrusted content, creating indirect prompt-injection risks that existing evaluations address only with limited automation and scope. MUZZLE introduces an adaptive, fully automated framework that discovers and refines attacks across diverse web-agent settings.

  • Motivation: Web agents combine browser interaction with autonomous reasoning and are deployed for tasks including information gathering, form filling, shopping, and account management.They act through browser operations such as clicking, scrolling, typing, and switching tabs.
  • Motivation: Browser defenses struggle to enforce user intent, context, and outcomes because they were designed around human behavior rather than autonomous, goal-driven agents.Agents can chain authorized actions, reuse permissions, and adapt behavior at scale without bypassing browser controls.
  • Threat: Indirect prompt injections embed malicious instructions in web content, potentially overriding user intent and causing confidentiality, integrity, or availability violations.Web agents’ access to full browser context can amplify the consequences of successful injections.
  • Research gap: Prior evaluations manually selected pages, injection locations, or instructions, while some lacked live-environment evaluation and did not directly transfer from coding or RAG agents.These limitations leave automated, realistic attack discovery for web agents insufficiently addressed.
  • MUZZLE: MUZZLE adaptively discovers indirect prompt injections by generating end-to-end attack trajectories, ranking vulnerable UI elements, and iteratively synthesizing payloads.The framework supports diverse web applications, agent implementations, and LLM backends while targeting confidentiality, integrity, and availability violations, including cross-application attacks.

2 Background & Problem Statement

LLM web agents operate in iterative perception–action loops over dynamic, interconnected web environments, making indirect prompt injection a serious security risk. The paper frames automated red-teaming as a problem of discovering contextual, multi-step attacks under realistic black-box constraints.

  • Web agents: LLM web agents observe web content, reason over task progress, and issue browser actions while maintaining context through reasoning traces and memory components.Their observations may come from the DOM and screenshots, and their actions include searches, clicks, and form interactions.
  • Security risk: Indirect prompt injections place adversarial instructions in external content, causing LLM systems to follow attacker directions that can exfiltrate data, perform unauthorized actions, or alter task outcomes.Web agents are particularly exposed because they autonomously navigate sites and process untrusted content.
  • Evaluation environments: Existing environments provide useful web-agent evaluation, but earlier benchmarks offer limited control over environment state or adversarial manipulation, while sandboxed platforms improve reproducibility.The Zoo supports realistic workflows across interconnected applications with shared state and deterministic re-initialization.
  • Problem statement: The paper seeks an end-to-end system that automatically discovers, conducts, and evaluates prompt-injection attacks across long-running trajectories and interconnected web applications.The framework should support complex strategies that orchestrate multiple applications and modify encountered web content.
  • Challenges: Automated red-teaming must search an exponentially growing attack space, generate context-sensitive instructions, and adapt strategies to dynamic states and agent trajectories.This extends beyond local optimization of fixed HTML fields and motivates holistic attack discovery.
  • Threat model: The threat model assumes a black-box adversary can observe agent–LLM traffic during discovery, submit malicious client-facing content, and host attacker-controlled applications without modifying server logic or models.The adversary targets confidentiality, integrity, and availability using only execution traces and LLM input/output information.

3 MUZZLE System Design

MUZZLE is an automated, multi-agent framework for red-teaming web agents against indirect prompt injections. It discovers attack paths and injection surfaces from agent trajectories, synthesizes context-aware payloads, and refines attacks through feedback in a sandboxed environment.

  • System Goals: MUZZLE aims to automate reproducible red-teaming across web agents, LLMs, and applications with minimal human involvement.Operators provide the target agent, benign task, dependencies, and adversarial objectives; evaluation runs in a sandboxed environment.
  • Architecture Overview: The framework discovers end-to-end attack paths, vulnerable UI elements, and adversarial payloads that hijack agents toward specified objectives.Its stated objectives cover confidentiality, integrity, and availability violations.
  • Architecture Overview: MUZZLE uses three phases—Reconnaissance, Attack Synthesis, and Reflection—to select surfaces, generate payloads, and adapt strategies from observed behavior.The Explorer executes benign and adversarial tasks while collecting telemetry and interaction traces.
  • Reconnaissance Phase: Reconnaissance records agent execution telemetry and identifies high-leverage injection surfaces along the benign task trajectory.The Explorer also manages credentials and deterministic backend reinitialization in the sandbox.
  • System Goals: The task specification encodes the victim configuration, benign task, dependencies, and adversarial objectives used to drive discovery.The example includes credentials, an application URL, and a repository-deletion objective with an optional assertion.
  • Reconnaissance Phase: Candidate vessels pair UI-element descriptions with exploitation methods and scores, then are ranked by expected exploitability.Ranking considers visibility, adversarial privilege, available instruction space, and truncation likelihood.
  • Attack Synthesis Phase: Attack synthesis grounds payloads in execution traces and tests their placement in the agent’s context before full deployment.MUZZLE re-executes the benign task with a placeholder, locates when it becomes visible, and evaluates candidate payloads in the modified request.

4 Experimental Evaluation

MUZZLE discovers diverse indirect prompt injection attacks across applications, objectives, victim models, and agent scaffolds. Its evaluation reveals cross-application compromise, credential exfiltration, scaffold-dependent vulnerability, and runtime costs dominated by web-agent execution and LLM computation.

  • Overall Results: 44 distinct end-to-end attacks were uncovered across four web applications, multiple LLMs and scaffolds, with each attack manually verified.The attacks resulted in catastrophic outcomes for users.
  • Discovered IPI Attacks: Postmill produced 4 successful credential-exfiltration attacks through agentic phishing, plus 2 scam-posting attacks and 1 account-deletion attack.MUZZLE reframed malicious actions as intermediate verification steps using a spoofed authentication page.
  • Discovered IPI Attacks: Classifieds yielded 3 attacks deleting competing listings, 1 transferring account ownership, and account deletion causing permanent loss of access and data.The most successful objective was deleting competing listings across five runs.
  • Cross-Application Attacks: Two cross-application attacks achieved full compromise by deleting the Northwind orders table from a benign Classifieds task across five runs.The attacks leveraged stored credentials to authenticate into the separate destination application.
  • Victim LLM Choice: GPT-4.1 generally achieved higher end-to-end success than GPT-4o, including 4 successful account deletions out of five runs.GPT-4o more often disengaged from adversarial trajectories.
  • Web Agent Scaffold Choice: Agent-E more often completed attacks after hijacking, while BrowserUse sometimes recovered mid-execution; the difference reflects their scaffold architectures.Agent-E’s Planner receives only boolean executor feedback, whereas BrowserUse retains a unified reasoning loop.
  • Runtime Performance: 34.8% of runtime came from web-agent execution, 36.4% from environment initialization and task seeding, and 54.8% from aggregate LLM-dependent computation.Network proxying contributed 2.0%, while summarization, judging, and UI identification each contributed less than 4%.
  • Runtime Performance: MUZZLE’s overhead is concentrated in intrinsic LLM inference and web-agent execution, supporting its practical scalability for large-scale evaluations.The authors suggest model-serving latency or batching improvements would directly yield end-to-end speedups.

5 Related Work

Prior web-agent red-teaming approaches rely on fixed scenarios, manually selected injection surfaces, restricted environments, or attack-string optimization. These limitations leave automated discovery of dynamic, multi-step, and cross-application attacks insufficiently addressed.

  • Feedback-driven methods such as PAIR and TAP iteratively refine attacks, with TAP additionally searching multiple paths and pruning unpromising branches.
  • Existing benchmarks compile fixed datasets of known prompt-injection scenarios rather than discovering new attacks.
  • VWA-Adv manually selects injection vessels, starts agents at pre-selected locations, and does not support arbitrary attacker behavior within web environments.
  • AdvAgent evaluates agents on frozen HTML-image snapshots, preventing dynamic visible modifications and multi-step, cross-application attack evaluation.
  • AgentVigil mutates injection seeds using partial-success signals but inherits fixed per-scenario strategies and optimization limited to injection strings.

6 Conclusion

Web agents combine promising automated task capabilities with invalidated security assumptions and adversarial control over system-ingested content. MUZZLE addresses this gap through end-to-end automated red teaming and discovers attacks beyond restricted prior settings.

  • MUZZLE holistically discovers, refines, and evaluates indirect prompt-injection attacks against web agents through an end-to-end automated red-teaming framework.
  • MUZZLE discovers attacks including a sophisticated cross-application attack and an agent-tailored phishing attack that prior works are not equipped to discover.
  • MUZZLE provides a foundation for evaluating current and future web-agent systems against indirect prompt-injection attacks.

Ethical Considerations

The study confines all attacks to a closed sandbox and describes disclosure to relevant stakeholders. It frames the work as an automated evaluation framework while acknowledging its dual-use nature and the absence of dedicated defenses in evaluated agents.

  • All attacks were conducted exclusively within The Zoo, without accessing real infrastructure, live services, or user data.
  • The authors disclosed MUZZLE’s specific attack vectors to BrowserUse, Agent-E, and The Zoo’s developer under CFP ethics guidelines.
  • The paper distinguishes its contribution—the automated red-teaming framework—from the prior discovery of indirect prompt-injection threats.
  • None of the evaluated agents implement dedicated indirect prompt-injection defenses.

A Ablations

The ablation studies examine MUZZLE’s reflection, UI-element identification, and payload-generation mechanisms, including how attack discoveries vary across reflection iterations.

  • The ablations study MUZZLE’s Reflection, UI-element identification, and payload-generation mechanisms.
  • Table 7 reports Partial and End-to-end attacks discovered at different reflection iterations, using five runs per specification with GPT-4o as the target model.

A.1 Reflection Insights

MUZZLE often discovers end-to-end attacks in its first reflection iteration, but iterative refinement becomes important for complex cross-application attacks. Termination varies with the ranked UI elements available for exploration.

  • Reflection Insights: Termination varies because the Grafter’s candidate UI-element ranking differs across runs.Some objectives exhaust ranked elements early, whereas others continue exploring alternative UI vessels through later iterations.
  • Reflection Insights: The Northwind cross-application attack first succeeds at i = 2 and reaches 2 end-to-end attacks only at i = 5.This case illustrates the value of iterative refinement when attacks require multi-step coordination across application boundaries.

A.2 Component Ablation

The ablation evaluates MUZZLE’s Grafter and Payload Generator by comparing UI-element selection and payload-generation strategies on Gitea. Effective attacks require both suitable injection vessels and optimized payloads.

  • Component Ablation: MUZZLE’s ablation isolates the Grafter’s UI-element identification and the Payload Generator’s adversarial payload crafting.The experiment uses Gitea’s “add unauthorized collaborator” objective, evaluates variants over k = 5 runs, and deactivates reflection.
  • Component Ablation: Random UI-element selection produces no partial or end-to-end attacks across all payload variants.The red-team agent often cannot modify the randomly selected element, so attack synthesis is never reached.
  • Component Ablation: Fixed issue-title selection also fails because payloads exceed the field’s character limit and cannot adapt without reflection.The fixed strategy therefore fails entirely across the tested payload variants.
  • Component Ablation: Only MUZZLE’s optimized payload succeeds with Grafter-identified elements, producing 2 end-to-end attacks.Grafter prioritizes issue comments and bodies because their larger textarea areas keep injected payloads fully visible to the victim agent.

B Defense Evaluation

The evaluated prompt-classification guardrails perform poorly against MUZZLE’s discovered injections. Detection is weak for browser observations, and raw-payload detection varies substantially across methods.

  • Defense Evaluation: Browser-observation classification yields a Pareto frontier of 14%, 22%, and 32% TPR at 0%, 1.8%, and 5.5% FPR for LlamaGuard4, Llama PromptGuard 1, and DataSentinel, respectively.ProtectAI V2 instead produces an extremely high FPR of >98%.
  • Defense Evaluation: Raw-payload classification reaches detection rates up to 60% for several methods, including Llama PromptGuard 2 and LlamaGuard 3 and 4.Detection improves for some methods relative to full browser-content classification.
  • Defense Evaluation: DataSentinel’s raw-malicious-content TPR falls to 1.3%.Thus, some methods detect raw malicious text less effectively than full browser content.

C Examples

MUZZLE’s execution artifacts trace attack discovery from raw agent transcripts through attack evaluation. The examples also include representative indirect-prompt-injection payloads for web-application objectives.

  • Examples: The section presents example snapshots of indirect-prompt-injection payloads tied to representative adversarial objectives for each web application.These examples accompany the intermediate execution artifacts.
  • Examples: The artifacts cover raw execution transcripts, summarized traces, vulnerable UI-element identification, attack dispatching, payload generation, and attack evaluation.Together, these stages expose the intermediate outputs of MUZZLE’s red-team agents.

C.2 Snapshots

The snapshots illustrate MUZZLE’s end-to-end evaluation workflow: capturing web-agent executions, extracting attack surfaces, dispatching payloads, and reflecting on outcomes. They also show how the Judge distinguishes an injected instruction being read from it actually bypassing safeguards.

  • Execution snapshots: MUZZLE captures browser-agent execution transcripts as sequential time steps assembled from LLM API requests and responses.
  • Execution snapshots: Its Summarizer distills transcripts into step-wise traces without agent-scaffold-specific reasoning tags.
  • Attack construction: The Grafter ranks vulnerable UI elements using salience and exploitability, explaining why each surface can influence agent behavior and how it could be exploited.
  • Attack construction: The Dispatcher converts high-ranked UI elements and exploitation methods into imperative instructions for a web agent operating under malicious credentials.
  • Attack construction: Payload generation iteratively refines malicious instructions when earlier attempts fail to redirect the agent.
  • Reflection: The Judge can classify an attack as partial when the agent reads the injected instruction but refuses an unusual or risky operation, attributing the outcome to the instruction.
Loading 2602.09222v2…