PaperlayerView exact revision on arXiv

Source-linked AI summary

EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage

Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, Huan Sun

arXiv:2409.11295v5cs.CRcs.AIcs.CLcs.LG

TL;DR

Generalist web agents may expose users’ private information when they interact with compromised websites, a privacy risk that has received limited study. The paper develops a threat model and Environmental Injection Attack (EIA) for adaptive webpage-based leakage, evaluating them on realistic tasks. EIA reaches up to 70% ASR for specific PII and 16% for full user requests, while well-adapted attacks are difficult to detect and mitigate.

  • Problem

    Privacy risks from compromised websites remain largely unexplored for generalist web agents performing tasks that involve users’ PII.

  • Method

    The paper proposes a threat model covering specific PII and full-request theft, then develops EIA with webpage-adaptive injection strategies and evaluates it on 177 realistic PII-related action steps using SeeAct.

  • Results

    EIA achieves up to 70% ASR for stealing specific PII and increases full-request leakage from 0% with standard EIA to 16% with Relaxed-EIA using GPT-4V.

  • Takeaways & Limitations

    Well-adapted EIA attacks are difficult to detect and mitigate, and additional attacker effort can make compromised webpages appear nearly identical to benign ones.

  • Takeaways & Limitations

    Human inspection can detect attacks that are not well adapted to a webpage, but its reliability declines when attackers invest extra effort in seamless adaptation.

Abstract

from arXiv · show

Generalist web agents have demonstrated remarkable potential in autonomously completing a wide range of tasks on real websites, significantly boosting human productivity. However, web tasks, such as booking flights, usually involve users' PII, which may be exposed to potential privacy risks if web agents accidentally interact with compromised websites, a scenario that remains largely unexplored in the literature. In this work, we narrow this gap by conducting the first study on the privacy risks of generalist web agents in adversarial environments. First, we present a realistic threat model for attacks on the website, where we consider two adversarial targets: stealing users' specific PII or the entire user request. Then, we propose a novel attack method, termed Environmental Injection Attack (EIA). EIA injects malicious content designed to adapt well to environments where the agents operate and our work instantiates EIA specifically for privacy scenarios in web environments. We collect 177 action steps that involve diverse PII categories on realistic websites from the Mind2Web, and conduct experiments using one of the most capable generalist web agent frameworks to date. The results demonstrate that EIA achieves up to 70% ASR in stealing specific PII and 16% ASR for full user request. Additionally, by accessing the stealthiness and experimenting with a defensive system prompt, we indicate that EIA is hard to detect and mitigate. Notably, attacks that are not well adapted for a webpage can be detected via human inspection, leading to our discussion about the trade-off between security and autonomy. However, extra attackers' efforts can make EIA seamlessly adapted, rendering such supervision ineffective. Thus, we further discuss the defenses at the pre- and post-deployment stages of the websites without relying on human supervision and call for more advanced defense strategies.

1 INTRODUCTION

This work studies privacy risks when generalist web agents interact with compromised websites and introduces Environmental Injection Attack (EIA), which adapts malicious webpage content to mislead agents into leaking specific PII or full user requests. Experiments show substantial leakage success, while detection and mitigation remain difficult, especially when attacks are well adapted to webpages.

  • Motivation: The paper addresses privacy risks arising when generalist web agents perform tasks requiring sensitive PII on potentially compromised websites.Examples of sensitive information include phone numbers and credit card details.
  • Threat model: The threat model considers two adversarial targets: stealing users’ specific PII and stealing full user requests.The attack is designed under realistic website-compromise scenarios.
  • Method: Environmental Injection Attack (EIA) adapts malicious web elements and persuasive instructions to webpage environments, using Form Injection and Mirror Injection strategies.The injected elements exploit CSS and JavaScript to improve stealthiness and manipulate agents during state-changing actions.
  • Evaluation: 177 action steps from realistic Mind2Web-derived websites show that Mirror Injection reaches up to 70% ASR for stealing specific PII when placed near the target element.The experiments target SeeAct, a two-stage generalist web agent framework.
  • Evaluation: 16% ASR for leaking the full user request is achieved by Relaxed-EIA, compared with 0% for standard EIA using GPT-4V.Relaxed-EIA makes injected elements slightly visible, influencing both action grounding and action generation.
  • Detection and defenses: EIA is hard to detect and cannot be countered by a defensive system prompt, although poorly adapted attacks may be found through close human inspection.Additional attacker effort can make compromised webpages visually identical or nearly identical to benign pages, limiting the reliability of human supervision.

2 RELATED WORK

Prior work studies direct or indirect prompt injection, web agents built for retrieval or webpage interaction, and limited attacks against web agents. This paper differs by targeting generalist agents performing complex tasks on realistic websites under a more realistic privacy threat model.

  • Direct and Indirect Prompt Injection: Direct prompt injection manipulates input messages, whereas indirect prompt injection inserts malicious instructions into information retrieved by an AI system.Indirect attacks remotely alter model behavior through retrieved application information.
  • Web Agents: Some web agents augment language models with website retrieval, while others take raw HTML or rendered screenshots to perform direct website interactions.SeeAct uses rendered screenshots alongside HTML and is presented as a stronger two-stage framework for web tasks.
  • Existing Attacks against Web Agents: Existing attacks include backdoor triggers, manipulated item images, and injections into retrieved content, but few studies examine HTML injections against generalist web agents.Prior retrieval-focused work assumes knowledge of user requests and mainly disrupts summarization rather than targeting complex website tasks.
  • Existing Attacks against Web Agents: This work targets generalist web agents capable of complex tasks on realistic websites rather than retrieval-augmented systems limited to information-seeking or summarization.Its focus is a more realistic privacy threat model involving agents that act on websites.

3 ENVIRONMENTAL INJECTION ATTACK AGAINST WEB AGENTS

This section formulates EIA as an environment-adapted injection attack against web agents, defining its threat model, injection strategies, and mechanisms for stealthy data exfiltration.

  • 3.1 BACKGROUND ON WEB AGENT FORMULATION: SeeAct generates actions from webpage screenshots and HTML, then grounds textual action descriptions into web events.Its default implementation uses screenshots for action generation and textual choices for action grounding.
  • 3.2 THREAT MODEL: EIA targets either specific PII or the complete user request under assumptions requiring task-agnostic attacks that preserve intended task completion.The threat model considers compromised websites arising from benign developers using contaminated tools or related website compromises.
  • 3.3 ENVIRONMENTAL INJECTION ATTACK STRATEGIES: EIA injects persuasive instructions into benign HTML while adapting opacity and position to the target web environment.The attack is formulated as h* = E(h, PI, α, β), with α controlling visibility and β specifying a position relative to the target element P0.
  • 3.3 ENVIRONMENTAL INJECTION ATTACK STRATEGIES: EIA uses zero-opacity injected elements and JavaScript auto-submission to reduce visibility and exfiltrate typed private information without a button click.The implementation monitors typing and submits the information after a one-second interval once typing stops.
  • 3.3 ENVIRONMENTAL INJECTION ATTACK STRATEGIES: Zero-opacity injections can still occupy webpage space, while additional attacker effort can make them more seamlessly adapted to individual websites.The paper notes that automatic injection may introduce visible whitespace when adaptation quality is low.
  • 3.3 ENVIRONMENTAL INJECTION ATTACK STRATEGIES: Form Injection creates an HTML form, whereas Mirror Injection mirrors the target element to blend malicious instructions into the webpage.Form instructions may appear in text or aria-label attributes; the form carrier supports data submission through common HTML functionality.

4 EXPERIMENTS

The experiments evaluate EIA on 177 PII-related action steps using SeeAct and three LMM backbones. EIA reaches high specific-PII leakage, while Relaxed-EIA enables limited full-request leakage by affecting screenshot-based action generation.

  • 4.1 EXPERIMENTAL SETTINGS: The evaluation uses GPT-4V, Llava-1.6-Mistral-7B, and Llava-1.6-Qwen-72B as SeeAct backbones on realistic Mind2Web-derived websites.The dataset contains 177 manually verified action steps involving diverse PII categories and task types.
  • 4.1 EXPERIMENTAL SETTINGS: Automatic EIA injection may reduce adaptation quality by introducing extra whitespace, although attackers can customize attacks for specific webpages.The paper identifies this as a boundary of scalable evaluation rather than a limitation on attackers’ potential effort.
  • 4.2 EIA TO STEAL SPECIFIC PII: Up to 70% ASR is achieved for specific-PII leakage against GPT-4V, remaining relatively effective across LMM backbones despite differing benign step success rates.The reported attack success concerns selecting the injected element and typing values similar to the ground-truth target values.
  • 4.2 EIA TO STEAL SPECIFIC PII: Injections near the target perform better, with Mirror Injection at P+1 reaching 70% ASR for GPT-4V.The paper attributes this pattern partly to positional bias toward the malicious element appearing immediately before the target.
  • 4.2 EIA TO STEAL SPECIFIC PII: Mirror Injection attains the highest ASR, likely because it mirrors webpage styles and naming conventions, while Form Injection appears more consistent across positions.Mirror Injection nevertheless shows lower average ASR and higher variance in the reported comparison.
  • 4.3 EIA TO STEAL FULL USER REQUESTS: Standard EIA yields 0% ASR for full-request leakage because zero-opacity content does not affect screenshot-only action generation.The unaffected generation stage continues producing descriptions whose value corresponds to the specific PII field rather than the full request.
  • 4.3 EIA TO STEAL FULL USER REQUESTS: Relaxed-EIA raises full-request leakage to 16% ASR by making injected content slightly visible and influencing both action-generation and action-grounding stages.The approach sets opacity to a low non-zero value and uses the compromised generated value to guide subsequent grounding.

5 ATTACK DETECTION AND MITIGATION

The evaluation examines whether EIA is detectable through security tools, agent functional integrity, or defensive prompts. EIA remains difficult to detect and does not materially disrupt subsequent agent actions or respond to the tested prompt defense.

  • Traditional Web Security Tool: VirusTotal flagged none of the webpages after EIA or Relaxed-EIA injection.The injected malicious content consists of innocuous natural language in HTML rather than executable code.
  • Agent Functional Integrity: ASRpt remains very close to ASR for both EIA and Relaxed-EIA, indicating minimal effect on subsequent agent actions.The authors partly attribute this to their auto-submission mechanism.
  • Defensive System Prompt: A defensive system prompt leaves ASRs nearly identical to those under the default system prompt for both EIA and Relaxed-EIA.The tested prompt warned agents about prompt injection and discouraged unusual webpage elements or actions.

6 DISCUSSIONS

The discussion frames EIA as a privacy threat beyond traditional web attacks and examines the limits of human supervision and existing defenses. It emphasizes that stronger defenses must preserve web-agent functionality while addressing attacks that can be adapted to look benign.

  • Human Supervision: Human supervision creates a trade-off: reduced oversight increases autonomy but can make agents more vulnerable, while sustained visual monitoring burdens users.Users may spot poorly adapted visual changes more easily when actively monitoring accessed websites.
  • Human Supervision: Supervision should vary by task type, with close oversight for PII tasks and greater autonomy for information-seeking tasks.Constant supervision can be impractical during activities such as booking a flight while driving, and information-seeking requests can also expose privacy.
  • Human Supervision: Well-adapted EIA pages can have minimal or no visual changes, making them extremely difficult for users to detect even under supervision.The paper reports five examples of attacks adapted almost seamlessly into webpages.
  • Pre- and Post-Deployment Defenses: Keyword filtering and zero-opacity filtering face practical limits because benign webpage text and invisible elements can resemble attack content.Blanket removal of zero-opacity elements could disrupt intended webpage flow or functionality.
  • Pre- and Post-Deployment Defenses: Defense strategies that indiscriminately prioritize instructions over webpage data may impair navigation, motivating defenses that preserve original functionality.Descriptive text and aria labels embedded in webpage data provide context needed for web interaction.
  • Uniqueness and Importance of EIA: EIA can leak a user’s full request, unlike traditional attacks targeting PII typed into webpage fields.The full request contains information beyond specific PII and may create more serious privacy risks.

7 CONCLUSION

The paper studies privacy leakage by generalist web agents through a threat model, EIA attacks, and experiments with SeeAct. It finds that attacks can leak private information while remaining difficult to detect or mitigate, motivating broader investigation and defenses.

  • Conclusion: The study develops a realistic threat model and applies EIA to SeeAct using experiments on diverse PII-related action steps.It also examines adaptation strategies, detection, mitigation, human supervision, and website defense stages.
  • Conclusion: EIA attacks can leak users’ specific PII and full requests while remaining challenging to detect and mitigate.The conclusion reports that extra attacker effort can make attacks seamlessly adapted to webpages, reducing the reliability of human supervision.
  • Conclusion: The study underscores the need for more comprehensive exploration of privacy leakage risks in generalist web agents.It also emphasizes EIA’s distinction from traditional web attacks and implications for defenses without human supervision.

ETHICS STATEMENT

The ethics statement presents EIA as a privacy-risk study conducted without compromising real user privacy. It also notes that current web-agent technology is still early-stage, limiting immediate real-world threat claims.

  • Ethical Data Use: The evaluation uses public Mind2Web data cached offline rather than attacking live websites, avoiding compromise of real user privacy.The methodology is intended to investigate potential leakage risks safely.
  • Scope of Risk: The reported attacks do not pose immediate real-world threats because web-agent technology remains in an early developmental stage.The paper notes that the field is rapidly evolving toward more powerful multimodal models.

REPRODUCIBILITY STATEMENT

The materials document the evaluation setup and visual examples used to assess EIA’s adaptation and stealthiness across realistic websites. They include benign and injected webpage screenshots, including cases where human effort makes injections visually indistinguishable.

  • Reproducibility materials: The evaluation materials include LMM backbone details, Mind2Web-derived PII tasks, success-rate definitions, ASR variants, and evaluation thresholds.The authors state that related materials and running results will be open-sourced upon acceptance.
  • Benign reference: The benign GameStop webpage serves as the visual reference for comparing injected versions.The original rendered screenshot is reused as the benign reference in the evaluation figures.
  • Adapted injections: Five webpage pairs show original versions alongside counterparts marking the locations of almost seamlessly adapted EIA injections.The examples span GameStop, HealthGrades, United Airlines, BenefitsCal, and Amazon.
  • Adaptation conditions: Four examples can be well adapted at studied positions β = Pn, where n ∈ {±1, ±2, ±3}.The GameStop example required additional adjustment beyond exploring injection positions.
  • Adaptation conditions: Human effort adjusted the GameStop injection’s position and size until the malicious element became visually indistinguishable from the benign webpage.The authors describe this fine-tuning as reasonable and expected in real attack scenarios.
  • Visual artifacts: An injection with zero opacity can still occupy webpage space, while Relaxed-EIA uses low opacity and may visibly alter the screenshot.The examples contrast a noticeable white-space artifact with low-opacity placement at P−1.

F LIMITATIONS

The stated limitations concern evaluation scope and the need for more realistic end-to-end assessment. The study uses offline action-step evaluations rather than monitoring privacy risk throughout interactive task completion.

  • Evaluation scope: The evaluation adapts Mind2Web action steps offline, but does not assess end-to-end request completion and ASR throughout a real-time interactive web environment.The authors identify this as important for fully assessing web-agent capabilities and associated risks.
  • Evaluation scope: The curated evaluation data and detailed ASR results are reported separately, with domain, subdomain, and risk-type results averaged over eight injection positions.The positions are represented by β values studied in the experiments.

G.1 DATA DISTRIBUTION PLOTS

The appendix materials characterize the PII evaluation set by domain, subdomain, PII category, and attack success rate. They also document prompt variants and the eight-position full-request analysis.

  • Data distribution: The evaluation tasks are distributed across domains and subdomains, with subdomain counts shown separately.Figure 18 uses an inner circle for domains and an outer circle for subdomains.
  • Data distribution: The PII-category frequency plot counts dataset instances containing each type of PII.The figure is a pie chart of category-instance frequencies.
  • Attack success analysis: ASR is reported across domains, PII categories, and subdomains.These plots organize attack success by task and privacy-data dimensions.
  • Prompt design: Identity verification language and capital letters were incorporated into the template to improve full-request leakage effectiveness.The template instructs the agent to type the task description exactly before entering the specified PII category.
  • Defensive prompts: Three defensive system prompts were tested, and Figure 4 reports their averaged result.The prompts instruct the agent to avoid unusual webpage elements and rely on screenshot-visible content.
  • Attack success analysis: The full-user-request analysis reports ASR across eight injection positions and three backbone models.This comparison isolates positional and backbone variation for full-request leakage.

J.2 MORE ANALYSIS BETWEEN EIA AND RELAXED-EIA

The analysis explains why standard EIA fails to leak full requests and why Relaxed-EIA can succeed by influencing both action-generation and grounding stages. It also connects this capability to broader privacy and web-agent risks.

  • Metrics: ASRo measures correct selection of the injected element regardless of whether the typed values are correct, whereas ASR requires both selection and typed values to be correct.This distinction separates partial steering from successful full-request leakage.
  • Standard EIA: 0% ASR is obtained by standard EIA across all tested positions and three backbone models.Agents can select the injected element, but fail to type the full user request into it.
  • Standard EIA: The failure occurs because action grounding follows action-generation reasoning and textual descriptions, while action generation processes screenshots rather than compromised HTML.Standard EIA affects grounding but leaves the generated description aligned with the user’s intended task.
  • Relaxed-EIA: Relaxed-EIA slightly improves ASRo, but selecting the injected element alone does not guarantee successful full-request leakage.The authors distinguish element selection from correctness of the resulting typed content.
  • SeeAct pipeline: The SeeAct examples use two stages: action generation followed by grounding through textual choices.The appendix includes examples of both action generation and textual-choice grounding.
  • Privacy implications: Full-request leakage exposes information beyond specific PII and is unavailable to traditional attacks that only capture values entered into benign webpage fields.Relaxed-EIA can redirect the agent to type the high-level user instruction into an injected field.
  • Broader implications: Web-agent use expands attack surfaces because environmental injections may target disclosure or other agent behaviors beyond traditional web attacks.The paper gives wrong-item purchasing as an example of a possible non-privacy behavior manipulation.
Loading 2409.11295v5…