PaperlayerView exact revision on arXiv

Source-linked AI summary

WIPI: A New Web Threat for LLM-Driven Web Agents

Fangzhou Wu, Shutong Wu, Yulong Cao, Chaowei Xiao

arXiv:2402.16965v1cs.CRcs.AI

TL;DR

Web Agents can retrieve external webpages, raising the question of whether they are secure against malicious instructions embedded in that content. This paper proposes WIPI, a black-box attack using crafted webpage prompts and stealth techniques; it reports average attack success above 90% across several Web Agent types. The authors conclude that current Web Agents are vulnerable and highlight the need for more secure systems.

  • Problem

    Web Agents retrieve and process external webpages, but their security against malicious instructions embedded in those pages is an open question.

    Source: §1 · pp. 1–2

  • Method

    WIPI embeds crafted natural-language instructions in public webpages and uses prompt-template and stealth techniques to attack the integrated Web Agent system in a black-box setting.

    Source: §1 · p. 2, §3.2 · p. 6

  • Results

    Over 90% average attack success was reported across 7 ChatGPT web plugins, 8 Web GPTs, and open-source Web Agents, with robustness across user-prefix instructions.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors conclude that current LLM-driven Web Agents are vulnerable to WIPI and highlight the urgency of building more secure Web Agents.

    Source: §1 · p. 3

    The attacker cannot inspect or modify system prompts or model parameters, or directly control other users’ conversation sessions.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

With the fast development of large language models (LLMs), LLM-driven Web Agents (Web Agents for short) have obtained tons of attention due to their superior capability where LLMs serve as the core part of making decisions like the human brain equipped with multiple web tools to actively interact with external deployed websites. As uncountable Web Agents have been released and such LLM systems are experiencing rapid development and drawing closer to widespread deployment in our daily lives, an essential and pressing question arises: "Are these Web Agents secure?". In this paper, we introduce a novel threat, WIPI, that indirectly controls Web Agent to execute malicious instructions embedded in publicly accessible webpages. To launch a successful WIPI works in a black-box environment. This methodology focuses on the form and content of indirect instructions within external webpages, enhancing the efficiency and stealthiness of the attack. To evaluate the effectiveness of the proposed methodology, we conducted extensive experiments using 7 plugin-based ChatGPT Web Agents, 8 Web GPTs, and 3 different open-source Web Agents. The results reveal that our methodology achieves an average attack success rate (ASR) exceeding 90% even in pure black-box scenarios. Moreover, through an ablation study examining various user prefix instructions, we demonstrated that the WIPI exhibits strong robustness, maintaining high performance across diverse prefix instructions.

1 Introduction

Web Agents’ access to external webpages raises security questions because malicious instructions can be embedded in content they retrieve. The paper proposes WIPI, a black-box attack on the integrated Web Agent system, and reports high attack success across multiple agent types.

1 Introduction

  • WIPI embeds crafted natural-language prompts in public webpages to make Web Agents execute unauthorized instructions, targeting the integrated system rather than only its LLM.The attack is designed for an online setting with externally deployed webpages and no access to or modification of system prompts or model parameters.Source: §1 · p. 2
  • The attack’s template counters preset instructions, preempts confirmation requests, and repeats instructions to maintain attention amid ordinary webpage content.The authors also place the instructions at the front of the webpage as an optimal position.Source: §1 · p. 2
  • Stealth techniques use natural-language prompts and manipulate font size, color, opacity, or layout to make instructions hard to detect or see.The paper notes that natural language can resemble ordinary content and is flexible across paraphrases and languages.Source: §1 · p. 2
  • Over 90% average attack success was reported across 7 ChatGPT web plugins, 8 Web GPTs, and open-source Web Agents, with robustness across user-prefix instructions.The introduction does not specify the open-source agent count in this result passage.Source: §1 · p. 3
  • The authors frame WIPI as a new web threat and call for more secure Web Agents based on their reported vulnerability findings.They also report evaluating individual attack strategies through ablation studies.Source: §1 · p. 3

2 New Web Threat: WIPI

The paper motivates WIPI by identifying gaps in prior indirect-injection evaluations and defines a practical black-box threat against Web Agents. Initial attacks reveal retrieval and execution obstacles that can prevent embedded instructions from being followed.

2.1 Motivation

  • Prior indirect-injection studies examined LLMs with local webpage datasets, but did not evaluate real-world web tools and the integrated Web Agent system.The authors contrast this with direct prompt injection, where the attacker is the current user and the target is the LLM’s output.Source: §2.1 · p. 3

2.2 Threat Model

  • WIPI uses crafted prompts in external webpages to induce unauthorized actions, while the black-box attacker cannot inspect system prompts or model parameters or control other users’ sessions.The attacker can use normal Web Agent functionality and manipulate website content.Source: §2.1 · p. 3, §2.2 · p. 3
  • WIPI’s natural-language payloads resemble ordinary page content, while its system-level target spans the LLM, external tools, and users.The paper describes retrieval of mixed page content and its subsequent return to the LLM for instruction execution.Source: §2.2 · p. 4

2.3 Challenges

  • 88% average success with prompts placed at the webpage head still left execution vulnerable to overlooking instructions amid normal content.When prompts appeared in the middle or tail, web tools could truncate them while retrieving the page.Source: Table 1 · p. 4, §2.3 · p. 5
  • During execution, prompts could be treated as ordinary content, rejected when misaligned with the page, or held for user confirmation.The paper attributes treating prompts as ordinary content to their small proportion relative to normal text, and names the mismatch obstacle “content-prompt misalignment.”Source: §2.3 · p. 5

3 Methodology

The methodology combines retrieval positioning and execution-focused instruction strategies with source/display separation to pursue executable yet stealthy WIPI attacks.

3.1 Solutions to Challenges in Retrieval Step.

  • Placing indirect prompts before normal webpage content aims to improve the chance that Web Agents retrieve and attend to them.The authors report that prompts placed in the middle or tail may not be retrieved by plugins.Source: §3.1 · p. 5

3.2 Solutions to Challenges in Execution Step.

  • To counter preset instructions and page-content distraction, WIPI adds counter-prompts and a prohibition on summarization, then places confirmation statements before and after the payload.The authors report that ChatGPT may refuse contradictory indirect instructions and may request confirmation before executing them.Source: §3.2 · pp. 5–6
  • Sentence-level repetition highlights lead instructions, while paragraph-level repetition repeats the full payload to improve attack stability.Sentence-level repetition also highlights confirmation and preset-prompt instructions; paragraph-level repetition excludes the preset-prompt-neglect prompts.Source: §3.2 · p. 6

3.3 Steathiness in the Wild.

  • Stealth strategies hide prompts in source code while keeping them readable to Web Agents; font size is the main strategy used in the experiments.Other strategies set prompt color to the background, opacity to zero, or position outside the screen; these are discussed in section 4.5.Source: §3.3 · p. 6

4 Experiments

WIPI was evaluated in black-box settings across plugin-based ChatGPT agents, Web GPTs, and open-source agents. The results show high attack success across these systems, robustness to user prefixes, and improved performance from the designed template.

4.1 Experimental Settings

  • In a black-box evaluation, the study tested 7 plugins and 8 Web GPTs, built open-source Web Agents with an Internet-retrieval extension, using 10 payloads across 4 webpage carriers and 5 trials per prompt.The 10 payloads comprised five prompts from the ACP dataset and five constructed by the authors.Source: §4.1 · pp. 7–8

4.3 Robustness on Preset Prompts.

  • With user prefixes, ASR remained 92.5% under four instructions, while testing a fixed prefix across four plugins produced an average drop of around 9%.For Web Pilot, the average drop versus direct URL input was 4%.Source: Table 7 · p. 10, §4.3 · p. 10, Table 8 · p. 10

4.4 Effectiveness of Prompt Template Design

  • Ablations yielded 35.5% ASR in the vanilla setting, 31% without summarization prohibition, 79% without sentence repetition, 81.5% without paragraph repetition, and 67% without confirmation.Removing both repetition strategies resulted in 75.5% ASR.Source: §4.4 · pp. 10–12, Table 9 · p. 11

4.5 Effectiveness under Stealthiness Strategies

WIPI remained executable under the tested stealth strategies, and case studies demonstrated several ways malicious webpage instructions could affect Web Agents. The authors also found that common scanners missed WIPI and that it could work through keyword search.

4.5 Effectiveness under Stealthiness Strategies

  • The four tested hiding strategies all achieved high ASR and did not degrade indirect-prompt executability.For most prompts, the strategies achieved almost 100% ASR.Source: §4.5 · p. 12

4.6 Case Study of Potential Security Threats

  • Case studies showed embedded prompts redirecting Web Agents, soliciting a phishing link, and displaying a hacker image.The authors describe possible user deception and property losses from malicious redirects, and risks including identity theft, financial fraud, malware infections, and compromised privacy from phishing links.Source: §4.6 · p. 12

4.7 Stealthiness under Web Safeguards

  • Across 10 prompts, one of 91 VirusTotal vendors flagged webpages as suspicious, while IPQS marked all tested webpages secure; both scanners missed language-based indirect redirects.The cited results describe the same vendor flagging the vanilla page2 and the version with injected prompts.Source: §4.7 · pp. 12–13

4.8 Exploration on Search-based WIPI

  • A keyword search for an anonymous webpage containing prompt1 successfully caused ChatGPT to execute the indirect instruction.The page appeared like a normal personal blog, and the demonstrated instruction made ChatGPT act as a Linux terminal.Source: §4.8 · p. 13, Figure 8 · p. 13, Figure 9 · p. 13

5 Related Works

Related work covers LLM-driven Web Agents and prompt injection, including indirect attacks through retrieved online resources. The paper distinguishes its focus on practical, whole-system attacks from earlier work centered on individual LLMs.

Web Agents

  • Prior Web Agent work includes simulated web environments for answering questions or purchasing products [61] [65], screenshot-and-HTML navigation [25], and agents with retrieval capabilities [22] [59] [66].Most described systems take text-formatted content as input; WebGUM [25] also uses screenshots and HTML.Source: §5 · p. 13

Prompt Injection

  • Prompt injection uses user-input prompts to bypass predefined rules, while indirect prompt injection can act through retrieval databases or accessible online resources [37] [38] [45] [46] [47] [48] [54] [62] [63].Greshake et al. introduce indirect prompt injection and analyze scenarios affecting LLM applications.Source: §5 · p. 13
  • The paper identifies a gap in prior studies that focused on individual LLMs rather than practical, whole-system Web Agents attacked through webpages [29].The text describes this threat as increasingly critical as users rely more on LLM applications for browsing and searching.Source: §5 · pp. 13–14

6 Conclusion

The paper presents WIPI as a natural-language web threat targeting entire Web Agent systems. It introduces a prompt template and hiding techniques, then reports average ASR above 90% across tested agents, including in black-box conditions.

6 Conclusion

  • WIPI uses natural language rather than executable code to target the entire Web Agent system.The paper contrasts this with earlier works targeting only the LLMs within Web Agents.Source: §6 · p. 14
  • The authors develop a universal prompt template for executing payload instructions and four hiding techniques focused on font style and layout location.The study considers seven ChatGPT web plugins, eight Web GPTs, and three open-source Web Agents.Source: §6 · p. 14
  • Over 90% average attack success rate was achieved across the tested Web Agents, even under black-box conditions.The authors say their findings reveal Web Agent vulnerabilities and provide insights for future secure LLM system design.Source: §6 · p. 14

A Investigation on Open-sourced Web Agents

The authors investigated whether open-source Web Agents could retrieve content from their deployed webpages. None of the eight tested agents succeeded, motivating development of their own open-source agents.

A Investigation on Open-sourced Web Agents

B Case study for Open-Sourced Web Agent

The case study shows WIPI misleading an open-source Web Agent despite a strong prefix instruction not to follow website instructions. The paper also reports that open-source models seem more vulnerable than commercial models like ChatGPT.

B Case study for Open-Sourced Web Agent

C Detailed Results for the ASRpage of attacking plugin-augmented GPT4

The appendix points readers to Figure 11 for detailed ASRpage results from attacks on plugin-augmented GPT-4.

C Detailed Results for the ASRpage of attacking plugin-augmented GPT4

D Detailed Payload Instructions

The appendix directs readers to Table 14 for the payload instructions used in the experiments.

D Detailed Payload Instructions

E Detailed Results for Web GPTs

The appendix directs readers to Table 15 for detailed results on attacks against GPT-based Web Agents.

E Detailed Results for Web GPTs

Loading 2402.16965v1…