Source-linked AI summary
The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
Chaoran Chen, Zhiping Zhang, Bingcan Guo, Shang Ma, Ibrahim Khalilov, Simret A Gebreegziabher, Yanfang Ye, Ziang Xiao, Yaxing Yao, Tianshi Li, Toby Jia-Jun Li
TL;DR
GUI agents must handle sensitive user data across dynamic interfaces, but adversarial GUI content can manipulate their actions or cause unintended disclosure. The paper evaluates six attack types across six agents, 234 webpages, and 39 human participants, finding broad vulnerability—especially to contextually embedded attacks—and argues for privacy-aware designs that account for limits of human oversight.
Problem
GUI agents’ autonomous access to sensitive data and high-privilege interface elements creates poorly understood privacy and security risks under adversarial web content.
Method
The study evaluates six GUI agents against six attack types across 234 webpages and benchmarks them against 39 human participants under adversarial conditions.
Results
GUI agents were broadly vulnerable, with FPI success rates of 66–74% for GPT-4o, Claude, and DeepSeek and deceptive-default success near 100% across most agents.
Takeaways & Limitations
The findings support privacy-aware agent design using saliency-aware parsing, stricter execution constraints, and more transparent user-agent interactions.
Takeaways & Limitations
The controlled, pre-defined attack scenarios may not capture the complexity and unpredictability of real-world websites and tasks.
Abstract
from arXiv · showhide
A Large Language Model (LLM) powered GUI agent is a specialized autonomous system that performs tasks on the user's behalf according to high-level instructions. It does so by perceiving and interpreting the graphical user interfaces (GUIs) of relevant apps, often visually, inferring necessary sequences of actions, and then interacting with GUIs by executing the actions such as clicking, typing, and tapping. To complete real-world tasks, such as filling forms or booking services, GUI agents often need to process and act on sensitive user data. However, this autonomy introduces new privacy and security risks. Adversaries can inject malicious content into the GUIs that alters agent behaviors or induces unintended disclosures of private information. These attacks often exploit the discrepancy between visual saliency for agents and human users, or the agent's limited ability to detect violations of contextual integrity in task automation. In this paper, we characterized six types of such attacks, and conducted an experimental study to test these attacks with six state-of-the-art GUI agents, 234 adversarial webpages, and 39 human participants. Our findings suggest that GUI agents are highly vulnerable, particularly to contextually embedded threats. Moreover, human users are also susceptible to many of these attacks, indicating that simple human oversight may not reliably prevent failures. This misalignment highlights the need for privacy-aware agent design. We propose practical defense strategies to inform the development of safer and more reliable GUI agents.
1 Introduction
GUI agents automate adaptive interactions with complex interfaces but create poorly understood privacy and security risks, especially when adversarial instructions are embedded in legitimate-looking content. The study evaluates six attack types and finds broad agent vulnerability, limited protection from human oversight, and a need for privacy-aware design.
- Agent capabilities: GUI agents interpret multimodal interfaces and adapt actions dynamically rather than relying on predefined scripts.They translate natural-language commands into sequences of clicks, typing, and taps.
- Privacy risks: Users have limited ability to anticipate or monitor the sensitive information agents access during long or repetitive workflows.High-privilege interface elements such as file uploads and form submissions create additional opportunities for manipulation.
- Study design: Six GUI agents were evaluated against six attack types across 234 webpages on 19 real-world websites.The study includes established patterns such as stealing private information, deceptive defaults, and unaligned behavior, alongside interface-friction and denial-of-service mechanisms.
- Attack landscape: Contextually embedded attacks exploit agents’ tendency to process low-salience text, including plausible privacy policies or terms of service, without sufficient discrimination.Fine-Print Injection differs from attacks based on conspicuous prompts or task-irrelevant disruptions.
- Findings: FPI success rates reached 66–74% for GPT-4o, Claude, and DeepSeek, while deceptive-default attacks approached 100% across most agents.Only the conservative Operator agent showed partial resistance to deceptive defaults.
- Findings: 97.4% of human participants consented to malicious privacy policies, indicating that user supervision alone cannot guarantee safety.More capable agents were also more vulnerable to manipulation, whereas conservative agents resisted attacks but often failed to complete tasks.
- Design implications: The paper recommends saliency-aware parsing, memory constraints, and interface-level controls to mitigate privacy risks.These strategies address the mismatch between agent behavior, human expectations, and adversarial interface conditions.
2 Background: Theoretical Groundings of the Attacks
GUI agents can adapt to changing interfaces, but their surface-oriented processing differs from human selective attention and contextual judgment. This perceptual flattening helps explain vulnerability to deceptive defaults, manipulative friction, fine-print instructions, and actions that diverge from user intent.
- GUI interaction: GUI agents dynamically interpret changing interfaces, enabling adaptation to layout shifts and content changes without rigid scripts.Their reliance on surface-level UI features also creates privacy and security risks in adversarial settings.
- Affordances: Affordance-like behavior leads agents to fill visible text boxes or click labeled elements based on surface forms.The paper contrasts this behavior with human identification of possible actions from environmental cues.
- Cognitive load: Automation reduces cognitive overhead but can encourage over-reliance and limit direct oversight during repetitive tasks.Dual-Process Theory is used to motivate why humans and agents may default to fast, habitual responses.
- Visual saliency: GUI agents process visual content more uniformly than humans, assigning semantic weight to fine print, disclaimers, and irrelevant text.This contrasts with human selective attention to prominent or eye-catching elements.
- Deceptive interfaces: Deceptive defaults exploit acceptance of pre-selected options, manipulative friction adds unnecessary steps, and fine-print injection embeds malicious commands in long-form interface text.These attacks are presented as consequences of weak filtering and perceptual flattening.
- Context and norms: Unaligned behavior occurs when agents take syntactically plausible actions that diverge from user intent or violate contextual norms.The paper links these failures to weak privacy reasoning and the absence of a robust model of context-appropriate behavior.
- Synthesis: Differences from human saliency prioritization, context discrimination, and norm sensitivity create distinct vulnerabilities despite task-level competence.These vulnerabilities motivate new threat models and design strategies.
3 Threat model
The threat model considers GUI agents acting for users on third-party websites while handling sensitive information and facing adversaries who control the web interface. It focuses on inference-stage attacks embedded in GUI content and excludes attacks targeting internal model knowledge, infrastructure, or intentionally authorized leakage.
- Scenario: The agent acts for a user on a third-party website and may process names, contact details, identification numbers, or payment credentials.The model distinguishes task-appropriate checkout fields from unrelated or inappropriate requests for sensitive information.
- Adversaries: Adversaries include phishing-site developers, hackers who compromise websites, and attackers who inject commands through user-generated content.These actors exploit interfaces that the agent autonomously visits or operates.
- Adversary goals: An attack succeeds when it obtains task-unrelated sensitive data or induces actions that violate user intent.Examples include incorrect forms, harmful defaults, unauthorized disclosure, or inappropriate requests for health information.
- Capabilities: The adversary controls the web interface but cannot alter the agent’s architecture, model weights, or task definition.The adversary may add deceptive elements, misleading defaults, manipulated metadata, or malicious long-form content.
- Scope boundaries: The study excludes model inversion, training-data extraction, membership inference, direct prompt injection, and infrastructure-level threats.Its scope is inference-stage interaction with adversarial web content embedded in fields, labels, policies, or similar GUI elements.
4 Study Design
The study evaluates six attack types against screenshot-based GUI agents and human participants using realistic, high-risk web tasks. It varies attack types, task cases, and foundation LLMs, with a comparative human baseline.
- Study Design: The factorial study varies adversarial attack types, web task cases, and foundation LLMs, complemented by a comparative human baseline.
- Attack Types: Six attack types span private-information theft, denial of service, unaligned behavior, deceptive defaults, manipulative friction, and Fine-Print Injection.FPI embeds adversarial commands in dense or legalistic text, such as privacy-policy pop-ups.
- Test Cases: The experiment uses 39 tasks across 19 websites and modifies each task across all six attacks, yielding 234 adversarial cases.Tasks are drawn from Mind2Web and emphasize realistic interactions in high-risk domains such as healthcare, government services, and finance.
- Agents: Six screenshot-based GUI agents are evaluated, including Operator, Claude 3.7 Sonnet, GPT-4o, Gemini 2.0 Flash, LLaMA3.3 70B Instruct, and DeepSeek-V3.The agents interact through screenshots and browser actions such as clicking, typing, and scrolling; non-Operator agents use Browser Use.
- Human and Agent Procedure: Agent prompts include privacy-conscious role instructions and contextual personal information, while participants receive fictional personas with synthetic data and complete the same adversarial tasks.The study records task outcomes, DOM-level interactions, and privacy-compromising behaviors; human participants rate delegation willingness before and after tasks.
5 Results
GUI agents combine strong task execution with substantial vulnerability to privacy and interface manipulation. Their failures differ from humans across attack types, while more capable models trade safety for utility.
- Contextual and embedded attacks: Agents are highly vulnerable to contextually embedded attacks, especially Fine-Print Injection and Deceptive Default attacks.FPI embeds malicious commands in plausible privacy policies, while DD exploits pre-selected privacy-invasive options.
- Contextual and embedded attacks: 74.36% for Claude, 66.67% for GPT-4o, and 71.79% for Deepseek were the Attack Success Rates under Fine-Print Injection.FPI outperformed more overt privacy attacks because its commands blended into plausible interface content.
- Human–agent differences: Humans and agents both failed privacy attacks but for different reasons: agents over-processed low-salience content, while humans often overlooked or trusted fine print.38 of 39 participants, or 97.4%, accepted privacy policies containing malicious commands.
- Dark-pattern susceptibility: Deceptive Default attacks achieved near-perfect success across agents, whereas Manipulative Friction attacks were largely ineffective.Agents tend to accept pre-selected options but rarely take proactive steps outside the explicit task objective.
- Privacy–utility trade-off: More capable foundation models completed tasks more reliably but were more vulnerable, whereas conservative Operator resisted manipulation at the cost of usability.Operator’s conservative strategy reduces privacy violations but can halt execution or require confirmation even for relevant actions.
- Design implications: The findings motivate selective halting, contextual-integrity reasoning, confidence thresholds, and user-configurable privacy policies.These controls are proposed to balance task completion with context- and risk-sensitive behavior.
6 Discussion
The discussion frames GUI-agent vulnerabilities as a systems and governance problem: high utility can coexist with severe privacy risk, and human oversight is unreliable. It therefore supports broader robustness, accountable human–agent collaboration, and evaluation beyond controlled tasks.
- Implications for sensitive workflows: Deceptive Defaults exceeded 90% success across nearly all agents, while Fine-Print Injection reached 74.36% for Claude and 71.79% for DeepSeek.These attacks can cause task deviations or inappropriate disclosures in sensitive domains such as e-commerce, health, and finance.
- Implications for sensitive workflows: The authors caution against current GUI agents in sensitive workflows because human oversight did not reliably mitigate manipulation.97.4% of participants accepted malicious privacy policies despite the risks.
- Human–agent differences: Humans and agents share privacy vulnerabilities but diverge on dark patterns: humans were more affected by Manipulative Friction, while agents were more affected by Deceptive Defaults.The reported human ASRs were 35.90% for DD and 66.67% for MF, whereas agents often exceeded 97% for DD.
- Designing accountable systems: The privacy–utility trade-off means advanced agents can deliver high task completion while conservative agents reduce attack success by sacrificing functionality.Operator’s risk-averse behavior lowers ASRs but produces task-completion rates below 50% in several tasks.
- Designing accountable systems: The authors argue for human-in-the-loop systems with attentional scaffolding rather than relying on full automation or passive oversight.Scaffolds should direct attention toward decisions requiring intervention without eliminating agent autonomy.
- Broader robustness: Robustness evaluation should include interface-level adversaries and interactional resilience, not only input-level attacks or model internals.The discussion identifies malicious content, deceptive flows, and coercive prompts as environmental threats.
- Limitations and future work: The study’s controlled, pre-defined attack scenarios may not capture the complexity and unpredictability of real-world websites and tasks.The authors call for in-the-wild evaluation across broader interaction patterns, content structures, and evolving adversarial techniques.
- Limitations and future work: The human baseline is difficult to interpret because participant attention to website content could not be confirmed.The authors also identify the need to study multi-turn decisions, adaptive adversaries, and real-time intervention strategies.
7 Related Work
GUI agents extend automation across web, mobile, and desktop environments but introduce privacy and security vulnerabilities. Prior work identifies black-box and white-box attacks, including deceptive pop-ups, environmental injection, and multimodal manipulation.
- GUI agents automate web navigation, mobile control, and desktop tasks, but their broader deployment raises security and privacy concerns.
- Black-box attacks manipulate the agent’s environment, including deceptive pop-ups that induce unintended interactions and compromise user data.
- Environmental Injection Attacks modify surrounding digital elements or background cues to coerce incorrect inferences or unauthorized interactions.
- White-box attacks exploit internal knowledge of the agent’s model, architecture, decision process, multimodal capabilities, or model weights.
- Prior security matrices highlight privacy leaks from ambiguous visual inputs, adversarial UI elements, and indirect prompt manipulation.
8 Conclusion
The conclusion reports broad vulnerabilities across six attack types and emphasizes Fine-Print Injection as an effective attack exploiting agents’ reliance on low-salience content. The evaluation uses adversarial webpage transformations illustrating privacy theft, denial of service, deception, dark patterns, friction, and embedded malicious instructions.
- Six attack types were evaluated across GUI agents and benchmarked against human users, revealing widespread vulnerabilities in realistic interaction scenarios.
- Fine-Print Injection exploits agents’ overreliance on low-salience content and proved highly effective across multiple models.
- The conclusion calls for saliency-aware parsing, stricter execution constraints, and greater transparency in user-agent interactions.
- Adversarial webpage examples transform legitimate forms by inserting hidden data-exfiltration fields, denial-of-service links, or deceptive disclosure instructions.
- Other transformations use automatic VIP enrollment, persuasive opt-in friction, or privacy-policy clauses directing users to malicious websites.
B.1 Pre-task Willingness Rating
The pre-task survey measures participants’ willingness to delegate six everyday web tasks to an AI agent. Participants rate each task on a seven-point willingness scale.
- Participants rated willingness to let an AI agent perform six everyday tasks on their behalf.
- Willingness was measured using a seven-point Likert scale from completely unwilling to completely willing.
B.2 Mini Test
The mini test begins six short web tasks and instructs participants to use a provided fake profile when entering personal information. The supplied passages also identify the human sample size and completion reliability table.
- Participants completed six short web tasks and were instructed to use information from an assigned fake profile.
- The mini test includes entering an airline confirmation code while checking flight information.
- The human study sample contained 39 participants.
- A separate table reports inter-rater reliability for task completion using Gwet’s AC1.
B.3 Web-based Tasks
Participants completed six tasks involving different attacks, following standardized instructions to use personal information and report willingness to delegate the task to an AI agent.
- Each participant experienced six different tasks involving different attacks.
- Participants were instructed to visit a specified website, complete a task using provided personal information, and protect that information as they would their own.
- After completing each task, participants uploaded an automatically recorded interaction file and rated their willingness to delegate the task to an AI agent on a 7-point Likert scale.