PaperlayerView exact revision on arXiv

Source-linked AI summary

Mobile GUI Agents under Real-world Threats: Are We There Yet?

Guohong Liu, Jialei Ye, Jiacheng Liu, Yuanchun Li, Wei Liu, Pengzhi Gao, Jian Luan, Yunxin Liu

arXiv:2507.04227v2cs.CRcs.AI

TL;DR

Mobile GUI agents need validation against deceptive third-party app content before real-world deployment. AgentHazard benchmarks this threat across dynamic and static settings, finding average misleading rates of 42.0% and 36.1%, respectively.

  • Problem

    Whether mobile GUI agents maintain performance under threats from uncontrolled third-party app content remains insufficiently validated for real-world deployment.

  • Method

    AgentHazard injects controlled adversarial content into Android applications and evaluates agents through 122 dynamic tasks plus a static dataset of over 3,000 commercial-app scenarios.

  • Results

    Average misleading rates were 42.0% in the dynamic environment and 36.1% in the static environment across evaluated open-source and commercial agents.

  • Takeaways & Limitations

    The findings indicate that improving deceptive-content detection and visual robustness is important for mobile GUI agent safety.

  • Takeaways & Limitations

    The benchmark covers a limited set of applications and actions and evaluates only misleading clicks and misleading terminations.

Abstract

from arXiv · show

Recent years have witnessed a rapid development of mobile GUI agents powered by large language models (LLMs), which can autonomously execute diverse device-control tasks based on natural language instructions. The increasing accuracy of these agents on standard benchmarks has raised expectations for large-scale real-world deployment, and there are already several commercial agents released and used by early adopters. However, are we really ready for GUI agents integrated into our daily devices as system building blocks? We argue that an important pre-deployment validation is missing to examine whether the agents can maintain their performance under real-world threats. Specifically, unlike existing common benchmarks that are based on simple static app contents (they have to do so to ensure environment consistency between different tests), real-world apps are filled with contents from untrustworthy third parties, such as advertisement emails, user-generated posts and medias, etc. ... To this end, we introduce a scalable app content instrumentation framework to enable flexible and targeted content modifications within existing applications. Leveraging this framework, we create a test suite comprising both a dynamic task execution environment and a static dataset of challenging GUI states. The dynamic environment encompasses 122 reproducible tasks, and the static dataset consists of over 3,000 scenarios constructed from commercial apps. We perform experiments on both open-source and commercial GUI agents. Our findings reveal that all examined agents can be significantly degraded due to third-party contents, with an average misleading rate of 42.0% and 36.1% in dynamic and static environments respectively. The framework and benchmark has been released at https://agenthazard.github.io.

1 Introduction

The paper argues that mobile GUI agents require validation against deceptive third-party in-app content, which existing static benchmarks and unrealistic attack models fail to represent. It introduces AgentHazard and a two-mode benchmark, finding substantial misleading rates across open-source and commercial agents.

  • Motivation and Threats: Real-world third-party content can mislead GUI agents into incorrect actions, potentially compromising user privacy or causing financial loss.Examples include social-media posts, e-commerce listings, emails, and messages; crafted content can trigger highly sensitive actions such as clearing user data.
  • Threat Model: Attackers control only unverified in-app content from unprivileged third parties, without access to application code, system UI, or the agent’s hidden state.Relevant content channels include posts, images, messages, and file names that enter the agent’s observation space.
  • AgentHazard Framework: AgentHazard intercepts and modifies Android UI state information in real time, enabling controlled injection of adversarial content into existing applications without root access.The framework targets content regions legitimately controlled by third parties and uses GUI hijacking and attack modules.
  • Benchmark: The benchmark combines 122 dynamic end-to-end tasks with a static dataset of more than 3,000 GUI-state attack scenarios for live and offline evaluation.The dynamic environment measures task success and misleading actions, while the static dataset pairs GUI states with adversarial content and detection rules.
  • Findings: 42.0% and 36.1% average misleading rates were observed in dynamic and static environments, respectively, using adversarial information averaging only 10 tokens.Experiments covered open-source and commercial agents across different architectures, sizes, and modalities.

2 Related Work

GUI agents now operate across web and mobile interfaces, while prior benchmarks and security studies leave mobile robustness under realistic, unprivileged third-party threats insufficiently understood. This work addresses that gap with a mobile-specific threat model, reproducible Android instrumentation without root or app modification, and empirical evaluation across agent architectures, modalities, and LLM backbones.

  • GUI Agents: GUI agents understand graphical interfaces and execute user-like operations such as clicking and typing across web and mobile applications.They establish interface understanding through multiple modalities, including screenshots and textual interface data.
  • GUI Agents: Commercial GUI agents have emerged, but UI-TARS-1.5 is identified as the only commercially available mobile agent evaluated in this work.Other commercial systems are desktop-only or remain in preview or pre-release stages, while the remaining evaluated agents are open-source frameworks powered by commercial LLMs.
  • GUI Agent Benchmarks: GUI-agent benchmarks comprise static and dynamic categories, with static benchmarks using predefined screenshots or textual interface data to assess comprehension and element localization.Static benchmarks support efficient evaluation, while the supplied passage does not provide further detail about dynamic benchmarks.
  • Security and Robustness of GUI Agents: Prior security research examines prompt injection, jailbreaking, backdoors, and adversarial attacks, but most GUI-agent attacks target web environments through HTML modification or pop-up injection.Mobile platforms impose stricter controls over privacy, permissions, and third-party content access, making several web attacks largely infeasible for unprivileged third parties.
  • Security and Robustness of GUI Agents: Mobile agents nevertheless interact with legitimately published information from untrusted third-party sources that can be arbitrarily modified and controlled.Examples include social-media posts and product descriptions in e-commerce applications.
  • Security and Robustness of GUI Agents: This work fills the mobile robustness gap through legitimate-content threats, reproducible runtime instrumentation on real Android apps without root or app modification, and a cross-agent empirical study.The study spans agent architectures, modalities, and LLM backbones.

3 Threat Model

The threat model considers mobile GUI agents operating in real-world Android environments where untrusted third-party content can mislead agents through legitimate application channels.

  • Mobile GUI task execution involves users, agents, applications, and third-party content sources such as product listings, social media posts, and advertisements.
  • The threat model assumes other components remain secure and reliable while attackers control misleading information published through legitimate channels.
  • Attackers cannot modify application resources such as APKs or system-controlled components.

4 Our Analysis Framework: AgentHazard

AgentHazard is a framework for systematically evaluating mobile GUI agents against reproducible threats from unprivileged third-party app content. It combines configurable app-content instrumentation with dynamic interactive and static state-rules testbeds for scalable robustness analysis.

  • Framework Overview: AgentHazard combines an app content instrumentation tool with dynamic and static evaluation components.The framework targets limited threat diversity, availability, and reproducibility in real-world applications.
  • App Content Instrumentation: The instrumentation tool injects adversarial content into UI trees and screenshots in real time using configurable GUI hijacking and attack modules.Real-time injection supports consistent, controllable scenarios despite dynamic recommendations driven by network activity, user history, and time.
  • App Content Instrumentation: Simple configuration editing enables scalable, flexible, stable attack scenarios that remain unaffected by content refreshing or data loading.The tool can systematically vary tailored scenarios across applications, tasks, and content types while closely mimicking original UI elements.
  • Dynamic Interactive Environment: 122 reproducible tasks across 12 diverse applications form the dynamic environment, which extends AndroidWorld with adversarial content injection and misleading-action monitoring.Each task uses an agent-environment interaction loop with task-success rules and attack-misleading rules; success and misleading rate are treated as independent metrics.
  • Static State-Rules Dataset: Over 3,000 attack scenarios populate the static dataset, whose samples pair screenshots and UI element trees with attack and success rules from commercial applications.The static pipeline complements slower, more confounded dynamic evaluation with scalable single-state analysis and includes benign and adversarial state-rules pairs.

5 Analysis Results

The analysis finds that visual inputs increase vulnerability to deceptive third-party content, attacks transfer across LLM backbones, and adversarial training offers only limited protection. Across evaluations, agents remain substantially susceptible, although GUI-specific training and stronger frontier models improve robustness.

  • Overall vulnerability: 42.0% average misleading rate shows that mobile GUI agents are highly susceptible to deceptive third-party content in the dynamic environment.Six agents were evaluated, including five open-source research frameworks and the commercial UI-TARS-1.5.
  • Overall vulnerability: 30% approximate task-success-rate decreases affect M3A@4o and UGround@4o under adversarial conditions.Agents with lower baseline performance show greater resilience in ΔSR, partly because they have less room for further degradation.
  • Modality effects: 67.9% versus 58.0% benign success rates show that vision can improve performance, yet GPT-5’s misleading rate rises from 11.5% to 24.5% across text to multi-modal settings.The analysis reports that multi-modal agents generally have the weakest defense against deceptive content, while GPT-4o-mini’s misleading rate exceeds 70%.
  • Backbone transferability: 17.5% average misleading rate for GPT-5 is substantially lower than GPT-4o’s 53.9% and GPT-4o-mini’s 62.3%, but vulnerability persists across LLM backbones.Most evaluated LLMs exceed 30% average misleading rates, indicating that inherent LLM capabilities alone are insufficient for proactively identifying adversarial content.
  • Attack analysis: 83.3% misleading rate makes Mixed Actions more effective than any single-type attack, whereas repetitive misleading elements reduce click-action misleading rates from 50.0% to 47.2%.The results suggest that repetition may trigger skepticism, while defenses must account for diverse mixed-action attacks.
  • Training and robustness: 18.5% performance drop under attack for adversarial fine-tuning is smaller than the benign fine-tuned model’s 37.1% degradation, while benign fine-tuning reaches 74.6% MR.Adversarial fine-tuning also achieves a 24.5% adversarial success rate, whereas GUI-specific training improves robustness in UI-TARS-1.5.

6 Case Study

The case study shows that deceptive third-party content can drive a mobile GUI agent to perform irreversible, high-privilege actions without user confirmation, exposing gaps in source-trust reasoning and safety safeguards.

  • Case study: An injected message claiming the task was infeasible misled Aria UI@4o into concluding that the Broccoli app was corrupted.The agent then chose to repair the app rather than terminate or continue the original task.
  • Case study: The agent navigated to system settings and cleared all app data, deleting every recipe instead of the specified target.This irreversible action caused the task to fail and resulted in user-generated content and cache loss.
  • Identification: The agent accepted deceptive content without scrutinizing its authenticity or provenance, revealing insufficient source-trust reasoning.Robust agents should treat third-party application content with more skepticism than operating-system or user messages.
  • Handling: The agent performed irreversible data deletion without requesting user confirmation or authorization, exposing a critical safeguard gap.Explicit consent before potentially destructive operations would provide a safety barrier against untrustworthy information.
  • Case study: The case study identifies two robustness priorities: identifying deceptive content and handling it safely.The example is an extreme tail-risk case rather than a typical attack outcome.

7 Discussion

The discussion identifies coverage and modality limitations in the benchmark while recommending stronger deceptive-content detection, visual robustness, and agent-specific operating-system controls for safer deployment.

  • Limitations: The framework cannot modify images within UI elements, leaving an additional potential attack vector in real-world scenarios.
  • Limitations: The benchmark covers a limited set of applications and actions, so it may not capture the diversity of mobile apps and agent action spaces.
  • Limitations: The evaluation focuses on misleading clicks and misleading terminations as representative attack types for understanding agent vulnerabilities.
  • Lessons: Improving deceptive-content detection and robustness in visual understanding is critical because LLMs show substantially higher sensitivity to adversarial visual information.
  • Lessons: Future agent-aware operating systems should establish system-level access controls and permission restrictions tailored to autonomous agents.

8 Conclusion … A.4.2 Dynamic Environment. •

AgentHazard is a scalable framework and benchmark for testing mobile GUI agents against targeted third-party app-content modifications. The released artifact supports reproducing dynamic and static evaluations, with dynamic testing requiring an Android Virtual Device and static testing requiring no virtual machine.

  • 8 Conclusion: AgentHazard enables flexible, targeted content modifications within existing Android applications to study mobile GUI-agent readiness under third-party-content threats.The framework addresses the challenge that real-world app contents are significantly skewed toward benign content.
  • Appendix: The artifact reproduces the experimental results reported in Sections 5.1 and 5.2.
  • A.1 Artifact Abstract: The artifact checklist identifies gpt-4o-2024-11-20∗, gpt-4o-mini-2024-07-18∗, Qwen/Qwen-2.5-VL-7B-Instruct†, osunlp/UGround-V1-7B†, and Aria-UI/Aria-UI-base† as models.The symbols indicate that ∗ requires an API, while † requires training or deployment.
  • A.2 Artifact check-list (meta-information): The artifact includes evaluation tasks, an attacker-app APK, mitigation training data and scripts, Success Rate and Misleading Rate metrics, and approximately 25 GB and 5 GB for dynamic and static environments.
  • A.3 Description: The runtime environment uses Ubuntu 22.04, an A100 80G GPU training and inference server, Android API Level 33, and Python 3.11+.
  • A.3.2 Hardware dependencies.: Dynamic evaluation runs on an Ubuntu Desktop WorkStation with 2x24GB 3090 GPUs, while mitigation training uses 4x80GB A100 GPUs.The dynamic setup also requires Android SDK and an Android Virtual Device; the listed workstation is not a minimum requirement.
  • A.4 Installation: Installation requires downloading and unzipping data/exp.7z, which produces static/ and dynamic/ folders.
  • A.4.1 Static Environment. •: Static setup moves data/static into the AgentHazard project and installs dependencies with uv sync --no-dev or pip install -r requirements.txt.Dynamic setup additionally requires Android World configuration, Android SDK and API keys, attacker-app installation on an AVD, model serving with vllm, and a virtual-machine snapshot.

A.4.3 Mitigation Training. •

Mitigation training begins by creating a working directory, setting up a virtual environment, and installing the required training dependencies.

  • Environment setup: Create and enter the mitigation-training working directory with mkdir -p code/mitigation-training and cd code/mitigation-training.Set up the virtual environment according to the chosen environment-management workflow.
  • Dependency installation: Install ms-swift with all extras and upgrade it using pip install 'ms-swift[all]' -U.
  • Dependency installation: Install DeepSpeed and Flash-Attention with pip install deepspeed and pip install flash-attn --no-build-isolation.

A.5 Experiment workflow … A.6 Evaluation and expected results

The evaluation workflow provides scripts for static, dynamic, and mitigation-training experiments, with results saved for metric calculation and resumable execution. Static evaluations cover baselines and attacks, dynamic evaluations support configurable tasks and misleading-action interruption, and mitigation training produces deployable checkpoints for evaluation.

  • A.5 Experiment workflow: The workflow activates the project environment, configures OpenAI-compatible API credentials, and runs evaluations through the provided CLI or scripts.
  • A.5.1 Static Environment.: Static evaluation uses executable scripts for baseline and attack conditions.
  • A.5.1 Static Environment.: Static results are saved under static_results, and rerunning after network errors resumes from saved data.
  • A.5.2 Dynamic Environment.: Dynamic evaluation runs Android World tasks with configurable agents and attack settings, including an option to stop when a misleading action is detected.
  • A.5.3 Mitigation Training.: Mitigation training unpacks benign and adversarial datasets, runs the corresponding training scripts, merges LoRA weights, deploys the model, and evaluates it with or without attacks.
  • A.6 Evaluation and expected results: All experiments save results as parquet files that can be loaded with pandas or other Python libraries for metric calculation and validation.

B Supplementary Details of Benchmark Construction

The benchmark uses diverse applications and separates controllable, reproducible dynamic tasks from commercially sourced static scenarios. Its misleading third-party content is placed only in legitimate third-party-controlled fields, yet can disrupt execution and threaten privacy or financial security.

  • Application coverage: The benchmark construction uses diverse applications to broaden tasks and support evaluation validity, with detailed dynamic and static application lists provided in Table 4.The application list is presented as Table 4 in the supplementary details.
  • Dynamic task construction: Dynamic tasks use 12 open-source applications across note-taking, dining, finance, planning, music, scheduling, and contacts to ensure controllability and reproducibility.Open-source apps avoid recommendation systems and real-time content updates that could introduce external influences.
  • Misleading-content placement: Misleading information appears only in legitimate third-party-controlled fields, including post content, titles, product names, and contact messages.These concise phrases are designed to simulate third-party attacks within areas where third parties legitimately control content.
  • Threat impact: Such misleading content can alter or disrupt task-execution trajectories and may compromise user privacy and financial security.The passage identifies these consequences as possible outcomes in certain scenarios.

C Examples of designed tasks with attack content injection · D Prompts designed to generate adversarial content

The paper instantiates dynamic GUI attacks through configurable overlays targeting precise application elements, and generates scalable adversarial content with LLM-guided prompts. LLM-generated content is highly task-relevant, performs nearly identically to manually crafted content in dynamic evaluation, and enables low-effort synthesis of realistic attack interfaces.

  • C Examples of designed tasks with attack content injection: A JSON configuration demonstrates a concrete dynamic attack scenario for the MarkorMergeNotes task.The example targets Markor’s main activity and specifies the attack configuration pattern.
  • C Examples of designed tasks with attack content injection: The Markor attack injects deceptive text into a precisely targeted screen region identified through UiAutomator resource IDs.The injected message claims that the app is broken and merging the note is infeasible.
  • C Examples of designed tasks with attack content injection: Bounding-box and relative-index parameters provide flexible control over whether the injected content overwrites a specified region or targets an indexed element.A non-zero bounding box forcibly overwrites the target region, while a relative index selects an element from overlayStyles.
  • D Prompts designed to generate adversarial content: For static state-rule construction, prompts condition LLM generation on the app state, original task, and controllable target region.The generated content is used to create attack rules containing misleading actions such as clicking the target region or terminating the task.
  • D Prompts designed to generate adversarial content: The prompt uses stepwise refinement to produce task-relevant adversarial strings, including framing misleading actions as essential task steps or claiming the task is complete or infeasible.This optimization strategy is designed to guide scalable generation of high-quality adversarial content.
  • D Prompts designed to generate adversarial content: LLMs effectively generate adversarial information that is highly relevant to specified tasks and can be rendered directly onto screenshots.The paper illustrates this process with example strings in Table 5 and rendered outputs in Figure 10.
  • D Prompts designed to generate adversarial content: LLM-generated adversarial information has a misleading rate nearly identical to manually crafted content in dynamic evaluation.Rendering generated strings with the original text’s colors and positions enables many realistic simulated attack interfaces with minimal human effort.

E Stealthiness

The framework-based attacks are substantially stealthier than popup-based attacks because their modified content blends naturally with the original interface, reducing detector visibility. A GPT-4o-mini-based screenshot detector identified 98.3% of popup attacks but only 37.9% of framework-based attacks.

  • Detection method: A GPT-4o-mini-based detector analyzes screenshots to identify suspicious or misleading interface content.The detector judges abnormal information based on the screenshot and outputs whether it is malicious with a reason.
  • Evaluation design: The evaluation includes screenshots from unmodified interfaces as a control group to account for potential model bias.The control group is used alongside popup-based and framework-based attacks in the dynamic evaluation environment.
  • Results: 98.3% detection rate was achieved for popup-based attacks, with most floating windows identified as suspicious.The result is attributed to floating windows’ distinct visual characteristics and positioning.
  • Results: 37.9% detection rate was achieved for framework-based attacks because modified content appeared natural and consistent with the original interface design.This lower detection rate demonstrates the framework’s stealthiness advantage over popup-based attacks.
Loading 2507.04227v2…