Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents

1 Introduction

• GUI agents interpret natural-language commands and multimodal interfaces to execute dynamic actions, extending automation beyond fixed scripts and supporting everyday accessibility needs.They can automate data retrieval, form filling, price comparisons, and email responses, including through assistive technologies.Source: §1 · pp. 1–2

1.1 Privacy and Security Risks in GUI Agents

• The paper organizes GUI-agent privacy and security risks into amplified data leaks, diminished user control, and insufficient guardrails.These risks arise from sensitive-data access, autonomous processing, and susceptibility to adversarial manipulation.Source: §1.1 · pp. 2–3
• Frequent automated interactions with third-party services can expose sensitive information repeatedly and create persistent behavioral profiles vulnerable to misuse or unauthorized inference.Examples include medical searches revealing health interests and repeated flight-price checks broadcasting location data.Source: §1.1 · p. 2
• Insufficient safeguards can let GUI agents disclose sensitive information to phishing sites or malicious injected fields while continuing tasks unaware of the breach.The examples cover both screenshot-based and structured-file-processing agents.Source: §1.1 · p. 3

1.2 Challenges in Evaluations

• Existing evaluations emphasize effectiveness and efficiency, while privacy concerns remain underexplored and may conflict with helpfulness.PrivacyLens [37] is cited as showing that lower leakage rates can coincide with worse helpfulness.Source: §1.2 · p. 3
• The proposed framework calls for human-centered privacy and security assessment, privacy measures in agent development, and in-context consent to improve user awareness.Because privacy judgments depend on context and informed preferences can impose cognitive burdens, evaluation should account for individual risk-reward considerations and oversight challenges.Source: §1.2 · p. 3

2 GUI Agents vs. Traditional GUI Automation

• Traditional tools such as Selenium, AutoIt, and Robot Framework execute scripted interaction sequences but require reconfiguration when interfaces or scenarios change.Their fixed workflows suit repetitive tasks but lack adaptability.Source: §2 · p. 3
• GUI agents use multimodal AI, dynamic reasoning, and semantic or symbolic methods to adapt to layout, content, and element changes without explicit reprogramming.The Test-Agent framework is cited as enabling adaptation to new UI configurations.Source: §2 · pp. 3–4
• Compared with rule-based automation, GUI agents dynamically generate data-processing strategies, increasing uncertainty around sensitive-data access, retention, and external transmission.Risks include exposure of passwords, financial data, personal messages, and interaction logs.Source: §2 · p. 4
• Real-time interpretation makes GUI agents susceptible to phishing, UI dark patterns, prompt injections, and other manipulated inputs that can trigger disclosure or unintended actions.These threats arise because agents process changing user-interface content rather than following static scripts.Source: §2 · p. 4

3 GUI Agents as a Specialized Class of LLM-powered Autonomous Agents

• GUI agents differ from fully autonomous LLM agents by integrating sequential automation with user-interactive workflows, explainable action steps, and intervention opportunities.General autonomous agents more often generate and execute multi-step plans without user validation.Source: §3 · p. 4
• By operating in both text and action spaces, GUI agents translate user intent into real-world clicks, entries, and scrolling, reducing interaction friction while limiting reflection and correction.This expanded action space can amplify privacy and security risks when unintended actions are difficult to detect.Source: §3 · p. 4
• GUI agents may access sensitive on-screen information, but their step-by-step execution also enables users to assess and mitigate privacy risks in context.This creates a distinctive balance between automation and active privacy-aware oversight.Source: §3 · p. 4

4 Evaluation Metrics of GUI Agents

• GUI-agent evaluation is organized around effectiveness, efficiency, and safety, measuring objective achievement, speed and resource use, and avoidance of unintended or noncompliant actions.The three areas provide the paper’s overall metric taxonomy.Source: §4 · p. 4

4.1 Effectiveness

• An 88% task completion rate was reported in structured environments, with decreased performance in unstructured workflows, underscoring adaptability challenges.Task Completion Rate, Success Rate, and Task Progress capture full completion, unaided completion, and partial progress.Source: §4.1.1 · p. 5

4.1.1 Task-wise metrics.

• Step-wise metrics measure correct action execution, errors, and generalization across UI environments, with macro-averaging used to weight tasks proportionally.Step Success Rate and Adaptability expose failure points that task-level completion can conceal.Source: §4.1.2 · p. 5

4.2 Efficiency

• Speed is assessed through time cost and step cost, where fewer execution steps generally correspond to lower latency and greater responsiveness.These measures characterize how quickly an agent completes an instruction.Source: §4.2.1 · p. 5

4.2.1 Speed.

• Resource efficiency measures internal computational consumption and external costs such as LLM calls, balancing deployment feasibility with reliable performance.Reducing unnecessary API interactions is presented as one way to improve cost-effectiveness.Source: §4.2.2 · p. 5

4.3 Safety

• Safety evaluation covers safeguards, policy-compliant completion, and risk ratio, with confirmation mechanisms intended to prevent harmful critical actions.Higher safeguard rates and lower risk ratios indicate stronger protective reliability within the described metrics.Source: §4.3 · p. 5

• Privacy evaluation for GUI agents lacks systematic evaluation frameworks and benchmarks, despite studies examining web agents’ resistance to malicious attacks.Liao et al. [28] proposed environmental injection attacks to test whether web agents protect personally identifiable information.Source: §5 · p. 5
• Existing privacy research primarily evaluates model vulnerabilities through membership inference, data extraction, prompt leakage, jailbreak, and privacy-audit techniques.These approaches assess model-level privacy risks and privacy reasoning, rather than GUI-agent behavior in interactive environments.Source: §5 · p. 5
• Agent-level work has begun formalizing security attacks and privacy leakage in agent actions, but broad security benchmarks do not specifically focus on privacy.Shao et al. [37] found that agents’ answers to privacy questions can diverge from their privacy-related behavior during instruction execution.Source: §5 · pp. 5–6
• Model-level privacy evaluations alone are insufficient because GUI agents combine textual commands with visual interfaces that expose them to additional threats.Beyond adversarial jailbreaking, GUI agents may be manipulated by misleading UI elements, nudges, or obfuscated privacy settings, including environmental injections that humans may not detect.Source: §5 · p. 6

6 Human-Centered Evaluation for GUI Agents

• Human-centered evaluation supplements automated assessment because LLMs can be vulnerable when making ethical or privacy judgments.The paper distinguishes human oversight from user-engaged algorithm auditing as two existing approaches.Source: §6 · p. 6, §6.1 · p. 6

6.1 Human Oversight and Auditing

• Human oversight can combine real-time monitoring, user confirmation of significant actions, and post-auditing to support safety and privacy.Operator’s Watch Mode, User Confirmations, and Detection Pipeline illustrate these mechanisms.Source: §6.1 · p. 6
• User-engaged algorithm auditing involves end-users in assessing safety, legality, and ethical compliance through real-time review or post-hoc verification.Real-time auditing examines outputs during daily tasks, whereas post-hoc auditing supports verification of past or simulated examples at scale [38].Source: §6.1 · p. 6
• GUI agents make human-centered evaluation harder because multimodality, system complexity, increased agency, and invisible data transmission create knowledge, mental-model, trust, privacy-awareness, and cognitive-burden challenges.These factors complicate effective oversight and require reconsideration of evaluation goals.Source: §6.1 · p. 6

6.2 Knowledge Barriers and Mental Model Challenges for Human Evaluators

• Effective oversight requires technical and domain knowledge, yet GUI agents’ perception interfaces and hidden data flows make accurate mental models difficult to develop.Screen-visual, HTML/DOM, and accessibility-based interfaces expose different risks, while environmental injections can manipulate agent perception invisibly [28] [32].Source: §6.2 · p. 6
• End-users face greater difficulty than trained evaluators in understanding GUI agents, anticipating privacy risks, and auditing actions across integrated databases, applications, and services.Their increasingly important oversight role therefore intensifies the knowledge and mental-model challenge.Source: §6.2 · pp. 6–7

6.3 Overtrust, Lack of Privacy Awareness, and Increased Cognition Burden in Evaluation

• Overtrust and limited privacy awareness can weaken oversight, while multimodal action sequences can overload evaluators and hinder careful, consistent review.Contextual privacy norms increased users’ cognitive effort and awareness of privacy risks, whereas merely observing agent actions often did not reveal leakage [57].Source: §6.3 · p. 7

6.4 Rethinking the Evaluation Goals

• Evaluation should not target users’ actual privacy behavior alone, because behavior may conflict with informed preferences and the privacy interests of other affected parties.The paper advocates assessing privacy implications holistically across individuals, interpersonal relationships, and broader societal contexts [8] [15] [57].Source: §6.4 · p. 7

7 Call for Actions

• Trustworthy deployment requires coordinated actions spanning human-centered risk assessment, privacy-aware agent creation, and user awareness through in-context consent.The proposed actions address the distinct oversight and privacy challenges of GUI agents.Source: §7 · p. 7

7.1 Human-Centered Evaluation for Privacy Risk Assessment

• Human-centered evaluations should assess privacy risks across UI perception, intent generation, and action execution while measuring unintended exposure, trust, and satisfaction.In-context user oversight is needed because GUI agents have invisible data flows, multimodal outputs, and users with incomplete mental models.Source: §7.1 · pp. 7–8

7.2 Enhancing Users’ Privacy Awareness with In-Context Consent

• Explicit warnings, contextualized privacy explanations, structured consent, and configurable settings should help users recognize risks and retain control over sensitive actions.Consent should precede privacy-sensitive operations such as sending emails or conducting transactions.Source: §7.2 · p. 8

7.3 Integrating Privacy Measures into Agent Creation

• Privacy safeguards should span prompt-based and training-based development through consent requirements, limited retention, privacy-focused data, breach-preventing fine-tuning, and reward for protective behavior.These measures are intended to make privacy a core design principle rather than an afterthought.Source: §7.3 · p. 8