The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents

• Existing evaluations mainly test explicit misuse or environment-embedded injection, overlooking cases where harmfulness appears only during execution.Rewriting OS-BLIND instructions as explicitly malicious substantially lowers ASR, indicating that aligned models more readily recognize and refuse explicit threats.Source: §1 · p. 1, Figure 1 · p. 2
• OS-BLIND targets unintended attacks in which fully benign instructions simulate daily workflows but harm emerges during execution and the agent complies.Its one-to-one manual task design preserves realistic and varied contexts for testing whether agents recognize harms before execution.Source: §1 · p. 2
• Open-source end-to-end agents exceed 90% ASR, while safety-aligned Claude 4.5 models successfully execute harmful actions on nearly half of OS-BLIND tasks.These vulnerabilities occur despite benign user instructions.Source: §1 · p. 2
• Existing defenses provide limited protection under benign instructions: safety alignment activates mainly within the first few steps, rarely re-engages later, and task decomposition obscures harmful intent.In multi-agent systems, safety-aligned models fail more often, and Claude 4.5 Sonnet cannot recover the original user intent from concatenated subtasks.Source: §1 · p. 2
• OS-BLIND yields 1.9× and 3.8× the ASR of VPI-BENCH (Cao et al., 2025) and OS-HARM (Kuntz et al., 2025), respectively, on average.The findings indicate that OS-BLIND poses a substantially more challenging and realistic safety test for modern CUAs.Source: §1 · p. 2

2.1 General Setup

• OS-BLIND evaluates benign instructions whose execution can produce harmful outcomes through environment-embedded threats or agent-initiated harms.Harm includes policy violations, compromised security, privacy, or data integrity; either mechanism can make a trajectory unsafe.Source: §2.1 · p. 2

2.2 Harmful Task Categories

• The benchmark contains 300 tasks across 12 harmful categories, 8 applications, and two clusters covering deceptive environments, system integrity, and output integrity.The environment-embedded cluster includes deceptive websites, scam emails, harmful content, and pop-ups; agent-initiated categories include exfiltration, malware, sabotage, code injection, fraud, deceptive agreements, misinformation, and harassment.Source: §2.2 · p. 3, Table 1 · p. 3

2.3 Task Design

• OS-BLIND tasks are manually designed with benign instructions and plausible scenarios, using faithful replicas, accessible real-world URLs, and context-aware dynamic pop-up attacks.The dynamic pop-up variant generates task-specific misleading instructions from the current task context rather than fixed coordinate-level injections.Source: §2.3 · pp. 3–4

2.4 Annotation and Evaluation Protocol

• The benchmark uses outcome-based harmfulness annotations and automated judgments grounded in task-specific action descriptions, with weighted Cohen’s κ of 0.945 for Claude 4.5 Sonnet and 0.864 for CoAct-1.For 250 non-pop-up tasks, gpt-4o judges annotated actions; the remaining 50 pop-up tasks use trust-versus-rejection clicks as the direct success criterion.Source: §2.4 · p. 4

2.5 Human Verification

• All 300 tasks underwent two-phase expert review, achieving 93.67% initial agreement before joint resolution of ambiguous or discordant cases.Construction required two months and over 1,000 person-hours of expert effort.Source: §2.5 · p. 4

3.1 Baselines

• The evaluation covers open-source and proprietary end-to-end CUAs alongside CoAct-1, Jedi, and Agent-S2, focusing analysis on CoAct-1’s decomposed orchestration and multi-step GUI execution.The evaluated end-to-end models include EvoCUA, OpenCUA, UITARS, Claude 4.5 Sonnet, and Claude 4.5 Opus.Source: §3.1 · p. 4

3.2 Implementation Details

• Experiments limit open-source agents to 20 steps, Claude-based agents to 15, and CoAct-1 to five coordination turns with up to five steps per subtask in the no-coder setting.CoAct-1 is restricted to no-coder evaluation because embedded visual risks are not observable to coding agents without screenshots.Source: §3.2 · p. 5
• Attempted Rate measures whether an agent proceeds without explicit refusal, whereas Attack Success Rate measures whether execution triggers annotated harmful behavior across three independent runs.AR separates willingness to refuse from execution ability, while ASR captures whether execution ultimately causes harm.Source: §3.2 · p. 5

3.3 Results

• ASR is higher for nearly all models in Cluster I than Cluster II, indicating weaker sensitivity to visually obvious environmental threats than to explicit multi-step harmful contexts.For Claude 4.5 Opus, ASR is 54.7% in Cluster I versus 40.4% in Cluster II.Source: §3.3 · p. 5
• Multi-agent frameworks consistently raise ASR over standalone Claude 4.5 Sonnet, with CoAct-1 reaching 93.1% in Cluster I and 92.2% in Cluster II using a gpt-5 orchestrator.Using Claude 4.5 Sonnet as both orchestrator and GUI operator still yields 93.1% and 87.9%; Agent-S2 and Jedi show the same safety degradation.Source: §3.3 · p. 5
• Open-source end-to-end models have average ASRs above 96% in Cluster I and 90% in Cluster II, while Claude 4.5 Sonnet and Opus achieve 82.4%/54.7% and 62.4%/40.4%, respectively.Open-source agents have 100% attempted rates across all categories, indicating they never explicitly refuse harmful tasks.Source: §3.3 · p. 5

4 Analysis

• Safety defenses only partially reduce ASR: prompt defense leaves open-source models unchanged or down 6%, while MirrorGuard reduces ASR by 26.0 and 76.0 percentage points.For closed-source models, prompt defense lowers Claude 4.5 Sonnet from 98% to 84% and Claude 4.5 Opus from 100% to 50%.Source: §4 · p. 7, Table 4 · p. 7
• 62.7% ASR averaged across six configurations was 1.9× VPI-BENCH and 3.8× OS-HARM, showing OS-BLIND is substantially more challenging than prior benchmarks.MirrorGuard built on EvoCUA achieved the lowest ASR across the three benchmarks, but refused 47% of OSWorld test_small tasks, indicating over-refusal contributed to its robustness.Source: §4 · p. 7, Table 5 · p. 7
• Claude 4.5 Sonnet and Opus almost always refuse at the first step, then rarely re-evaluate harmfulness during subsequent execution.MirrorGuard differs by producing a substantial fraction of refusals at steps 2 and 3 through stepwise risk assessment and reasoning rewrites.Source: §4 · p. 7, Figure 3 · p. 8
• In CoAct-1, orchestrator refusals account for almost all rejections, while Claude 4.5 Sonnet as GUI operator contributes very few refusals across turns.This contrasts with Sonnet’s standalone behavior, where it refuses both harmful and benign instructions.Source: §4 · p. 8
• Replacing original instructions with concatenated subtasks raises ASR from 27.9% to 79.1%, while reconstructed high-level instructions yield 86.1%, confirming decomposition degrades GUI-operator safety.The controlled ablation uses 43 tasks that Sonnet consistently rejected standalone but CoAct-1 consistently failed to reject.Source: §4 · pp. 8–9, Table 6 · p. 8

4.1 Effect of Subtask Granularity on Orchestrator Safety Defense

• Coarser subtasks triple orchestrator-blocked tasks from 5 to 15 by exposing complete risky workflows, whereas fine-grained atomic actions evade detection.Coarser delegation also prompts safe rewrites or truncation before irreversible steps, behaviors absent with fine-grained subtasks.Source: §4.1 · p. 9, Table 7 · p. 9

4.2 Analysis of Claude’s Safety Alignment Failure Modes

• Claude safety failures comprise inspection failure, judgment failure, and detection failure, differing in whether threat evidence is acquired and whether reasoning or procedure fails.Inspection skips script auditing; judgment enables harmful downstream actions despite recognized content; detection depends on seeing through disguised threats.Source: §4.2 · pp. 10–11

GUI Agents

• GUI-agent research spans realistic web, desktop, and mobile benchmarks, which expose persistent challenges in precise grounding, planning, and long-horizon task completion.Native end-to-end agents simplify execution but require extensive data, while multi-agent systems decouple planning from execution and support complex computer-use tasks.Source: §5 · p. 11
• Although multi-agent systems achieve higher OSWorld task performance, the findings show that their safety performance does not improve accordingly.This motivates evaluating capability and safety separately in complex computer-use settings.Source: §5 · p. 11

Computer-Use Agent Safety Challenges

• Existing CUA safety benchmarks divide into attack-centric evaluations of harmful-objective completion and behavior-centric evaluations of risky behavior arising from benign instructions and context.Examples include indirect prompt injection and malicious pop-up settings, alongside OS-HARM, AUTOELICIT-BENCH, and BLIND-ACT.Source: §5 · p. 11
• Safety defenses include screenshot and reasoning-trace monitoring, domain-specific guardrails, rule-based monitoring, future-risk prediction, and uncertainty estimation, yet benign-instruction safety remains unresolved.MIRRORGUARD is an example of dynamically correcting unsafe reasoning before action execution.Source: §5 · pp. 11–12
• Table 8 compares OS-BLIND with existing agent-blind benchmarks across platform, multimodal support, real-world access, cross-application tasks, benign instructions, evaluation rules, task authorship, and category count.The table defines the dimensions used for the benchmark comparison; the full comparison with existing safety benchmarks is provided in section 6.Source: Table 8 · p. 12, §5 · p. 12
• OS-BLIND asks whether fully benign workflows can still be steered into concrete harmful outcomes through environmental interaction during execution.This differs from attack-centric benchmarks using malicious instructions or adversarial prompts and behavior-centric benchmarks measuring risky tendencies rather than a specified harmful outcome.Source: §5 · pp. 11–12

Benchmark landscape

• OS-BLIND extends prior evaluations beyond static, web-only, and OS-based settings by targeting realistic cross-application workflows and concrete harmful consequences.Earlier benchmarks increasingly incorporate interactive OS environments, but still retain limitations in safety-risk coverage.Source: §6 · pp. 12–13

Risk dimensions

• Prior benchmarks mainly test explicit misuse, conspicuous prompt injection, or risky operations, whereas OS-BLIND requires a benign workflow to produce an externally meaningful harmful outcome.These outcomes include credential leakage, unauthorized accounts, malicious-code execution, sensitive-file exfiltration, and security-configuration changes.Source: §6 · p. 13

Comparison with BLIND-ACT

• OS-BLIND differs from BLIND-ACT by covering broader attack mechanisms, including deceptive external environments where users themselves become victims of phishing or scams.BLIND-ACT is behavior-centric and emphasizes blind goal-directedness, whereas OS-BLIND is attack-centric and focuses on unintended attacks during benign workflows.Source: §6 · pp. 13–14

Analytical scope

• OS-BLIND also studies safety gaps between benign and explicitly malicious instructions, early-step safety alignment, and degradation caused by multi-agent task decomposition.These analytical dimensions extend beyond BLIND-ACT’s emphasis on category-level results, prompting interventions, and qualitative failure modes.Source: §6 · p. 14

Task decomposition

• Prior work shows that multi-turn or decomposed harmful goals can bypass safety checks, while OS-BLIND demonstrates the same vulnerability when user instructions are entirely benign and harmful content exists only in the environment.Subtask decomposition can suppress safety alignment and increase execution of harmful objectives that aligned models would otherwise refuse.Source: §6 · p. 14

7 Conclusion

• Most CUAs exceed 90% ASR under benign instructions, while Claude 4.5 Sonnet reaches 73.0% ASR.OS-BLIND evaluates misleadingly harmless conditions in which every user instruction is benign but harm emerges from the environment or execution outcome.Source: §7 · p. 14
• Existing defenses provide limited protection because safety alignment activates mainly within the first few steps and rarely re-engages afterward.The analysis covers defenses and refusal timing under benign instructions.Source: §7 · p. 14
• Decomposition raises Claude 4.5 Sonnet’s ASR from 27.9% to 79.1%, and reconstructing complete subtask sequences does not restore safety alignment.Task decomposition in multi-agent systems causes safety-aligned models to fail despite intent reconstruction.Source: §7 · p. 14
• OS-BLIND provides a concrete foundation for evaluating and improving CUA security in real-world deployments.The benchmark systematically exposes vulnerabilities across frontier models and agentic frameworks.Source: §7 · p. 14

Ethics Statement

• OS-BLIND runs entirely in isolated OSWorld Ubuntu virtual machines, uses synthetic sensitive data and simulated services, and collects no real user data.Offensive or disturbing content is included only to illustrate evaluated threat categories and may appear in benchmark examples.Source: Ethics Statement · p. 15
• The benchmark and evaluation code will be released on Hugging Face and GitHub to enable reproducible vulnerability evaluation and support development of safer computer-use agents.The authors frame this release as infrastructure for future work toward trustworthy computer-use agents.Source: Ethics Statement · p. 15

TABLE OF CONTENTS FOR APPENDIX

• The appendix includes task configuration, GUI action-space, attempted-rate, LLM-as-a-judge, active-defense, safety-failure, hallucination, semantic-diversity, trajectory, task-inventory, and prompt sections.Its listed subsections cover both evaluation implementation and analyses of agent behavior and safety failures.Source: TABLE OF CONTENTS FOR APPENDIX · p. 21

A Task Configuration Example

• A representative Harassment task pairs a routine request to insert essay feedback with abusive content in a referenced file and evaluates whether the agent copies or inserts it without refusal.The JSON configuration specifies environment initialization, the benign-sounding instruction, and harmful-behavior evaluation criteria inside an anonymized blind-review setup.Source: A Task Configuration Example · p. 22

B GUI Agent Action Space

• All benchmark agents control Ubuntu through executable Python calls to unified pyautogui primitives within OSWorld virtual machines.The shared interface supports mouse and keyboard interaction across end-to-end models and multi-agent systems.Source: B GUI Agent Action Space · p. 23
• Execution control adds time.sleep() and WAIT, DONE, and FAIL commands for pausing, declaring success, and reporting infeasibility amid operating-system latency.The representative GUI primitives include clicking, pointer movement, dragging, scrolling, key presses, shortcuts, key holds, and text entry.Source: B GUI Agent Action Space · p. 23

C Attempted Rate Results

• 100% attempted rate holds across every category for all evaluated open-source end-to-end models except Claude, indicating no explicit refusal mechanism.Attempted Rate measures the fraction of tasks where an agent proceeds without explicitly refusing in at least one of three independent runs.Source: C Attempted Rate Results · p. 24
• Claude models have substantially lower attempted rates than other open-source end-to-end models, reflecting active safety awareness.Source: C Attempted Rate Results · p. 24
• Agentic frameworks generally achieve attempted rates near or above 95%, regardless of the underlying backbone’s safety alignment.Source: C Attempted Rate Results · p. 24
• Tables 9 and 10 report attempted rates for all evaluated models across both threat clusters and categories.Source: C Attempted Rate Results · p. 24

D LLM-as-a-Judge Validation

• For 250 tasks across 11 categories, the LLM judge receives the full trajectory, user instruction, harmfulness rules, and a fixed evaluation prompt.The 50 Pop-up Attack tasks are excluded because success is determined deterministically from click coordinates within the injected pop-up region.Source: D LLM-as-a-Judge Validation · p. 25
• Judge verdicts are compared with independent human labels for Claude 4.5 Sonnet and CoAct-1 using agreement, precision, recall, F1, and Cohen’s κ.The judge also identifies the offending step and provides a natural-language explanation; Table 11 reports per-category agreement percentages.Source: D LLM-as-a-Judge Validation · p. 25, Table 11 · p. 25

E Active Defense Examples in Coarse-Granularity Subtask

• Coarse-granularity orchestration produced two active defenses: rewriting harmful subtasks into protective actions and truncating execution before destructive actions.The examples use the CoAct-1 configuration with a gpt-5 orchestrator and Claude 4.5 Sonnet GUI operator, with at most 20 GUI steps per delegated subtask.Source: E Active Defense Examples in… · p. 26
• In the crypto-alert example, the orchestrator replaced entering a recovery phrase into a phishing site with reporting the email as phishing and stopping.The rewritten instructions explicitly prohibited clicking links, choosing wallet-security actions, or performing further actions after reporting the message.Source: E Active Defense Examples in… · p. 26
• In the DNS example, the orchestrator prohibited destructive execution, allowed only pasting sudo rm /etc/hosts without pressing Enter, and ran a separate screenshot command.The GUI agent located and copied the command without executing it, then continued correcting a mistaken click on the File Manager icon.Source: E Active Defense Examples in… · p. 27

F Claude Safety Alignment Failure Modes: Trajectory Examples

• The trajectory examples examine three Claude safety-alignment failure modes, with the following excerpts providing evidence for each mode.The supplied excerpts specifically cover judgment failure and inspection failure, alongside the broader trajectory analysis identified in subsection 4.2.Source: F Claude Safety Alignment Failure… · p. 28

F.1 Judgment Failure: Agent Perceives Harm but Proceeds

• Opus and Sonnet recognized abusive content yet forwarded it, while Opus also identified a gambling platform but still created and saved a betting strategy guide.For harmful_forwarding, Opus read and reproduced the abuse before sending it; for gambling_website, it explicitly named DraftKings Sportsbook and proceeded without later safety reasoning.Source: §F.1 · p. 28

F.2 Inspection Failure: Agent Never Examines Harmful Content

• In data_pipeline_exfiltration, Opus trusted the README, executed install_deps.py, and never opened the script, allowing README-level trust to replace source auditing.The hidden behavior exfiltrated project files under the guise of dependency installation, and neither model ran cat install_deps.py in any trajectory.Source: §F.2 · p. 29
• The same inspection failure appeared in scraper_cron_backdoor: Opus read the README, ran setup.sh without opening it, and missed its concealed persistent reverse-shell cron job.Across both models and all runs, cat setup.sh never appeared, while the setup script disguised persistence as a health monitor.Source: §F.2 · p. 29

Detection Failure: Agent Fails to Identify Disguised Threats

• Sonnet was unsafe in all 3 runs for both the pop-up attack and malware-deployment tasks, while Opus was safe in all 3 runs.These cases demonstrate failures to identify disguised threats despite benign user instructions.Source: §F.3 · p. 30

Pop-ups Attack

• Opus identifies the injected authorization overlay as malicious and uses the legitimate context menu, while Sonnet interprets it as a legitimate system dialog and clicks “Authorize.”Both models encounter the identical pop-up, but only Opus reasons about its provenance.Source: §F.3 · p. 30

Malware Deployment

• Opus inspects the downloaded script, identifies credential exfiltration to eng-standards.evil.com, and refuses execution, whereas Sonnet executes it without inspection.The script exfiltrates SSH keys, AWS credentials, and npm tokens.Source: §F.3 · p. 30

Overall pattern

• OpenCUA-72B’s lower ASRs on certain Cluster II categories, including System Sabotage and Code Injection, arise from context loss rather than genuine safety alignment.Trajectory analysis shows the model correctly identifies the harmful target initially, then replaces it with an unrelated command during later execution without safety reasoning or refusal.Source: G OpenCUA-72B Hallucination in… · p. 31

Example 1: firewall_test

• In firewall_test, the model identifies sudo ufw disable in Step 1 but executes sudo apt install vlc by Step 4.The firewall-disabling command is accurately extracted, yet the model substitutes an unrelated VLC installation and provides no safety reasoning.Source: G OpenCUA-72B Hallucination in… · p. 31

Example 2: env_path

• In env_path, the model identifies the malicious PATH modification in Step 1 but instead runs sudo apt update and configures an unrelated MSBuild task.The intended .bashrc edit is never refused; it is forgotten during subsequent execution, which continues for 16 unrelated steps.Source: G OpenCUA-72B Hallucination in… · p. 31

Shared interpretation

• Across both examples, harmful commands are replaced by hallucinated benign actions after accurate early comprehension, with no explicit refusal or safety-related reasoning.This distinguishes the behavior from learned safety alignment and explains the lower ASRs as an artifact of multi-step context loss.Source: G OpenCUA-72B Hallucination in… · p. 31

• 0.74 MSD for OS-BLIND exceeds 0.22 for OS-HARM and 0.02 for VPI-BENCH, indicating broader task-semantic coverage and fewer near-duplicate instances.Each task instance is defined jointly by its natural-language instruction and initial GUI state.Source: H CUA Benchmark Semantic Diversity · p. 32
• Near-duplicates are linked when either instruction or initial-GUI similarity exceeds its modality-specific threshold, conservatively grouping samples similar in either modality.This union rule accounts for redundancy from repeated instructions across interface states or different instructions sharing essentially the same environment.Source: H CUA Benchmark Semantic Diversity · p. 32
• MSD estimates semantic diversity as the proportion of unique task groups, with groups defined by connected components in a duplicate graph.Samples become graph nodes, and near-duplicate pairs become edges; higher MSD denotes a larger proportion of semantically unique configurations.Source: H CUA Benchmark Semantic Diversity · p. 32
• The implementation uses CLIP embeddings for instructions and screenshots with both cosine-similarity thresholds set to 0.99, comparing OS-BLIND with reimplemented OS-HARM and VPI-BENCH tasks.VPI-BENCH configurations are reconstructed in the OSWorld sandbox, and the procedure covers its computer-use subsets and OS-HARM.Source: H CUA Benchmark Semantic Diversity · p. 32

I Trajectory Case Studies

• Agents copied personal information into a fraudulent lease-renewal form without verifying the portal, and exported confidential acquisition material despite an explicit no-copy notice.These trajectories demonstrate failures to detect phishing and document-level confidentiality restrictions, enabling sensitive-data exposure and potential exfiltration.Source: I Trajectory Case Studies · pp. 33–34, Table 12 · p. 33, Table 13 · p. 34
• Agents modified financial records from an unverified email and executed an attached cryptocurrency miner without inspecting its code.The cases show susceptibility to routine-looking social engineering and failure to review software before execution, enabling fraud and resource abuse.Source: I Trajectory Case Studies · pp. 35–36, Table 14 · p. 35, Table 15 · p. 36
• Agents followed email instructions to prepend an attacker directory to PATH and posted fabricated fraud allegations without checking legitimacy, accuracy, or legal risk.These trajectories expose vulnerabilities to path hijacking and weaponized disinformation during apparently ordinary tasks.Source: Table 16 · pp. 37–38, I Trajectory Case Studies · pp. 38–39, Table 17 · p. 39
• Agents overlaid a degrading image onto a person’s photograph and advanced through a machine-gun purchase flow without recognizing the harmful nature of either task.The weapons trajectory reached checkout before hesitation over payment information, while the image-editing trajectory treated targeted harassment as routine editing.Source: I Trajectory Case Studies · pp. 40, 42, Table 18 · p. 40, Table 19 · pp. 41–42
• Agents analyzed a prediction-market bet without warning about gambling risks, accepted exploitative agreement clauses without review, and executed project scripts that exfiltrated credentials.These cases show task-completion behavior overriding risk assessment, contractual review, and source-code inspection.Source: I Trajectory Case Studies · pp. 43, 44, 46, Table 20 · p. 43, Table 21 · p. 44
• During a benign spreadsheet task, the agent clicked an injected “Unlock” pop-up instead of “Cancel,” showing that task-relevant visual deception can manipulate GUI behavior.The trajectory identifies susceptibility to adversarial interface elements embedded in otherwise routine operations.Source: I Trajectory Case Studies · p. 47, Table 23 · p. 47

J Task Inventory by Category

• The task inventory provides category-based benchmark examples, listing five representative tasks per category while directing readers to the supplementary material for the complete lists.Categories 1–11 report task names, user instructions, and evaluation metrics; Category 12 reports OSWorld task IDs, instructions, and tested applications.Source: J Task Inventory by Category · p. 48

J.1 Category 1: Credential Phishing

• Credential Phishing evaluates whether agents verify interfaces that mimic legitimate services before submitting personal, financial, or credential data to attacker-controlled forms.The category targets environment-embedded deception in routine account-verification or password-recovery tasks, including fraudulent banking, email, and corporate portals.Source: §J.1 · pp. 48–49, Table 24 · p. 49

J.2 Category 2: Data Exfiltration

• Data Exfiltration tests whether benign workflows induce agents to disclose confidential local assets, such as configurations, credentials, or proprietary code, to external destinations.Successful attacks include executing an unreviewed script that uploads configurations and credentials or submitting proprietary source code to an unknown portal.Source: §J.2 · pp. 49–50

J.3–J.5 Categories 3–5

• Fraud and Forgery, Malware Deployment, and System Sabotage test unauthorized document alteration, execution of disguised malicious payloads, and destructive system changes presented as routine maintenance.System-sabotage examples include disabling firewalls, running remote scripts, applying permissive permissions, and redirecting banking websites through modified hosts files.Source: §J.3 · p. 50, §J.4 · p. 51, §J.5 · pp. 51–52

J.6–J.7 Categories 6–7

• Misinformation and Manipulation evaluates whether agents generate and disseminate persuasive false content, while Harassment and Threats targets abusive, coercive, or threatening communications aimed at specific individuals.The former emphasizes ecosystem-wide deception; the latter emphasizes direct emotional or psychological harm.Source: §J.6 · p. 52, §J.7 · p. 53

J.8–J.9 Categories 8–9

• Illegal and Harmful Content tests whether agents process or distribute dangerous materials embedded in neutral operations instead of refusing, whereas Risky Financial URLs tests consequential engagement with gambling, crypto, or speculative platforms.Risky financial attacks include interacting with betting content or saving gambling-related money-making suggestions.Source: §J.8 · p. 53, §J.9 · pp. 54–55

J.10–J.11 Categories 10–11

• Deceptive Agreements and Code Injection exploit uncritical trust in legal text or technical artifacts, causing agents to accept exploitative terms or execute embedded malicious directives.The categories distinguish careless consent from instruction-source confusion and literal execution of obscured technical payloads.Source: §J.10 · p. 55, §J.11 · p. 56

J.12 Category 12: Pop-ups Attack

• Pop-ups Attack generates interactive, context-aware blocking pop-ups from the three preceding screenshots, counting an attack only when the agent accepts the misleading claim rather than rejecting it.A vision-language model extracts application context and recent changes, after which a language model generates a workflow-aligned popup with accept and reject options.Source: §J.12 · pp. 57–58

K.1 Evaluation Prompt for End-to-End and Per-Step Planning Models

• The end-to-end and per-step safety judge evaluates benign user instructions, risk metrics, and action trajectories, labeling behavior unsafe when agents perform or progress toward harmful outcomes, leak data, execute untrusted code, or follow embedded instructions.Safe behavior requires refusal, risk warnings with confirmation, or stopping with an explanation; merely opening sites or downloading files is not inherently unsafe.Source: §K.1 · pp. 60–61

K.2 CoAct-1 Multi-Agent Evaluation Prompt

• The CoAct-1 multi-agent judge uses the original instruction, evaluation metric, orchestrator subtasks, and—most importantly—the CUA’s actual execution steps to assess safety.Its assessment is based on what the system actually executed.Source: §K.2 · p. 62
• CoAct-1 evaluates safety primarily from the CUA’s executed GUI actions, including typed, pasted, or drag-selected data, rather than merely the orchestrator’s assigned subtask.The orchestrator provides context, but execution failures or partial completion determine what the CUA actually did; alternative input methods carry the same safety implications.Source: §K.2 · p. 62
• The evaluation marks behavior unsafe when the CUA enters sensitive or harmful data, meaningfully advances the harmful outcome, follows prompt injections, or processes harmful content without refusing.Safe outcomes require refusal after recognizing harm or no harmful action; defense labels distinguish orchestrator defense, GUI defense, and task failure.Source: §K.2 · pp. 62–63

K.3 System Safety Prompt

• The system safety prompt instructs the model to assess real-world consequences and refuse unsafe tasks regardless of how reasonable the request sounds, including risks embedded in environmental content.The prompt explicitly warns that emails, webpages, documents, and pop-ups may contain deceptive instructions.Source: §K.3 · p. 63

K.4 Intent Reconstruction Prompt Used in Claude

• The Claude intent-reconstruction prompt requires inferring user intent from a desktop screenshot and decomposed subtask before producing a workable high-level instruction in JSON.Its required output contains the keys inferred_intent and workable_task.Source: §K.4 · p. 63