Source-linked AI summary
Safe, or Simply Incapable? Rethinking Safety Evaluation for Phone-Use Agents
Zhengyang Tang, Yi Zhang, Chenxin Li, Xin Lai, Pengyuan Lyu, Yiduo Guo, Weinong Wang, Junyi Li, Yang Ding, Huawen Shen, Zhengyao Fang, Xingran Zhou, Liang Wu, Fei Tang, Sunqi Fan, Shangpin Peng, Zheng Ruan, Anran Zhang, Benyou Wang, Chengquan Zhang, Han Hu
TL;DR
Phone-use safety evaluations can mistake harmless outcomes caused by inaction for genuinely safe decisions. PhoneSafety separates safe action, unsafe action, and failure to act at safety-critical moments, revealing that general capability does not reliably predict safe choices.
Problem
Harmless outcomes can reflect either safe choices or inability to act, but existing evaluations often conflate these distinct cases.
Method
PhoneSafety evaluates the next action at safety-critical moments using a three-way distinction between safe action, unsafe action, and failure to act.
Results
Across eight models, general phone-use capability does not reliably predict safe choices, while failures to act track broader capability and concentrate in demanding screens.
Takeaways & Limitations
Harmless outcomes alone are insufficient evidence of safety; evaluations should distinguish unsafe judgment from inability to act.
Takeaways & Limitations
The study uses offline evaluation on safety-critical moments in a Chinese mobile ecosystem.
Abstract
from arXiv · showhide
When a phone-use agent avoids harm, does that show safety, or simply inability to act? Existing evaluations often cannot tell. A harmful outcome may be avoided because the agent recognized the risk and chose the safe action, or because it failed to understand the screen or execute any relevant action at all. These cases have different causes and call for different fixes, yet current benchmarks often merge them under task success, refusal, or final harmful outcome. We address this problem with PhoneSafety, a benchmark of 700 safety-critical moments drawn from real phone interactions across more than 130 apps. Each instance isolates the next decision at a risky moment and asks a simple question: does the model take the safe action, take the unsafe action, or fail to do anything useful? We evaluate eight representative phone-use agents under this framework. Our results reveal two main patterns. First, stronger general phone-use ability does not reliably imply safer choices at risky moments. Models that perform better on ordinary app tasks are not always the ones that behave more safely when the next action matters. Second, failures to do anything useful behave like a capability signal rather than a safety signal: they are concentrated in more visually and operationally demanding settings and remain stable when the evaluation protocol changes. Across models, failures split into two recurring patterns: unsafe choices in settings where the model can act but chooses wrongly, and inability to act in more visually and operationally demanding screens. Overall, a harmless outcome is not enough to count as evidence of safety. Evaluating phone-use agents requires separating unsafe judgment from inability to act.
1 Introduction
Phone-use agents can avoid harm either through safe judgment or inability to act, but harmless outcomes alone cannot distinguish these causes. PHONESAFETY addresses this confound by evaluating the next action at safety-critical moments with a three-way classification.
- Introduction: Harmless outcomes may reflect recognizing a safety boundary and choosing correctly, or failing to understand the screen, locate the control, or act usefully.These cases provide different evidence: one indicates safe judgment, while the other indicates inability to act.
- Introduction: At a subscription page, the safe action is asking before payment, the unsafe action is subscribing, and a third outcome is failing to interact meaningfully.All three next actions can produce a harmless outcome, but only stopping and asking is evidence of safety.
- Introduction: PHONESAFETY evaluates 700 safety-critical moments from real phone interactions across more than 130 apps, classifying each decision as safe, unsafe, or not useful.The model receives the user instruction, recent interaction history, and current screen context, then chooses the next action at the decision point.
- Introduction: Across 8 representative models, general phone-use capability does not reliably predict safe choices, while failures to act usefully track broader capability and concentrate in operationally demanding screens.These failures also remain stable when the safe/unsafe boundary changes with the evaluation protocol.
- Introduction: Safety evaluation for action-taking agents must distinguish unsafe judgment from inability to act instead of treating all harmless outcomes as equivalent.The paper identifies apparent harmlessness as a central evaluation confound and addresses it through safety-critical-moment analysis.
2 Evaluating Safety-Critical Moments with PHONESAFETY
PHONESAFETY evaluates phone-use agents at safety-critical moments, classifying each next action as safe, unsafe, or failing to do anything useful. This separation prevents harmless outcomes caused by safe judgment from being conflated with failures to engage meaningfully with the decision.
- Evaluation unit: PHONESAFETY isolates the next decision at a safety-critical moment, where the model’s action can directly determine whether the interaction remains safe or becomes unsafe.The evaluation complements rather than replaces trajectory-level and online evaluation for long-horizon robustness and adaptive attacks.
- Outcome categories: Each moment receives exactly one of three outcomes: safe action, unsafe action, or failure to do anything useful.Safe actions engage with the decision and choose the safe side; unsafe actions meaningfully engage but cross the safety boundary; failures realize neither side.
- Interpretation: Separating the three outcomes is necessary because harmless outcomes can result either from safe choice or from failure to engage meaningfully with the decision.The third category is operational and does not by itself establish a deep or universal model inability.
- Benchmark construction: The benchmark comprises 700 safety-critical moments drawn from 4,512 realistic trajectories spanning 130+ apps and approximately 75K steps.The cases were screened, validated, and annotated with protocol-grounded safe and unsafe reference behaviors.
- Benchmark construction: The expanded benchmark includes visually and operationally demanding settings, requiring both successful screen understanding and action execution to produce the safe response.This broader coverage helps distinguish avoiding harm through safe choice from avoiding harm because the model cannot act effectively.
- Evaluation protocol: At each moment, the model receives the user instruction, recent action history, and current screen context, then predicts the next phone action for classification.The main evaluation uses the stated protocol, while protocol variation is examined later in an ablation study.
3 Results
PhoneSafety shows that general phone-use capability does not reliably predict safe decisionmaking, while failures to produce any relevant action behave more like a capability signal than a safety signal. Separating unsafe choices from inability to act is therefore necessary to interpret harmless outcomes and diagnose failures.
- Outcome decomposition: A three-way decomposition is needed because low harmful-outcome rates can reflect genuinely safe choices or failure to engage meaningfully with the critical decision.The framework distinguishes safe action, unsafe action, and failure to do anything useful.
- Capability and safety: General phone-use capability is not a reliable proxy for safe decisionmaking, with models occupying different positions on ordinary tasks and safety-critical safe-action rate.Claude Opus 4.6 ranks third on general evaluation performance but second on safe-action rate, while MobileAgent 3.5 ranks fourth and sixth, respectively.
- Failure profiles: Unsafe choices dominate Safety cases, where GELab-Zero 4B, MobileAgent 3.5, and MAI-UI 8B achieve safe-action rates of 3.1%–7.7% and usually continue harmful trajectories.These failures primarily indicate bad judgment rather than inability to reach the relevant decision space.
- Failure profiles: In Confirm, OP, TR, and PM cases, failures to do anything useful account for 80–98% of failures and arise in settings requiring precise grounding on demanding interfaces.The pattern suggests perception, grounding, or control limitations rather than the same weakness as unsafe choice.
- Failure as capability signal: The rate of producing any relevant action correlates with general capability, with Pearson r = 0.787 and Spearman ρ = 0.922, supporting its interpretation as a capability signal.Across 5,689 instances, 72% were partial or failed executions, 23% random or unrelated actions, and 5% genuinely unsafe; the rate was stable across protocol changes.
4 Conclusion
The conclusion argues that harmless outcomes do not establish safety because they may reflect safe choices or inability to act. Phone-use evaluation should therefore separate unsafe judgment from incapacity at safety-critical moments.
- Conclusion: Harmless outcomes may reflect either genuinely safe choices or inability to act, so treating them alike can overestimate weak models and misdiagnose stronger ones.The ambiguity also blurs poor judgment with inability to act.
- Conclusion: PHONE-SAFETY evaluates the decisive next action using three outcomes: safe action, unsafe action, or failure to do anything useful.This moves beyond asking only whether harm occurred by the episode’s end.
- Conclusion: General phone-use capability does not reliably predict safe choices under risk, while failures to do anything useful behave as a distinct capability-related pattern.The three-way decomposition exposes these two findings.
- Conclusion: Safety credit should depend on whether the model can act in the relevant interface region and choose the safe side of the decision.The study is limited to offline evaluation in a Chinese mobile ecosystem, but the authors state the underlying point is more general.
- Conclusion: Evaluating phone-use agents requires separating unsafe judgment from inability to act rather than treating harmless outcomes alone as evidence of safety.This is the conclusion’s broader implication for action-taking agents.
A Related Work
Prior work shows that mobile and computer-use agents face interface-specific safety risks, but evaluations often assess outcomes only at the task, trajectory, or attack-episode level. This work instead evaluates safety-critical moments to distinguish safe choice, unsafe choice, and inability to act.
- A Related Work: Mobile and computer-use agents face safety risks from dynamic interfaces, deceptive overlays, ambiguous inputs, environmental injection, privacy leakage, and action rebinding.Benchmarks and attack studies include MobileSafetyBench, GhostEI-Bench, RiOSWorld, GUIGuard, and related work.
- A Related Work: Most existing evaluations judge whether agents complete risky tasks, refuse, are successfully attacked, leak private information, or fail benignly at the task, trajectory, or attack-episode level.Some work distinguishes partial attack success from benign failure or separates risky intent from risky completion.
- A Related Work: Deployment-oriented safety methods use permission mechanisms, safety prompting, privacy-preserving mediation, runtime monitoring, and step-level guardrails to prevent harm or reduce exposure during execution.These efforts are complementary because they target safer deployment, whereas this work targets earlier diagnosis.
- A Related Work: The paper is an evaluation study that addresses the confound that apparent harmlessness may reflect either safe choice or inability to act.It separates safe actions, unsafe actions, and failures to do anything useful at safety-critical moments rather than proposing a new defense or relying on one aggregate score.
B Action Matching Rules
PhoneSafety matches model actions against expected safe and unsafe behaviors using action type and, when applicable, target specifications. Intention-like actions require type matching, while grounded actions and same-type alternatives require target-sensitive matching.
- Each safety-critical moment defines expected safe and unsafe actions by action type and, where applicable, target specification.
- Intention-like actions, including call_user, finish, answer, and wait, are classified by action type alone.
- Grounded actions such as click, type, scroll, and swipe require matching both action type and target.
- For clicks, the predicted and annotated coordinates match when normalized Euclidean distance is below 0.14, approximately 14% of the screen diagonal.
- When safe and unsafe behaviors share an action type, target proximity determines the label; actions far from both targets are failures to do anything useful.
C CFR Judge Audit Details
An independent audit of 5,689 CFR instances across eight models found that most cases reflected partial or unrelated actions, while only 5% were unsafe reclassifications. This supports interpreting CFR mainly as failure to produce relevant actions rather than concealed unsafe behavior, though AutoGLM 9B showed especially broad operational difficulty.
- Audit classification: 5,689 CFR instances across all 8 models were independently classified into partial execution, random/unrelated actions, or unsafe reclassifications.Each instance was assigned to one of three categories.
- Audit classification: 72% involved partial execution, where outputs were directionally correct but failed to fully realize the needed behavior.Examples included clicking the wrong area when closing a popup or producing malformed refusal output.
- Audit classification: 23% were random or unrelated actions, such as scrolling when a dialog required a button press or outputting coordinates far from interactive elements.These actions were unrelated to the current screen context.
- Audit interpretation: 5% were unsafe reclassifications, and this low rate supports interpreting CFR as genuine failure to produce relevant actions rather than systematically hidden unsafe behavior.Most reclassifications arose from coordinate-matching edge cases near the boundary threshold.
- Model-specific finding: AutoGLM 9B produced 42% random actions, suggesting its high CFR reflected broad operational difficulty rather than targeted failure at specific interface elements.The finding points to difficulty operating across the interface rather than failure concentrated on particular elements.
D Detailed Three-Way Results by Scenario Family
Tables 2 and 3 provide the full three-way decomposition of results by scenario family across all eight models.
- Three-Way Results: Tables 2 and 3 report the full three-way decomposition by scenario family.
- Three-Way Results: The decomposition covers every scenario family evaluated.
- Three-Way Results: The results include all 8 models.
E Benchmark Composition · F Safe-Action Rate by Scenario Family · G Protocol Ablation Details
The benchmark composition and scenario-family results are organized in Tables 2–5, while the protocol ablation reclassifies stored outputs under strict and minimal authorization rules. Its main signal is relative movement between safe and unsafe labels, not direct comparison of absolute rates with the main table.
- G Protocol Ablation Details: The ablation reclassifies the same stored model outputs with an auxiliary rule-based procedure instead of rerunning models under alternative prompts.
- G Protocol Ablation Details: Because the auxiliary classifier is simplified, absolute ablation rates are not directly comparable with the main table; relative protocol changes are the relevant signal.
- G Protocol Ablation Details: The strict and minimal protocols differ at the authorization boundary, while retaining the same action-matching rule.
- G Protocol Ablation Details: Under the strict protocol, incomplete, implicit, or ambiguous authorization makes directly proceeding unsafe and confirmation-seeking safe.
- F Safe-Action Rate by Scenario Family: Tables 2 and 3 provide three-way decompositions by scenario family for eight models, with percentages as the reported values.
- G Protocol Ablation Details: Under the minimal protocol, some directly performed actions are relabeled safe when the user’s instruction already sufficiently authorizes proceeding, primarily shifting cases between safe and unsafe columns.
- E Benchmark Composition: Table 4 organizes PhoneSafety by scenario family, consequence severity, risk-emergence phase, and authorization status.
- F Safe-Action Rate by Scenario Family: Table 5 reports safe-action rates by scenario family for all eight models.
H Limitations
PhoneSafety is an offline, decision-point evaluation focused on interpreting apparently harmless outcomes at safety-critical moments, complementing rather than replacing end-to-end online studies. Its scope is limited to authorization-sensitive action safety within a Chinese Android ecosystem and is not a comprehensive deployment evaluation.
- Evaluation scope: PhoneSafety targets interpretation of apparently harmless outcomes at safety-critical moments in phone-use agents through an offline, decision-point evaluation.It is designed to complement, rather than replace, end-to-end online studies of long-horizon interaction, recovery behavior, or adaptive attacks.
- Evaluation scope: The benchmark is grounded in a Chinese Android ecosystem and reflects its app interfaces, permission flows, and authorization conventions.
- Evaluation scope: PhoneSafety does not cover every deployment-relevant safety dimension beyond authorization-sensitive action safety and is not intended as a comprehensive deployment evaluation.The contribution is framed as a clearer evaluation lens for safety-critical action moments.
I Statistical Reporting · J Broader Impacts · K Declaration of LLM Usage
The paper reports exact corpus-level evaluation rates and bases its conclusions on converging analyses rather than marginal model differences. It also distinguishes the benchmark’s potential benefits and risks, while documenting methodological and non-methodological uses of LLMs.
- I Statistical Reporting: Statistical percentages are exact corpus-level measurements from fixed sets: 700 PHONESAFETY moments and 7,168 general phone-use anchor steps.The paper emphasizes full-set rates rather than per-model error bars.
- I Statistical Reporting: Inferential claims about capability relationships or ranking stability are treated as standalone evidence, not as substitutes for corpus-level measurements.The reported percentages are measurements on fixed evaluation sets rather than estimates from repeated randomized training runs.
- I Statistical Reporting: Protocol ablation found ∆CFR = 0.0 for all models and Spearman ρ = 0.874 (p = 0.005) between strict and minimal rankings.These results summarize the auxiliary-classifier comparison between strict and minimal safe-action rates.
- I Statistical Reporting: The main conclusions rely on converging patterns across the three-way decomposition, scenario-family breakdowns, CFR audit, and protocol variation.The paper does not base its conclusions on marginal differences between adjacent models.
- J Broader Impacts: Separating unsafe judgment from useless action may support better release decisions, targeted safeguards, and clearer diagnosis of perception, grounding, and control failures.The broader-impact rationale is improving evaluation of action-taking agents before deployment and avoiding safety credit for merely inert systems.
- J Broader Impacts: The benchmark could expose model weaknesses, encourage optimization around evaluation patterns, or have scores over-interpreted as deployment readiness.Safe real-world use still depends on product safeguards, runtime monitoring, and environment-specific protocol decisions.
- K Declaration of LLM Usage: LLMs contributed to documented methodological components, including evaluated agents and judge-based pipeline elements, and otherwise only to manuscript editing and polishing.They were not used to originate scientific claims, replace author verification, or determine conclusions beyond explicitly described procedures.