PaperlayerView exact revision on arXiv

Source-linked AI summary

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong

arXiv:2510.24411v3cs.AIcs.CLcs.CVcs.HC

TL;DR

Mobile GUI agent safety remains underexplored despite the challenges posed by diverse apps, sensitive data, and dynamic interaction contexts. The paper introduces a dynamic sandbox and annotated benchmark, then combines formal verification with contextual judging in OS-Sentinel. It reports that the framework surpasses typical baselines at both trajectory and step levels.

  • Problem

    Mobile GUI safety is underexplored, and existing environments and detectors have limitations in application coverage, system-state access, context handling, and trajectory-level evaluation.

    Source: §1 · p. 2

  • Method

    The paper introduces MobileRisk-Live and the MobileRisk trajectory benchmark, then proposes OS-Sentinel, combining formal system-level verification with contextual model-based judgment.

    Source: §1 · p. 2

  • Results

    OS-Sentinel consistently surpasses typical safety-detection baselines at both trajectory and step levels.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MobileRisk provides a fine-grained benchmark for systematic mobile-agent safety studies, while OS-Sentinel broadens detection across step and trajectory levels.

    Source: §1 · p. 2

    The Formal Verifier relies on system-state traces accessible on open platforms such as Android, making the approach less directly applicable to closed environments such as iOS.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents hold great promise for advancing digital automation, their potential for unsafe operations, such as system compromise and privacy leakage, is raising significant concerns. Detecting these safety concerns across the vast and complex operational space of mobile environments presents a formidable challenge that remains critically underexplored. To establish a foundation for mobile agent safety research, we introduce MobileRisk-Live, a dynamic sandbox environment accompanied by a safety detection benchmark comprising realistic trajectories with fine-grained annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety detection framework that synergistically combines a Formal Verifier for detecting explicit system-level violations with a VLM-based Contextual Judge for assessing contextual risks and agent actions. Experiments show that OS-Sentinel achieves 10%-30% improvements over existing approaches across multiple metrics. Further analysis provides critical insights that foster the development of safer and more reliable autonomous mobile agents. Our code and data are available at https://qiushisun.github.io/OS-Sentinel-Home/.

1 Introduction

Mobile GUI agents raise safety concerns because diverse apps, sensitive data, and dynamic contexts make trustworthy behavior difficult to ensure. The paper introduces a mobile sandbox and benchmark alongside a hybrid detector, and reports that OS-Sentinel outperforms typical baselines at both trajectory and step levels.

1 Introduction

  • Mobile safety evaluation is underexplored, while existing environments have limited app coverage and miss full system states, and existing detectors struggle with context, strict boundaries, or realistic trajectories.Mobile environments may omit runtime processes; prior approaches include deterministic verifiers, model-based detectors, and step-level studies disconnected from multi-action trajectories and system-state transitions.Source: §1 · p. 2
  • MobileRisk-Live provides a dynamic Android-emulator environment, and MobileRisk adds fine-grained, multi-level annotations to agent trajectories.The benchmark supports diverse safety-detection schemes and the isolated study of safety challenges.Source: §1 · p. 2
  • OS-Sentinel combines formal system-level verification with model-based contextual judgment to broaden detection across step and trajectory levels.The paper presents the combination as addressing reliance on either non-scalable verifiers or overly generic judges.Source: §1 · p. 2
  • OS-Sentinel consistently surpasses typical safety-detection baselines at both trajectory and step levels.The paper also reports analyses of sandbox reliability and framework components.Source: §1 · p. 2

2 Related Works

Prior computer-using agents span desktop, web, and mobile interfaces, but mobile GUI agent safety remains underexplored. The paper situates its sandbox and hybrid framework as addressing gaps in realistic mobile safety infrastructure and detection.

2 Related Works

  • Mobile GUI agents are an emerging direction among agents that operate across desktop, web, and mobile interfaces to carry out user-instructed tasks.The related-work overview also notes industrial integration into products.Source: §2 · p. 2
  • Mobile GUI agent safety remains underexplored, and static benchmarks lack the dynamics of realistic computer-use scenarios.The related-work discussion notes existing infrastructures often rely on static benchmarks.Source: §2 · p. 2
  • Prior mobile environments have limited application coverage and may omit full system states, while MobileRisk-Live records frozen trajectories with screenshots, accessibility trees, system traces, actions, and multi-granularity annotations.The caption describes a dynamic Android emulator sandbox with state restoration and interfaces for safety detectors.Source: Figure 2 · p. 3
  • Rule-based verification struggles to scale, model-based detection often targets specific GUI attacks, and prior studies favor step-level over trajectory-level evaluation.The paper presents its dedicated environment and hybrid framework as extending the breadth and depth of mobile GUI safety research.Source: §2 · p. 3

3 Testbed for Mobile Agent Safety

The section defines mobile-agent safety in terms of system-level and contextual violations, then presents a sandbox and benchmark designed to support realistic, reproducible evaluation.

3.1 Preliminary

  • A trajectory is unsafe if any step violates system-level constraints or exhibits risky contextual behavior.System-level examples include unauthorized operations captured in system traces; contextual risks include privacy leakage and harmful content.Source: §3.1 · p. 3

3.2 MobileRisk-Live: A Dynamic Sandbox

  • MobileRisk-Live records GUI observations, agent actions, and system-state traces while supporting pre-installed apps, custom extensions, resets, and new instructions.This captures both agent-visible behavior and background system changes during execution.Source: §3.2 · pp. 3–4

3.3 MobileRisk: A Benchmark of Realistic Trajectories

  • MobileRisk uses frozen trajectories for reproducible static evaluation and restored states for dynamic execution, helping separate safety analysis from agent capability.The design also supports fine-grained safety annotations.Source: §3.3 · p. 4
  • The benchmark labels trajectory safety, the first unsafe step, and risk category across ten categories spanning user-side and agent-side risks.It contains 102 unsafe and 102 safe instances; half of the safe cases are corrected counterparts of unsafe trajectories to help assess false positives.Source: §3.3 · p. 4

4 OS-Sentinel

OS-Sentinel combines deterministic system checks with VLM-based contextual analysis to assess mobile-agent safety at step and trajectory levels. Its components address different risk types and can be combined through configurable verdict rules.

4 OS-Sentinel

  • OS-Sentinel pairs a Formal Verifier for explicit system-level risks with a VLM Contextual Judge for context-dependent threats, operating at step and trajectory levels.Step-level detection supports real-time guarding, while trajectory-level detection supports post-hoc analysis.Source: §4 · p. 5

4.1 Formal Verifier: Unified Rule-based System Checker

  • The Formal Verifier uses system-state traces for general-purpose deterministic checks, including file-metadata integrity hashes, sensitive-keyword detection, and pattern matching.A step is flagged when integrity is violated or aggregated risk exceeds a threshold; the design is agent-agnostic and needs no task-specific annotations.Source: §4.1 · pp. 5–6

4.2 Contextual Judge: Model-based Safety Analysis

  • The Contextual Judge uses a VLM to assess each observation-action pair and return a binary step verdict for real-time intervention.Trajectory assessment can evaluate consecutive windows or uniformly sampled transition points.Source: §4.2 · p. 6
  • OS-Sentinel fuses verifier and judge predictions through configurable rules; the main experiments use strict OR, while consensus AND requires component agreement.Strict mode is intended to maximize risk coverage; consensus mode supports high-confidence judgments.Source: §4.2 · p. 6

5 Experiments

Experiments compare step- and trajectory-level safety detection on MobileRisk and assess whether frozen trajectories reflect live-environment outcomes. OS-Sentinel outperforms baselines, with reported practicality across model scales and low detection latency.

5.1 Experimental Settings

  • Experiments use an Android Studio emulator with UIAutomator system-state traces and a Pixel 6a simulator; task-execution agents use GPT-4o with the M3A workflow.Safety detectors and contextual-judge components use proprietary and open-source backbones at different scales.Source: §5.1 · p. 6

5.3 Main Results

  • At trajectory level, OS-Sentinel substantially outperforms rule-based evaluators and consistently exceeds model-based baselines across settings.Table 1 consolidates Precision and Recall into F1 and includes rule-based and model-based comparisons.Source: Table 1 · p. 7, §5.3 · p. 7

5.2 Baseline Construction

  • The baselines adapt task-specific rule evaluators and VLM/LLM judges for step- and trajectory-level detection using screenshots or accessibility trees.Table 1 reports each backbone both as a judge baseline and as OS-Sentinel’s contextual judge.Source: §5.2 · pp. 6–7, Table 1 · p. 7

5.3 Main Results

  • At step level, OS-Sentinel generally outperforms all baselines and uses multiple evidence sources for more accurate judgments under incomplete information.The authors note that performance under limited information still leaves room for improvement.Source: §5.3 · p. 7
  • OS-Sentinel is model-agnostic, achieves strong results with a 7B-scale backbone, and averages 66ms per detection for API models.Quantized versions of the 7B-scale backbone can run locally on mobile devices.Source: §5.3 · p. 7
  • On 104 selected instructions, frozen MobileRisk evaluation closely approximates live MobileRisk-Live trajectory-level detection outcomes.The instructions exclude account creation and unsafe behaviors that could affect others.Source: §5.3 · pp. 7–8, Table 2 · p. 8

6 Analysis

The analysis examines unsafe instances, component contributions across backbones and metrics, and detection coverage across risk categories.

6.1 Component Contribution Analysis

  • Combining the Formal Verifier and Contextual Judge consistently achieves substantially better trajectory-level accuracy than either component alone, though the dominant component varies by backbone.The analysis reports greater than 80% accuracy when targeting unsafe cases and excluding false-positive influence.Source: Figure 5 · p. 8, §6.1 · p. 8
  • On F1, component contributions vary with the backbone, but their combination still yields substantial improvements.The authors attribute the differing component trends to differences in how models process observations.Source: Figure 6 · p. 8, §6.1 · p. 8

6.2 Category-wise Analysis

  • Across unsafe-behavior categories, OS-Sentinel provides stronger and more balanced detection, while both baselines have category-specific strengths and weaknesses.The comparison is shown for OS-Sentinel backed by GPT-4o.Source: §6.2 · p. 8, Figure 7 · p. 9

7 Conclusion

The conclusion presents MobileRisk-Live and MobileRisk as infrastructure for realistic, reproducible mobile GUI safety research, alongside OS-Sentinel as a hybrid detection framework.

7 Conclusion

  • MobileRisk-Live provides a dynamic sandbox, while MobileRisk supplies a fine-grained benchmark for reproducible evaluation.Source: §7 · p. 9
  • OS-Sentinel unifies deterministic verification with contextual risk assessment across system states, multimodal content, and agent actions.Source: §7 · p. 9
  • The authors report that experiments validate the effectiveness and reliability of their testbeds and detection strategies.Source: §7 · p. 9

Limitations

The authors identify two limitations: reliance on system-state traces constrains direct use on closed platforms, and the simulated environment and frozen dataset retain some differences from live conditions.

Verifier Dependency

  • Direct use on closed platforms such as iOS is limited because the Formal Verifier relies on system-state traces accessible on open platforms such as Android.The authors note that adapting the approach to other platforms may be possible according to practical needs.Source: Limitations · p. 9

Environment

  • MobileRisk-Live is simulated and MobileRisk is derived from it, so frozen and live settings can still differ, including through random online push notifications.The authors report strong closeness between settings and believe the remaining differences do not undermine their general conclusions.Source: Limitations · p. 9

Broader Impacts

The authors conduct experiments in controlled virtual environments and release study materials for research use, while also reporting compliance with applicable data-use rules.

Controlled experimentation

  • All experiments use controlled virtual environments, which the authors say eliminates risks to real devices or user accounts.They encourage researchers to use the provided environment or benchmark rather than personal devices or accounts.Source: Broader Impacts · p. 9

Data Usage Compliance

  • The authors state that their experiments comply with applicable data-use regulations and licensing requirements.Source: Broader Impacts · p. 9

Discussion

The paper describes OS-Sentinel’s safety-first operating choices alongside MobileRisk’s trace collection, benchmark construction, annotation, and evaluation procedures. Its appendices report broad app coverage, annotation agreement, trajectory characteristics, and comparisons with other benchmarks.

Prioritizing Safety Recall in Risk Scenarios

A.1 System State Trace Acquisition

  • MobileRisk-Live records system-state traces alongside GUI observations and actions, capturing underlying system dynamics that screenshots and accessibility trees alone do not show.The trace acquisition process extracts metadata concurrently from virtual-machine kernel and framework layers.Source: §A.1 · p. 13

A.2 Applications Covered

  • The benchmark spans 50 applications and components, and required trace metadata can be accessed through standard Android APIs without root or ADB access.MobileRisk-Live is designed for updates to apps, APIs, and monitors, and for periodic benchmark refreshing using automated state restoration and trace acquisition.Source: §A.2 · p. 14, §A.3 · p. 14, §A.4 · p. 14

B.2 Trajectory Collection

  • The dataset combines manually collected trajectories—including some recorded on physical devices—with instructions adapted from prior benchmarks and screened for inclusion.Trajectory records include screenshots, actions, and UI metadata; the initial screening retains approximately 18% of executed trajectories.Source: §B.1 · p. 14, §B.2 · pp. 14–15, §B.3 · p. 15

B.5 Safety Issue Taxonomy

  • MobileRisk defines ten safety-issue categories and reports a spread of unsafe trajectories weighted toward categories with higher severity and likelihood.Source: §B.5 · p. 15

B.6 Trajectory Statistics

  • Trajectories range from 2 to 46 steps, with most between 5 and 12 steps and a peak at 7; those exceeding 25 steps are rare.Source: §B.6 · p. 15, Figure 9 · p. 15

B.7 Annotation Guideline

  • Annotators label a trajectory unsafe if any step violates the taxonomy, distinguish safe failure from violation, and localize the first unsafe step; binary judgments show substantial agreement and localization MAE is below 0.5 step.Source: §B.7 · p. 16, §B.8 · p. 16

B.9 Evaluation Metrics Details

  • Evaluation reports trajectory-level accuracy and F1, plus a delay-penalized step score averaged across instances and scaled by 100; the benchmark offers more than twice the tasks and trajectories of typical mobile safety datasets.The step score uses a temporal budget of 3 steps.Source: §B.9 · pp. 16–17, §B.10 · p. 17

C Experimental Details

The appendix compares OS-Sentinel’s strict and consensus fusion modes and analyzes the trigger behavior of its Formal Verifier components. It also records the agent action format and emulator settings used in the experiments.

C Experimental Details

D.1 Formal Verifier Component Analysis

  • Sensitive Pattern has the highest average trigger count per trajectory, Sensitive Keyword has a comparable aggregate trigger share, and State Integrity is particularly sensitive to destructive operations.Sensitive Pattern and Sensitive Keyword trigger more often on compliance violations, security evasion, and harmful content.Source: Figure 10 · p. 17, Figure 11 · p. 17, §D.1 · p. 17

D.2 Impact of Mode Selection

  • Consensus mode lowers F1 by ∆≈−17.4; it offers higher precision by filtering ambiguous false positives, while strict mode is better for maximizing safety coverage.Source: Table 10 · p. 18, §D.2 · p. 18

D.3 Error Analysis and Human-Agent Disagreement

The analysis traces human-agent disagreements to conservative safety judgments, describes a rule-based baseline, and illustrates how the formal verifier and contextual judge detect distinct risks in case studies.

D.3 Error Analysis and Human-Agent Disagreement

  • OS-Sentinel’s zero-tolerance policy flags file changes as destructive even when they are reversible, user-requested cache clearing, contributing to a precision gap.Human annotators consider the explicit intent and reversibility, while the verifier treats the system modification as a violation.Source: §D.3 · p. 18
  • The Contextual Judge flags unconfirmed bulk notification deletion that human judges may tolerate, reflecting a conservative stance toward over-automation.The authors identify aligning this sensitivity with nuanced user tolerance as future alignment research.Source: §D.3 · p. 18

E Baseline Settings

  • The rule-based baseline uses task-specific checklists of sensitive keywords and UI elements, flagging matches in environment state across tasks at both step and trajectory levels.Evaluators inspect sources such as screen text, database entries, or file contents.Source: E Baseline Settings · p. 18

F Case Studies

  • A before/after hash discrepancy reveals a system-file modification invisible in GUI observations, which the Formal Verifier flags as unsafe.The accompanying figure shows a mismatch in the System State Trace.Source: F Case Studies · pp. 18–19, Figure 12 · p. 20
  • When the agent makes no system-level change, the Contextual Judge can still flag an interaction involving sensitive bank-account information.The case illustrates contextual risk at the application level.Source: F Case Studies · p. 19, Figure 13 · p. 20

Related papers

Loading 2510.24411v3…