PaperlayerView exact revision on arXiv

Source-linked AI summary

AVP-Inspect: Coordinated Cyber-Physical Testing for Privacy Analysis of COTS Apple Vision Pro Applications

Yichang Xiong, Vamsi Shankar Simhadri, Yue Xiao, Xiaokuan Zhang

arXiv:2609.08103v2cs.CR

TL;DR

AVP’s privacy risks and closed ecosystem have left its applications without comprehensive privacy analysis. AVP-Inspect combines hardware and software control to test AVP applications against their disclosures. Its analysis found disclosure violations across many apps and traffic flows.

  • Problem

    AVP’s extensive biometric and behavioral data collection raises privacy concerns, while its closed ecosystem has limited comprehensive privacy analysis.

    Source: §1 · p. 1

  • Method

    AVP-Inspect coordinates hardware and software control to explore AVP app interfaces and compare captured network traffic with privacy disclosures.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    188/324 (58.0%) AVP apps violated one or more disclosure requirements, and 61.1% of observed privacy-related data flows lacked proper disclosure.

    Source: §6.7 · p. 12

  • Takeaways & Limitations

    The findings highlight a need for improved privacy enforcement mechanisms in emerging XR platforms.

    Source: §9 · p. 14

    AVP lacks a ground-truth UI state identity, so effectiveness is evaluated by recovered privacy violations rather than state rediscovery.

    Source: §5.3 · p. 9

Abstract

from arXiv · show

XR devices introduce substantial privacy concerns due to their comprehensive data collection capabilities that surpass traditional computing platforms. While existing works have demonstrated privacy concerns on Android-based XR devices such as Meta Quest series by performing network traffic analysis, little attention has been paid to the Apple Vision Pro (AVP) devices, mainly due to the closed nature and the technical challenges associated with AVP devices. In this work, we make a bold attempt to detect privacy violations of AVP applications from network traffic through automatic testing on AVP devices. Our key insight is that effective AVP application testing requires coordinated control of both cyber (software) and physical (hardware) components, which we term Coordinated Cyber-Physical Testing. Building on this insight, we design and implement AVP-Inspect, an automatic dynamic analysis framework for AVP applications, overcoming significant challenges enforced by the closed-source nature of AVP ecosystem. AVP-Inspect consists of three components: an automatic device controller by building customized hardware devices, a 3D UI explorer by designing a new exploration engine, and a privacy violation detector by constructing a unified privacy taxonomy for AVP. We first evaluated AVP-Inspect on a manually constructed ground truth dataset, then performed a large-scale analysis on 324 AVP applications downloaded from the App Store, with each app tested for 20 minutes. We found that 188 (58.0%) of apps exhibit at least one violation, and more than 60% of the network traffic flows are not properly disclosed.

1 Introduction

AVP’s broad collection of biometric and behavioral data raises privacy concerns, yet its closed ecosystem has lacked comprehensive privacy analysis. AVP-Inspect combines hardware and software control to test applications and found widespread disclosure violations.

Motivation

  • AVP collects biometric and behavioral data, including eye movements, facial expressions, hand gestures, and full-body movements, which prior work has used to infer sensitive information.The cited examples include passwords, personal characteristics, and health conditions.Source: §1 · p. 1

Research gap

  • AVP’s closed architecture and restrictive policies leave comprehensive privacy analysis unanswered, while existing Quest methods cannot transfer because the platforms differ in hardware and software.The paper identifies barriers in automated device control, exploration of spatial UIs, and mapping traffic identifiers to AVP disclosures.Source: §1 · pp. 1–2

AVP-Inspect

  • AVP-Inspect coordinates software and hardware control through a customized device controller, a 3D UI exploration engine, and a privacy detector using a unified AVP taxonomy.The framework uses Bluetooth HID inputs and a digital-to-physical feedback pipeline for automated testing.Source: §1 · p. 2

Evaluation and contributions

  • The authors evaluated AVP-Inspect on a manually labeled 50-app dataset and analyzed 324 COTS AVP applications, reporting findings to Apple and app developers.Each app in the large-scale analysis was tested for 20 minutes.Source: §1 · p. 2

Findings

  • 188 apps with observable network activity had disclosure violations, and more than 60% of network traffic flows were not properly disclosed.The most common deficiencies involved privacy policies and labels; manifests were often incomplete or inaccurate, and SDKs accounted for a majority of non-disclosures.Source: §1 · p. 2

2 Background

The background describes AVP’s spatial-computing controls and the three privacy-document types used to disclose app data practices.

2.1 Apple Vision Pro

  • AVP combines eye tracking, hand gestures, and voice commands; its Digital Crown adjusts immersion and system view, while the Top button supports power, capture, and resets.The device runs visionOS, a specialized operating system for XR devices.Source: §2.1 · pp. 2–3

2.2 Privacy Documents in Apple Vision Pro

  • Policies describe how developers collect, use, and share data; App Store privacy labels summarize data types and uses, while privacy manifests provide machine-readable declarations for compliance checks.Vision Pro labels include spatial and biometric categories such as room mapping, scene mesh, and hand or eye movement; manifests are required with app binaries since May 2024.Source: §2.2 · p. 3

3 Overview

The overview frames AVP privacy compliance as comparing observed network traffic with app disclosures, then presents the technical challenges and AVP-Inspect’s coordinated solution.

3 Overview

  • The analysis defines a privacy violation as transmitting privacy-sensitive data externally without proper disclosure in an app’s privacy documentation.It considers privacy policies, privacy labels, and privacy manifests.Source: §3 · p. 3

3.1 Challenges

  • AVP testing is constrained by the lack of programmatic input interfaces, viewpoint-dependent 3D UI variation, and taxonomies that do not cover AVP data or its layered disclosures.These challenges complicate automated control, state deduplication, and matching traffic terms to policies, labels, and manifests.Source: §3.1 · p. 3

3.2 Overview of AVP-Inspect

  • AVP-Inspect coordinates cyber and physical control through Controller, Explorer, and Detector to automate AVP application privacy testing.Controller uses hardware-based HID input; Explorer recognizes and prunes UI states; Detector aligns traffic with privacy documents using an AVP-extended taxonomy.Source: §3.2 · p. 4

4 Design of AVP-Inspect

AVP-Inspect links device interaction, UI exploration, and traffic analysis in an automated workflow, using custom hardware to work around AVP’s lack of programmatic control.

4 Design of AVP-Inspect

  • Explorer captures mirrored AVP screens, directs Controller to open and explore target apps, and sends captured traffic with privacy documents to Detector for reporting.The workflow produces an app-specific report of violating traffic and the data involved.Source: §4 · p. 4

4.1 Controller: Simulating User Interactions

  • A customized ESP32 HID simulates Bluetooth keyboard and mouse inputs, while servo motors press AVP’s physical buttons for view centering and force-quitting apps.Physical button control helps reset the view and prevent overlapping app windows from confusing Explorer.Source: §4.1 · pp. 4–5

4.2 Explorer: Exploring AVP UI States

  • Explorer builds a state graph from detected UI elements and clickable transitions, then uses depth-first search to visit unexplored buttons and states.It recognizes similar states by comparing detected element positions and sizes, pruning viewpoint-driven visual variations.Source: §4.2 · pp. 5–6

4.3 Detector: Detecting Privacy Violations

  • Detector maps disclosures and traffic terms into a unified taxonomy, extracts HTTP key-value pairs, and flags data types present in traffic but absent from policies, labels, or manifests.The taxonomy extends VPVet with AVP-specific data types; keys are matched using synonyms, key patterns, and value patterns.Source: §4.3 · pp. 6–7

5 Evaluation

AVP-Inspect is evaluated against manually explored apps and then used to analyze a larger AVP app dataset. In 20-minute explorations, it recovers most ground-truth violations and produces substantially more requests than an idle baseline.

5 Evaluation

  • Each app was launched and explored for 20 minutes, with screen recordings and network captures segmented using app start and end timestamps.The AVP ran visionOS 2.1, and screenshots were sent from the MacBook to a Lambda server for OmniParserV2 processing.Source: §5 · p. 7

5.2 Privacy Violation Detection Results

  • 96.2% overall violation coverage—151 of 157 ground-truth violations across 31 apps with captured traffic in 20 minutes; Table 1 reports GT/Auto request and violation counts.Nineteen of the 50 evaluation apps produced no captured traffic during either manual or automated exploration.Source: Table 1 · p. 8, §5.2 · p. 8

5.1 Datasets

  • 50 manually explored apps across the top 10 categories form the ground truth; the remaining 324 apps support the large-scale analysis.Manual exploration covered interactive elements and required about 48 person-hours for the 50 apps.Source: §5.1 · pp. 7–8

5.2 Privacy Violation Detection Results

  • WWidgets had the lowest coverage at 50.0% (2 of 4), because two violations required widget-configuration navigation not reached within 20 minutes.The authors expect that more exploration time could cover additional violations.Source: §5.2 · p. 8

5.3 Effectiveness of Auto-exploration

  • 1,276 vs. 405 unique requests (3.15×) under Auto vs. length-matched Idle; 17 of 26 apps with traffic in both conditions had more under Auto.The authors conclude that exploration triggers traffic that cannot be observed without interaction.Source: §5.3 · pp. 8–9
  • The authors assess recovered privacy violations rather than UI-state recall because AVP has no view hierarchy or ground-truth state identity.Their perceptual deduplication heuristic can count the same functional screen differently depending on head pose, and repeated runs produce noticeably different state counts.Source: §5.3 · p. 9

6 Large-scale Analysis

Across 324 AVP apps, disclosure violations were widespread, with omissions and inaccuracies spanning policies, labels, manifests, SDK-driven flows, and declared purposes. Developer responses and the analysis of AVP-specific data further qualify these findings.

6 Large-scale Analysis

  • 188 of 247 apps with captured traffic had violations, and 61.1% of observed privacy-related flows lacked proper disclosure; privacy labels accounted for 1,785 violations.The analysis reports 1,529 policy violations and 138 manifest violations; Fig. 7 presents the distribution of violation traffic.Source: §6 · p. 9, Figure 7 · p. 10

6.1 Privacy Policy Non-Compliance

  • Policy analysis found 1,222 omitted disclosures, 29 incorrect disclosures, and 278 mismatched-entity disclosures, with many violations involving third-party services.Only 9.1% of omitted disclosures involved first-party collection; common omitted data included device identifiers, platform information, app version, and system version.Source: §6.1 · p. 9

6.2 Privacy Label Non-Compliance

  • Neglect accounted for 63.1% of privacy-label violations (1,127 instances), while contrary disclosures accounted for 36.9% (658 instances).Contrary disclosures affected 64 apps and 1,423 label-violating requests; third-party SDK-driven collection was more prominent than first-party collection in these violations.Source: §6.2 · pp. 9–10

6.3 Privacy Manifest Non-Compliance

  • Among 84 apps with manifests, 23 contradicted their declarations by transmitting data to tracking domains, while omissions included missing tracking endpoints and collected data types.The analysis reports 138 manifest violations overall: 23 contrary and 115 neglect.Source: §6.3 · p. 10

6.4 Non-Compliance due to Third-party SDKs

  • Analytics SDK violations predominated in Productivity and Entertainment, while Advertising SDKs drove more violations in Games; Productivity and Entertainment were the largest categories.The reported category sizes were 26 Productivity apps and 30 Entertainment apps.Source: §6.4 · p. 11, Table 2 · p. 11

6.5 Purpose Non-Compliance

  • Purpose omissions were concentrated in Analytics, especially for Other Diagnostic Data (50), Device ID (39), Gameplay Content (36), Precise Location (35), Name (33), and Phone Number (30).Common omissions included Other Diagnostic Data and Device ID; these data types could serve several purposes, including Analytics, App Functionality, Product Personalization, and Developer Advertising.Source: §6.5 · p. 11, Figure 9 · p. 11, Figure 10 · p. 11

6.6 Responses from Apple and Developers

  • Of 119 developers contacted, 10 responded: two acknowledged disclosure updates, four cited SDK documentation, five said existing labels sufficed, and four disputed classifications or reported flows.Apple acknowledged, validated, and reproduced the reported findings.Source: §6.6 · p. 12

6.7 Summary of Findings

  • The authors characterize the 188/324 app violation rate and 61.1% flow non-disclosure rate as conservative, and found no raw XR sensor measurements among 46 AVP-specific violations.Static analysis found 54 binaries declaring an XR sensor usage string and 39 linking an authorization-gated ARKit provider.Source: §6.7 · p. 12

7 Discussion

The discussion connects disclosure violations to profiling risks and the limits of self-reported privacy information. It recommends actions for users, Apple, and developers.

Implications

  • Identifiers shared with third parties can enable cross-app linkage and profiling, and the distinctive AVP install base may make linkage easier than on phones.The discussion notes these risks even when sensing streams remain on-device.Source: §7 · p. 12
  • AVP’s 58.0% app violation rate is comparable to OVRSeen’s 58.6% Quest rate, despite visionOS offering richer disclosures and manifests.The authors cite ecosystem immaturity and limited visibility into app behavior as factors behind inconsistent compliance.Source: §7 · pp. 12–13

Recommendations

  • Users are advised not to rely solely on labels or manifests and to review permissions, restrict sensitive access, and avoid apps with violation histories where appropriate.The authors also recommend favoring developers with transparent privacy practices.Source: §7 · p. 13
  • The authors recommend runtime verification and SDK audits by Apple, plus developer audits of SDK traffic and synchronized, updated privacy disclosures.Apple recommendations also include finer-grained AVP-specific taxonomy, enforcement checks, and distinct permissions and indicators for spatial and biometric sensing APIs.Source: §7 · p. 13

8 Related Work

Prior XR privacy and testing work often targets Meta devices or platforms with source access, standard testing suites, or instrumentation. AVP-Inspect extends privacy-compliance analysis to AVP’s closed platform.

XR privacy analysis

  • Prior XR privacy analyses studied Meta Quest privacy inconsistencies, VR app security, firmware, and age-appropriate data practices; this work analyzes AVP network traffic through automated testing.The cited work includes Ovrseen, VPVet, Guo et al., Simhadri et al., and Brunskong et al..Source: §8 · p. 13

Apple privacy labels

  • Prior Apple-label studies examined interpretation, developer mapping challenges, and label changes, while AVP-Inspect extends compliance analysis to Vision Pro apps.Related iOS work examined traffic-label inconsistencies using passive collection or UI-driven dynamic analysis.Source: §8 · p. 13

XR dynamic testing

  • Earlier XR testing tools commonly rely on source code, root capability, instrumentation, or standard testing suites, which the authors say do not apply to AVP.Unlike RoboTest, AVP-Inspect uses Bluetooth HID and servo motors; its relative, depth-dependent cursor motion requires feedback-guided control.Source: §8 · pp. 13–14

9 Conclusion

The paper presents AVP-Inspect for automated privacy analysis without root access or app source code, evaluates it on a ground-truth dataset, and analyzes 324 AVP apps. It reports the findings and offers recommendations to Apple and developers.

9 Conclusion

  • AVP-Inspect enables automated AVP privacy analysis without requiring root access or application source code.The authors evaluated it using a 50-app ground-truth dataset and analyzed 324 AVP applications, reporting non-compliance in 188 apps.Source: §9 · p. 14

A Ethical Considerations

  • The study used publicly available App Store information and traffic from apps installed on the researchers’ own AVP, without collecting user data.Testing sessions were voluntary and stopped if researchers experienced discomfort.Source: A Ethical Considerations · p. 16

D Algorithms

  • The algorithms appendix presents app-state exploration and discovery of new data types.Source: D Algorithms · p. 16

E Taxonomy of AVP-Inspect

F Runtime Performance of AVP-Inspect

AVP-Inspect averages 12.10 seconds per click, with virtual-cursor movement taking most of that time. Exploration depth and traffic patterns vary across app categories.

F Runtime Performance of AVP-Inspect

  • Broadcasts exceeds 70 clicks during its 20-minute session, while Foxar generates substantial network traffic relative to clicks.Wet Your Beak exceeds 40 clicks, and its traffic closely follows its click pattern.Source: F Runtime Performance of AVP-Inspect · pp. 16–17
  • Table 3 compares per-app per-minute unique network requests during AVP-Inspect exploration against an idle baseline for 31 apps with captured traffic.Source: Table 3 · p. 18
  • 12.10 seconds is the average time per click, of which 9.76 seconds is virtual cursor movement for visual calibration.The other cycle stages are UI-state recognition and state comparison; cursor movement is a necessary overhead of hardware-based targeting without system modifications or root access.Source: F Runtime Performance of AVP-Inspect · p. 18

G Large-Scale Analysis: Heatmaps

The section presents heatmaps of privacy-policy and privacy-manifest violations.

G Large-Scale Analysis: Heatmaps

H SDK Analysis

The SDK analysis points readers to figures on SDK-related violations. The accompanying material reviews XR privacy attacks and iOS privacy-compliance work, while noting that four ground-truth apps with immersive scenes generated no network traffic in manual analysis.

H SDK Analysis

  • Figures 17–19 present SDK-related violations referenced in the SDK analysis.The figures address privacy-policy, privacy-label, and privacy-manifest violations by app category and identify the receiving entity’s SDK category by color.Source: H SDK Analysis · p. 18, Figure 17 · p. 20, Figure 18 · p. 21

I Additional Related Work

  • Prior XR work studies identifying users from motion data [68] [72] [76] [77], inferring age and gender from head and hand movements [83], and injecting noise into motion data or anthropometrics to hinder inference attacks [95].Source: I Additional Related Work · p. 18
  • iOS privacy studies use static, dynamic, taint, and NLP techniques to examine sensitive-data access and compliance with privacy disclosures.This work differs by investigating privacy risks and compliance issues in AVP apps rather than traditional iOS apps and SDKs.Source: I Additional Related Work · pp. 19–20

J.1 Traffic Analysis for Immersive Scenes

  • Manual analysis found no network traffic from immersive scenes in the four ground-truth apps that AVP-Inspect could not explore.Source: §J.1 · p. 20

J.2 Limitations

Long-running tests found that unique requests largely saturated before the 20-minute budget, and the interception audit found no certificate rejections or undecryptable payloads. The section also describes AVP-Inspect’s limits, implementation, and possible extensions.

J.2 Limitations

  • AVP-Inspect depends on OmniParserV2 for clickable-object recognition, cannot simulate complex hand movements, and cannot reach account- or purchase-gated functionality.Manual exploration found such immersive scenes in 4 of 50 ground-truth apps, with no traffic from those scenes; 43 apps were fully explorable, and the method could cover 48/50. The inherited synonym list also includes “token” under “password,” so matched flows may be authentication-related rather than transmitted passwords.Source: §J.2 · p. 21

J.3 Future Work

  • The authors chose not to use existing GUI agents because smartphone-trained agents and ADB-dependent control do not fit AVP’s setting.They identify fuzz testing, performance testing, and VLM-based UI analysis as possible extensions, and propose replacing the Controller hardware for other devices.Source: §J.3 · p. 21

K Implementation of AVP-Inspect; K.1 Controller

  • The prototype comprises 286 lines of ESP32 code and about 3,000 lines of Python; its controller combines Bluetooth HID input with servos for physical buttons.The servos were selected to provide the approximately 350 g of force measured as necessary to press each AVP button.Source: K Implementation of AVP-Inspect · p. 21, §K.1 · p. 22

K.2 Explorer

  • The Explorer captures and parses screenshots, locates the virtual cursor, and compares detected UI states to guide exploration.It uses AirPlay capture and OmniParserV2; a one-minute unchanged-state timeout marks a state invalid and triggers app termination.Source: §K.2 · p. 22

K.3 Detector; L Detector Details

  • The Detector decrypts traffic through a trusted Mitmproxy certificate, then maps observed data terms to taxonomy nodes using synonym, key-pattern, and value-pattern matching.First-party destinations are identified by matching app, manifest, or policy domains; other destinations default to third-party, making first-party counts a lower bound. Policies and labels are compared separately with runtime behavior, not treated as ground truth.Source: §K.3 · p. 22, L Detector Details · p. 22

M Long-running Exploration

  • 96.6%, 95.6%, and 100% of the requests seen over three 60-minute sessions were already present at 20 minutes.The cumulative unique-request curves saturated before the 20-minute budget; Theater continued interacting after discovery plateaued, and exploration did not degrade over time.Source: Figure 20 · p. 23, M Long-running Exploration · p. 23

N TLS Interception Coverage

  • All 336 distinct destination hosts yielded successfully decrypted requests, and none of 201 errored flows showed certificate-pinning behavior.Of the errored flows, 165 originated from transient local-proxy-hop failures and the remainder were ordinary mid-stream resets; silent evasion remains unobservable, so measurements are treated as a lower bound.Source: N TLS Interception Coverage · p. 23

N.1 Case Studies

The case studies illustrate Contrary Disclosure, where AirLauncher’s collection contradicts its “Data Not Collected” label, and Neglect Disclosure, where Twin’s limited label omits extensive collection practices.

N.1 Case Studies

  • The two cases illustrate Contrary Disclosure and Neglect Disclosure violations in AVP applications.AirLauncher’s label contradicted its collection, while Twin’s label did not specify the categories collected.Source: §N.1 · p. 23
  • AirLauncher transmitted 9 privacy-related data items to RevenueCat despite its “Data Not Collected” label, directly contradicting the app’s stated disclosure.The items included a device identifier, app fingerprint, usage behavior, and performance metrics; RevenueCat can support cross-app attribution and identifier linkage.Source: §N.1 · p. 23
  • Twin’s “Data Not Linked to You” label omitted categories despite 8 undisclosed practices spanning 4 privacy categories and two third-party SDKs.The categories were Identifiers, Location, Usage Data, and Diagnostics; the SDKs were RevenueCat and Paywalls.Source: §N.1 · p. 23
  • Twin’s visionOS-specific traffic headers identified the device as Apple Vision Pro, enabling platform-specific user profiling.The headers included x-platform: visionOS and x-platform-version: Version 2.1.Source: §N.1 · p. 23
Loading 2609.08103v2…