Source-linked AI summary
SoK: Motion Data Privacy in Extended Reality
Azim Ibragimov, Alina Vasina, Uliana Polshcha, Eric D. Ragan
TL;DR
XR motion tracking enables immersive interaction but exposes patterns that can reveal identity and sensitive attributes, while prior surveys treated motion across broader privacy domains. This SoK reviews 134 papers using a threat model and taxonomy, finding dominant recognition research alongside gaps in modality, risk, and evaluation coverage. It synthesizes attacks and defenses and provides recommendations for future XR motion privacy evaluations.
Problem
XR motion is increasingly easy to collect and can reveal identity and sensitive attributes, while prior XR privacy work did not treat motion as a distinct organizing dimension.
Method
The SoK systematically reviews 134 publications and codes them with an XR motion threat model and taxonomy covering modalities, representations, risks, adversaries, attacks, and defenses.
Results
Recognition dominates the 62 attack papers with 49 studies, while no study evaluates all five privacy risks or all five motion modalities simultaneously.
Takeaways & Limitations
The review identifies gaps in modality and risk coverage, multimodal evaluation, physical defenses, candidate-population sizes, and evaluation consistency, and recommends improved future evaluations.
Takeaways & Limitations
The literature has limited research on facial and full-body motion and on privacy risks beyond recognition.
Abstract
from arXiv · showhide
Extended Reality (XR) provides immersive, interactive 3D experiences. To enable these experiences, the devices must track user motion so the system can respond to actions such as grabbing, looking at, or moving an object. However, motion tracking has raised privacy concerns since it records a person's motion patterns. These motion patterns have been studied extensively across various fields (i.e., gait identification and profiling) and have been shown to reveal sensitive information. With the adoption of XR, these patterns became easier to record and obtain than ever. This creates a fundamental privacy tension: motion tracking enables core XR functionality yet requires users to compromise their privacy. Prior systematization-of-knowledge (SoK) studies on XR privacy have examined the field broadly, with motion-related research distributed across several privacy domains rather than treated as a distinct area of study. However, XR motion privacy has gained significant momentum since the prior SoK, with the literature nearly quadrupling in size and thereby warranting a dedicated systematization of this topic. This SoK examines 134 relevant papers on privacy concerns in motion patterns recorded by XR headsets, including how adversaries can obtain users' motion patterns, the inferences they can draw from them, and methods for protecting users. Based on this review, we synthesize a taxonomy of motion modalities, representations, and inference risks; develop an XR motion threat model; systematize the attack and defense approaches in the XR motion literature; identify gaps in the literature; and provide guidelines for future studies evaluating motion privacy mechanisms. Together, our SoK clarifies the state of XR motion privacy and provides recommendations for future evaluations.
1 Introduction
XR depends on continuous motion tracking for interaction, but those movements remain observable and can reveal identity and sensitive attributes. The rapid expansion of motion-focused research motivates treating XR motion privacy as a dedicated area.
- Motivation: XR systems continuously track head, eye, hand, body, and face signals to support immersive interaction.These signals enable actions including looking, navigation, object manipulation, social interaction, and avatar control.
- Privacy risks: Motion patterns remain observable through ordinary interactions and application APIs even when users conceal avatars and usernames.Changing visible identity cues does not remove the movement patterns associated with a person.
- Privacy risks: Motion data can reveal users’ identities, demographics, health information, cognitive characteristics, and behavioral traits.These inferences extend beyond recognition to sensitive personal attributes.
- Contribution: This SoK organizes 134 publications using a motion-focused threat model and taxonomy, identifies evaluation gaps, and releases review artifacts for reproducibility.Its recommendations address future defense design and evaluation practices.
- Motivation: XR reduces historical barriers to collecting high-quality motion data by sensing movement near the user and exposing motion through uploads and avatars.This has enabled large-scale XR motion datasets and broader access to recorded movement.
- Research gap: Prior XR privacy SoKs distributed motion research across broader privacy domains, while newer work has nearly quadrupled into a larger, more diverse body centered on motion.The expanded literature includes new modalities, inference targets, threat models, defenses, and evaluation practices.
3 Method: Data Collection & Analysis
The authors conducted a structured, hybrid literature review using database, manual, and citation searches, then screened and systematically coded the resulting XR motion privacy corpus. The final corpus contained 134 publications and was analyzed through an iteratively developed threat model and taxonomy.
- Review process: Three researchers performed a structured review, iteratively discussing interpretations, screening decisions, and coding differences.The review covered search, inclusion, screening, analysis, and coding stages.
- Search strategy: The hybrid search combined database searches, manual searches, and backward and forward citation chasing.Citation chasing was applied to included papers and used the same screening criteria for newly identified studies.
- Search strategy: Automatic searches covered XR, biometrics, and security venues using broad motion- and privacy-related terms across supported metadata fields.The search had no start-year restriction and included IEEE Xplore, ACM databases, ScienceDirect, and Elsevier.
- Screening: The inclusion criteria required XR relevance, motion data, and a privacy risk, attack, defense, system design, or inference method.General sensing research without a clear immersive-system connection was excluded.
- Corpus: The final corpus reached 134 publications, including 100 published after Garrido et al.’s August 2022 cutoff.The corpus contained 34 relevant publications before that cutoff, and the literature nearly quadrupled afterward.
- Analysis: The authors first identified recurring concepts and evaluation practices, then used the resulting threat model and taxonomy as a codebook.Coding covered modalities, exposure points, representations, risks, datasets, defenses, utility evaluations, and open-science artifacts.
4 XR Motion Threat Model
The threat model follows motion from users through XR systems to adversaries that access raw, real-time, or recorded data. It also distinguishes physical, stream, and aggregate intervention points, each with different protection boundaries and trade-offs.
- Adversaries: Motion can reach adversaries through ordinary XR sensing, interaction, or data sharing without a traditional breach.The model therefore treats access as part of the normal motion pipeline rather than requiring stolen credentials or intercepted traffic.
- Adversaries: XR device adversaries access raw motion directly, real-time adversaries access live streams, and post-hoc adversaries access recorded traces.Their access points differ in data quality, traceability, and the information preserved in available traces.
- Privacy types: Physical Privacy intervenes before sensing by altering movement or interfering with sensors, potentially protecting against all three adversary types.Its costs include discomfort, fatigue, behavioral effort, reduced tracking precision, and degraded interaction quality.
- Privacy types: Stream Privacy modifies motion after capture but before live exposure through application APIs or avatar behavior.It supports ordinary interaction without physically altering movement but cannot protect against device adversaries.
- Privacy types: Aggregate Privacy transforms stored or released traces after collection, enabling computationally expensive methods or formal guarantees such as differential privacy.It does not protect against device or real-time adversaries that already accessed the original motion.
5 Taxonomy of XR Motion Data Privacy Risks
The taxonomy organizes XR motion privacy around five modalities, their representations, and associated risks. Head, eye, and hand movements span all five risk categories, whereas facial and full-body motion have narrower coverage in the literature.
- Motion modalities: The taxonomy groups XR motion data into head, eye, hand, facial, and full-body modalities.It links each modality to representations used in XR systems and privacy attacks.
- Representations: Motion representations include features such as position, orientation, velocity, acceleration, gaze position, fixations, saccades, pupil diameter, blink rate, and scanpaths.The specific representations vary by modality and application.
- Privacy risks: The taxonomy identifies recognition, demographics, health and physiology, cognitive and affective states, and behavior, intent and preference as five privacy-risk categories.These categories cover identity linkage and inference of personal, physiological, mental, and behavioral attributes.
- Coverage: Head, eye, and hand movements cover all five privacy-risk categories, while facial and full-body movements have been studied across fewer risks.The taxonomy supports coding the corpus by modality, risk, adversary, and privacy type.
6 XR Attacks
The attack literature is concentrated on head, hand, and eye motion, with recognition dominating studied privacy risks. Motion data and multimodal access support broader inference capabilities, while simultaneous coverage of all modalities remains unexplored.
- Motion modalities: Head movement is the most frequently exploited modality, followed by hand and eye movement, while full-body and facial movement are rarely studied.Across 62 attack papers, head movement appears in 40, hand and eye movement in 28 each, full-body movement in 3, and facial movement in 1.
- Privacy risks: Recognition is the most studied privacy risk, far exceeding health, demographic, cognitive, and behavioral inference risks.Recognition appears in 49 papers, compared with 8 each for health and physiology and demographics, 7 for cognitive and affective states, and 3 for behavior, intent, and preference.
- Cross-risk inference: The same motion data can support multiple privacy inferences because supervised-learning pipelines can often retain the representation and algorithm while changing prediction labels.Seven papers consider at least two privacy risks, including repurposing recognition models for demographic prediction.
- Multimodal attacks: Multimodal access is common, with 30 papers evaluating at least two modalities and head-plus-hand motion the most frequent exact combination.The largest single-study combination includes four modalities: eye, head, hand, and facial movement.
- Multimodal attacks: Multimodal attacks can outperform single-modality attacks, and protecting only one stream leaves adversaries able to use another exposed stream.No corpus study considers all five motion modalities simultaneously, creating a gap for attack and defense research.
7 XR Defense
The defense literature is concentrated on eye, head, and hand motion and on stream and aggregate privacy, while physical privacy and broader modality coverage receive less attention. Evaluation practices vary substantially across candidate-set sizes, adversary assumptions, utility measures, user studies, and artifact release, limiting comparison and reproducibility.
- Protected modalities: Eye motion (28 papers), head motion (27), and hand motion (21) are the most frequently protected modalities, whereas full-body motion (7) and facial motion (2) receive substantially less attention.The distribution closely follows the attack literature, where eye, head, and hand motion are also most frequently studied.
- Privacy types: Stream privacy (27 papers) and aggregate privacy (20) dominate defense research, while physical privacy appears in only 3 papers.Physical privacy is therefore substantially less studied than the other two privacy types.
- Evaluation practices: Candidate populations range from 4 to 56,082 participants, with a median of 61 and a mean of 1534, complicating comparisons of identification accuracy.A 50% accuracy result can equal random guessing with two candidates but greatly exceed chance with 55,000 candidates.
- Evaluation practices: Only 5 defense papers evaluate both oblivious and adaptive adversaries, compared with 26 evaluating only oblivious and 11 only adaptive adversaries.Adaptive adversaries consistently achieve higher attack success in prior work, so adversary assumptions affect reported defense performance.
- Evaluation practices: Among 30 physical and stream defenses, only 10 include a user study, even though these defenses directly affect interaction with the XR system.Task, spatial, and temporal utility measures cannot fully capture comfort, perceived quality, or usability.
- Open science: Only 18 of 50 defense papers provide a publicly available artifact, limiting reproducibility, comparison under common settings, and future extensions.Artifacts can include privacy-mechanism implementations, attack and evaluation code, model configurations, or trained models, even when sensitive datasets cannot be released.
8 Guidelines for XR Motion Privacy Research
The guidelines tailor data collection, utility evaluation, adversary evaluation, and open-science practices to the privacy type and defense setting.
- Data Collection: Data collection should match the privacy type: dedicated datasets generally suit physical and stream defenses, while public datasets generally suit aggregate defenses.New datasets should record available modalities and relevant labels; public datasets should be assessed for modalities, labels, and participant counts.
- Utility Evaluation: Physical and stream defenses should be evaluated with user studies and real-time utility proxies, while aggregate defenses emphasize task utility.Suggested proxies include spatial and temporal precision; application-specific examples include in-game scores and area-of-interest accuracy.
- Adversary Evaluations: Studies should evaluate defenses against both oblivious and adaptive adversaries whenever possible.Multimodal evaluations can use one modality for an oblivious adversary and multiple modalities for an adaptive adversary.
- Adversary Evaluations: Defense results should be reported across multiple candidate population sizes to show how performance generalizes as the population varies.The paper gives L=5 through L=300 as an example range for a 300-participant study.
- Open Science: Researchers should release datasets and code whenever possible to improve reproducibility, comparison, and reuse.Code clarifies configurations, preprocessing, and evaluation procedures, while datasets support later studies of additional risks, modalities, and defenses.
9 Opportunities for Future Work
Future work should improve comparability through shared evaluation frameworks, broaden multimodal and multi-risk studies, and give greater attention to physical defenses.
- Reproduction Studies and Evaluation Frameworks: Reproduction studies and evaluation frameworks should test mechanisms under common adversary, population-size, user-study, artifact, and metric settings.These efforts aim to determine whether reported privacy gains hold across evaluation conditions and improve comparability and reproducibility.
- Broader Multimodal and Multi-Risk Evaluations: Multimodal and multi-risk evaluations should combine large participant populations, all five motion modalities, and labels supporting multiple privacy risks.Such evaluations can examine generalization across modalities and inference targets and evaluate defenses under multimodal exposure.
- Physical Defenses: Physical defenses warrant more research because they can protect motion before it reaches the XR device.Future studies should examine broader physical interventions across different motion modalities and privacy risks, especially when the headset or platform is untrusted.
10 Conclusion
The SoK organizes 134 publications into a motion-centered threat model and taxonomy, identifies major gaps, and offers evaluation and research directions for more comprehensive XR motion privacy work.
- 10 Conclusion: The SoK covers 134 publications and characterizes motion exposure, modalities, representations, inferable risks, and defense intervention points.It also identifies gaps in modalities, risks, multimodal evaluation, physical defenses, adversary assumptions, candidate populations, user studies, and artifacts.
- 10 Conclusion: The paper provides guidelines for data collection, utility and adversary evaluation, open science, and future research.These recommendations are based on the gaps identified across the systematization.
Ethical Considerations
The work is a literature systematization and did not involve new human-subject research or secondary analysis of existing human datasets.
- Ethical Considerations: The review analyzed published papers without recruiting participants, collecting new human-subject data, deploying a system, observing live XR users, or analyzing existing human datasets.Because it was a literature review, the authors did not seek new IRB review.
Open Science
The SoK supports reproducibility by releasing its review materials and maintains a living corpus for incorporating eligible future papers.
- The authors provide the screened-paper list, exclusion decisions, final corpus, and coding sheet in an anonymous repository.
- A webpage allows researchers to submit newly published papers for review under the SoK’s inclusion criteria.Eligible submissions are added to the online corpus alongside papers identified in the review.
AI Use
The authors used generative AI for limited manuscript editing and for illustrative visual elements, with human review and privacy-conscious image choices.
- Generative AI tools revised manuscript text for clarity, flow, grammar, spelling, and typographical errors, with all edits manually verified.The authors state that AI use did not extend to the bibliography and retain responsibility for the final paper.
- Image-based generative AI created illustrated characters and backgrounds for teaser and taxonomy figures instead of photographs of real people.The choice aimed to avoid privacy concerns and possible links to identifiable individuals.