Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data

1 Introduction

• Gaming drives VR adoption, yet the study examines whether spatial telemetry from gaming sessions can identify an otherwise anonymous player.The introduction notes that 91 of the 100 most popular VR applications were games as of early 2023.Source: §1 · p. 1
• The dataset contains over 2.5 million recordings from 55,541 real VR users across 40+ countries and 20+ device types.It is presented as more than 100 times larger than the largest prior result.Source: §1 · p. 1
• In a pool this large, static attributes such as height are insufficient; identifying users requires behavioral differences in movement patterns.The authors contrast this with identification in sets of 511 users or fewer, where static attributes can suffice.Source: §1 · p. 1
• The approach combines application-context-aware featurization with hierarchical classification to support identification across 50,000+ classes.The authors describe the hierarchical model as scalable to this class count.Source: §1 · p. 1
• 94.33% identification accuracy across 55,541 users, with 73.20% accuracy from 10 seconds of motion among 50,000+ candidates.The reported 94.33% result uses 100 seconds of motion; the study also provides detailed explainability results.Source: §1 · p. 1

2.1 VR Information Flow

• VR devices produce telemetry that applications use to render immersive stimuli, and multiplayer applications may share it with external systems that could infer private information.The paper focuses on head and hand motion telemetry: 18 tracked dimensions, usually sampled 60–144 times per second.Source: §2 · p. 2, §2.1 · p. 2

2.2 VR Threat Model

• The study targets game servers and other users, who receive filtered telemetry remotely and are among the harder adversaries to detect.The threat model treats each entity able to view a target user's telemetry as a potential adversary.Source: §2.2 · p. 2

2.3 Beat Saber

• Beat Saber maps present timed challenges: players hit colored, angled blocks, avoid bombs with their sabers, and avoid walls with their heads.Maps may use base-game, purchasable, or user-created tracks.Source: §2.3 · pp. 2–3

3 Dataset

• The dataset comprises 2,669,886 replays from 55,541 users across 713,013 play sessions, obtained with BeatLeader administrators.Replays ranged from 5 seconds to over an hour, with a median length of 2 minutes and 56 seconds.Source: §3 · p. 3

3.1 Replay Format

• Beat Saber Open Replay files contain metadata, head-and-hand telemetry, in-game stimulus context, and measures of response validity and accuracy.Telemetry is recorded at each rendered frame, usually 60–144 times per second.Source: §3.1 · p. 3

3.2 User Attributes

• Replay metadata includes device details and user-selected settings such as height and handedness; replay counts and relative performance indicate experience and skill.The identification models exclude metadata, though the authors use it to examine correlations with identification accuracy.Source: §3.2 · p. 3

3.3 Ethical Considerations

• BeatLeader users voluntarily installed the extension and shared replays, which were already publicly viewable to other users; the study used no originally collected data.The study was deemed IRB-exempt under 45 C.F.R. § 46.104(d)(4)(i).Source: §3.3 · p. 3
• The authors withheld the complete raw dataset and did not attempt to infer especially sensitive information such as health data.They offer deidentified, normalized data to qualified researchers upon reasonable request.Source: §3.3 · p. 3

4.1 Motion

• Motion studies used body tracking to identify people or infer attributes such as gender, age group, and skeletal structure.VR typically provides only three tracked locations, compared with the 8–40 used in the cited studies, leaving applicability to VR uncertain.Source: §4.1 · pp. 3–4

4.2 Virtual Reality

• Earlier VR studies identified users in laboratory pools of 16–511; Miller et al. reported 95% accuracy among 511 users using five minutes of telemetry.Other cited studies included 22 users with up to 40% accuracy, 16 users with 90% accuracy, and 30 VR users with 95% accuracy.Source: §4.2 · p. 4
• 55,541 users and 713,013 sessions make this study at least two orders of magnitude larger than the largest prior result, with real users in 40+ countries and 20+ device types.The authors report comparable accuracy despite the larger identification pool and uncontrolled environments.Source: §4.2 · p. 4

4.3 Machine Learning

• Gradient boosting performed best among the discussed classical methods, while simple deep-learning models performed poorly for this study’s more than 50,000 classes.The authors note that larger, more sophisticated deep-learning models could perform strongly in the future.Source: §4.3 · p. 4

5 Featurization

• Sessions are continuous replay sequences with no more than 10 minutes between replays; 70% were reserved for training, 10% for validation, and 20% for testing.The authors state that this split makes evaluation cross-session rather than dependent on session-specific features such as room position.Source: §5 · p. 5

5.1 Guiding Principles

• Different users show distinct motion responses when slicing the same blocks, motivating features that retain user differences while reducing telemetry dimensionality.The authors say observed differences may reflect physiology, learned motion patterns, random variance, or a combination.Source: §5.1 · p. 5

5.2 Motion Features

• 80.1% per-sample and 96.6% per-user accuracy followed from encoding orientation with four quaternion elements rather than three Euler angles.The five summary statistics over 18 motion streams yielded a 90-dimensional vector and 69.3% per-sample and 93.4% per-user accuracy before that modification.Source: §5.2 · p. 5

5.3 Context Features

• 14.8% per-sample and 43.8% per-user accuracy came from the 22 contextual features alone, which the authors say are insufficient without motion features.The features characterize a response to a block, including its properties and the cut’s angle, speed, location, accuracy, and relative space-time error.Source: §5.3 · p. 6

5.4 Hybrid Featurization

• 83.8% per-note and 98.2% per-user identification accuracy resulted from combining 105 motion features with 22 context features in a 127-dimensional vector.The vector summarizes one second of motion centered on contact with a block.Source: §5.4 · p. 6
• Combining rich event context with separate motion features from before and after the target event effectively achieved accurate identification on datasets significantly larger than 500 users; this 232-dimension hybrid featurization was used in subsequent models.Source: §5.4 · pp. 6–7

6.1 Algorithm Selection

• 93.2% was LightGBM’s best per-sample identification performance, exceeding XGBoost at 80.0% and the other tested classical algorithms.The authors selected LightGBM for subsequent models; tested deep-learning approaches also underperformed it, with GRU at 84.0%, LSTM at 83.0%, and MLP at 72.0%.Source: §6.1 · p. 7

6.2 Preprocessing Method

• 93.2% was the result for StandardScaler, the best-performing preprocessing method among the five evaluated.The authors describe StandardScaler as removing each feature’s mean and scaling it to unit variance.Source: §6.2 · p. 7

6.3 Hierarchical Approach

• Training time and memory for a single LightGBM classifier scale quadratically with class count, making a 50,000-class model infeasible on the dataset.A projection from classifiers trained with up to 5,000 users estimated over seven days and nearly 4 TB of RAM for 55,000 users.Source: §6.3 · p. 7
• The first two hierarchical layers each use 10 classifiers trained on about 5,000 users, then redistribute classes so their probability outputs can be combined.The final identity from those layers is based on the highest logarithmic sum of class probabilities.Source: §6.3 · p. 8
• A third layer trains models for connected components of users confused with one another, applying the corresponding model when an initial identity belongs to a component.The components are formed from a graph linking users misclassified as similar on an independent clustering set.Source: §6.3 · p. 8

6.4 Scalability

• Layer models can train in parallel for a 10–20x training-time reduction on a cluster, and adding a user requires retraining only one model per layer.Testing and inference can also be parallelized by evaluating models separately.Source: §6.4 · p. 8

7 Evaluation

• Testing required over 90 million classifications and about eight hours, but an individual identification took less than a second.The evaluation trained on 150 full-hybrid samples per user and split sessions 70/10/10/10 across training, clustering, validation, and testing.Source: §7 · p. 8

7.1 Results

• 94.33% identification accuracy is reached with 50 samples (100 seconds); a single layer achieves 90.1%, compared with 94.3% for the hierarchy.Table 2 reports accuracy at 50 test samples per user.Source: §7.1 · p. 9, Table 2 · p. 9
• 48.45% accuracy is achieved from 2 seconds of telemetry, 73.20% from 10 seconds, and 92.78% from one minute, with diminishing returns thereafter.The full 94.33% is reached with 100 seconds of data.Source: §7.1 · p. 9, Figure 11 · p. 9
• 97.25% of instances place the correct user among the model’s top three candidates.Source: §7.1 · p. 9

7.2 Impact Factors

• The impact-factor analysis used device and demographic metadata, which the authors excluded from the motion-only identification model.Source: §7.2 · p. 9
• Identification varied by device and country: Windows Mixed Reality users were less conducive to identification, while users from Japan and South Korea exceeded 99% accuracy.The authors suggest tracking quality may help explain device differences and detectable play-style differences may contribute to country differences.Source: §7.2 · p. 9, Figure 12 · p. 9
• Users with 100 or more replays were identified with over 99.5% accuracy, while users with five or fewer were significantly harder to identify.Replay count was the strongest factor regardless of the number of training samples; the authors suggest experienced players develop more repeatable movement patterns.Source: §7.2 · pp. 9–10, Figure 13 · p. 10

8 Explanations

• The eight most important physical-measurement features account for 6.8% of splits and 10.2% of gain, providing about 12 bits of entropy—enough to identify as many as 4,000 users.The top two measure maximum headset Y-position, and the next six appear to measure outstretched arm length.Source: §8 · p. 10, Figure 14 · p. 10
• Static physical features remain consistent across samples and sessions, while the importance of motion features can vary with sample context.Source: §8 · p. 10
• Motion features account for 73.9% of entropy gain, compared with 22.9% for static measurements and 3.2% for contextual features.The authors conclude that identifying over 50,000 users would not have been possible without motion features.Source: §8 · p. 10, Figure 15 · p. 10

9 Discussion

• Multiple accounts or devices may place one person in distinct classes, potentially making identification accuracy appear lower.Source: §9 · p. 11
• The session definition usually separates training and testing across different days, unlike Miller et al.’s same-day sessions.The authors characterize identification across wider periods as a more difficult task than correlating motion segments recorded in rapid succession.Source: §9 · p. 11
• The authors argue that identifying 50,000 users required actual movement patterns contextualized relative to virtual objects in repeatable activities.They contrast this with simpler physical measurements that can identify tens or hundreds of users in relatively static tasks.Source: §9 · p. 11

9.1 Limitations

• Beat Saber-specific features limit what can be concluded about generalization to other VR applications; height and handedness ground truth is partly self-reported.The authors say generalization requires further investigation and note that players may deliberately misconfigure height settings.Source: §9.1 · p. 11
• Deep learning empirically underperformed tree-based models, which the authors considered sufficient for their privacy focus and useful for model explanations.Source: §9.1 · p. 11

9.2 Future Work

• Future work could apply deep learning, especially transformer-based sequential models, potentially with distributed learning and more efficient techniques.Source: §9.2 · p. 11
• Suggested VR-game applications include cheating detection, score prediction, skill-based matchmaking, and map recommendations.Source: §9.2 · p. 11
• Future work could use surveys to infer user attributes from telemetry and develop replay-corruption techniques that obscure identifiable properties without impeding uses such as scoring.Source: §9.2 · p. 11

10 Conclusion

• A few seconds of head and hand motion can uniquely identify users, a scale of identification the authors say had been difficult to demonstrate before large VR datasets emerged.Source: §10 · p. 11
• The widespread adoption of VR has brought datasets with motion from tens of thousands of users into view for studying identification at scale.Source: §10 · p. 11
• VR telemetry essential to platform operation should be considered highly sensitive because it may reveal extensive information about users.The authors call for research into privacy-preserving technologies that enable VR use without revealing private information.Source: §10 · p. 11

• The featurization, normalization, training, and testing scripts, along with detailed logs, outputs, and results, are available in a public GitHub repository.Source: Availability · p. 12
• The full training dataset is not published for privacy and logistical reasons; researchers may contact the authors to request a copy for replication.Source: Availability · p. 12

• 55,541 users are represented, including 14,945 (26.9%) with ≤5 replays and 5,450 (9.8%) with ≥100 replays.Source: B Participant Distribution · p. 14
• SteamVR accounts for 42,035 users (75.7%) and OpenVR for 42,039 (75.7%); Oculus accounts for 11,269 users (20.3%) on the platform breakdown.Source: B Participant Distribution · p. 14
• Oculus Quest 2 (Standalone) is the largest listed headset group at 25,857 (46.6%), while Oculus Quest Controller is the largest listed controller group at 16,449 (29.6%).Source: B Participant Distribution · p. 14
• Most participants are right-handed (53,144; 95.7%), and the largest height range is 1.7 m–1.8 m (18,495; 33.3%).Source: B Participant Distribution · p. 14
• The country breakdown lists 55,541 total participants; among individually named countries, the US has the highest count at 15,142 (27.3%), followed by DE at 2,404 (4.3%).Source: B Participant Distribution · p. 14