Source-linked AI summary
MultiGait: A Multi-Sensor Multi-Perspective Multi-Session Biometric Inference Benchmark and its Dataset
Julian Todt, Felix Morsbach, Philip Dissert, Thorsten Strufe
TL;DR
Research on privacy risks from thermal, depth, lidar, and other smart-city sensors has been constrained by insufficient multi-sensor, multi-session evidence and unsubstantiated privacy-friendly claims. MultiGait provides a synchronized gait dataset and benchmark spanning sensors, perspectives, and sessions. The benchmark finds high identity-inference potential across tested technologies in single-session experiments, while current recognition systems do not reliably generalize across sessions.
Problem
Existing datasets do not adequately support rigorous comparisons of privacy risks across smart-city sensors, despite claims that some alternatives are more privacy-friendly.
Method
MultiGait records 199 individuals across eight sensing technologies, four perspectives, and multiple sessions in a synchronized, full-factorial gait-focused dataset for benchmarking identity inference.
Results
All tested sensing technologies showed high identity-inference potential in single-session experiments, while existing gait-recognition systems did not reliably generalize across sessions.
Takeaways & Limitations
Sensors presented as privacy-friendly can still entail substantial identity-inference risk, and multi-session evaluation exposes an important recognition research gap.
Takeaways & Limitations
The dataset is restricted to a lab environment and does not represent the entire population, being limited to able-bodied individuals and biased in ethnicity and age.
Abstract
from arXiv · showhide
A lack of suitable datasets has limited the research into the privacy risks of novel smart city sensors, such as thermal cameras, depth cameras, and lidar. Given the number of unsubstantiated privacy claims and their potential widespread deployment into many people's everyday life, understanding the privacy risks of these sensors -- in isolation and in like-for-like comparisons -- is crucial. With MultiGait, we collected the first multi-sensor, multi-perspective, multi-session gait-focused dataset, for the corresponding, and additional more far-reaching investigations. The dataset, validated with multiple state-of-the-art recognition systems, comprises various walking modes and annotated personal attributes for 199 individuals, to ensure the benefit for advanced studies including cross-sensor recognition and anonymization at the edge. MultiGait represents a foundation for rigorous privacy investigations, demonstrated through an extensive identity inference benchmark across eight sensors, four perspectives, and three recording sessions. Our benchmark incidentally reveals that sensors often assumed to be privacy-friendly do still entail considerable identity inference risks, while the poor cross-session generalization of existing methods underscores an important research gap.
1 Introduction
Public sensors capture biometric information that can support identity and sensitive-attribute inference, while claims that alternative sensors are more privacy-friendly remain insufficiently substantiated. MultiGait addresses the lack of comprehensive comparisons with a multi-sensor, multi-perspective, multi-session dataset and benchmark.
- Gait recordings can support inference of identities, activities, and sensitive attributes, making public-space sensors potential surveillance equipment.
- Existing datasets do not adequately support rigorous sensor comparisons because many are small, single-sensor, or single-session.
- 199 individuals were recorded synchronously across video, depth, thermal, mmWave radar, lidar, and WiFi sensors from four perspectives, producing 32 information sources per individual.
- MultiGait includes five activities, five poses, personal-attribute annotations, and repeated sessions for 32% of subjects, enabling identity, activity, and attribute inference studies.
- The benchmark uses state-of-the-art recognition systems to measure identity-inference accuracy across sensors, perspectives, and single- versus multi-session settings.
2 Background
Biometric data can reveal identity, attributes, and activities, creating privacy concerns when collected or processed without consent. Alternative sensors have been proposed to retain utility while reducing the identification risks associated with video cameras.
- Biometric data comprises behavioral or physiological information that can be used to infer identity, personal attributes, or current activity.
- Face and gait are biometric traits, and gait is described as particularly distinctive.
- Deep-learning systems have achieved high-accuracy identification from visual-light video using facial and gait information.
- Depth and thermal cameras, mmWave radar, lidar, and WiFi sensing have been proposed as alternatives intended to reduce privacy risks while preserving utility.
3 Related Work
Existing gait datasets rarely provide concurrent multi-sensor recordings, multiple sessions, and varied perspectives under controlled conditions. These gaps limit like-for-like privacy and utility comparisons, cross-sensor research, and realistic evaluation of recognition systems.
- Existing dataset surveys focus on the relatively small set of gait datasets containing multiple sensors, perspectives, or sessions.
- Concurrent recordings of multiple sensors are especially rare, with prior examples generally covering only two or three sensor types.
- Single-sensor datasets make it difficult to compare identification potential between sensors under otherwise unchanged circumstances.
- Only a few existing datasets record individuals across multiple sessions, while most use the same session for training and testing.
- Most datasets lack varied perspectives, and uncontrolled settings can make annotations imprecise and obscure the effects of environmental variation.
4 MultiGait Dataset
MultiGait is a full-factorial, multi-sensor, multi-perspective, multi-session dataset designed for controlled comparisons of sensing technologies and inference tasks.
- Dataset design: 199 participants contributed 120 walk samples recorded with every sensor, perspective, and session, yielding 32 information sources per individual and session.
- Sensors: The dataset includes eight sensor types: video, depth, NIR, LWIR, lidar, radar, CSI, and BFI.Including multiple similar sensing technologies supports fine-grained utility and privacy comparisons.
- Recorded behaviors: Each participant performed five poses, including standing, arms-forward standing, smartphone holding, crouching, and pretending to tie shoes.
- Recording setup: The controlled lab setup supports reproducible evaluation of multi-session inference and variation across the dataset’s dimensions.A lab setting was chosen because many sensors lack adequate evaluation even under controlled conditions.
- Perspectives: Every sensor type was recorded from four perspectives, including center-low, center-high, left, and right locations.Center-low is the hip-height baseline perpendicular to the walking line; the other perspectives include an overhead location 2.2m above ground.
- Inference validation: Activity inference exceeded 90% accuracy for all imaging-based sensors and lidar, while radar, CSI, and BFI remained above the 25% chance level.The reported disparity with related work motivated an identity-based split analysis.
- Inference validation: Identity-separated training and testing produced significantly lower activity-inference accuracy, indicating limited generalization beyond specific individuals.Within-identity splitting increased accuracy for CSI and BFI, bringing results closer to related work.
- Attribute inference: Gender was robustly inferred from all imaging-based sensors and lidar, whereas age inference rarely exceeded the 20% chance level.The authors attribute difficult age inference to the dataset’s limited 18–34 age range.
5 Biometric Identity Inference Benchmark
The benchmark evaluates identity inference across sensors, perspectives, and sessions, finding substantial single-session risk but weak multi-session generalization. Results also show that alternative imaging sensors can match or exceed video-camera accuracy, while some radar and WiFi systems perform poorly.
- Single-session results: Identity inference is possible with every sensing technology, with at least one system per sensor exceeding 80% accuracy and all sensors except radar exceeding 99%.The center-high LWIR perspective is a notable exception, where most systems lose about 20 percentage points and DeepGaitv2 reaches 7.3% accuracy because of silhouette-extraction inconsistencies.
- Single-session results: Almost all imaging-sensor measurements exceed 98% accuracy, suggesting limited recognition differences after silhouettes are extracted.GaitSet often performs worse than other recognition systems, possibly because it was designed for cross-view scenarios not tested here.
- Single-session results: Some non-imaging systems, including mmGaitNet, mID, and FreeSense, fail to achieve high identification accuracy on MultiGait and underperform compared with claims on other datasets.The benchmark reports high variance for mmGaitNet and conducts additional experiments to validate its implementation and experimental design.
- Multi-session results: Existing recognition systems do not generalize well across sessions: no accuracy exceeds 80%, and reliable cross-session identification is not yet possible.Imaging-based and lidar systems identify correctly in a majority of cases, whereas radar and WiFi systems do not significantly exceed chance-level accuracy.
- Single-session results: Thermal, depth, and lidar sensors do not lag video cameras in identification accuracy, while depth often outperforms video by more than 10 percentage points in center-low and center-high views.LWIR also outperforms video on the center-low perspective, indicating that alternative sensors are not necessarily privacy-friendly by design.
6 Discussion
The benchmark finds high identity-inference potential across tested sensing technologies, while multi-session recognition remains unreliable. MultiGait supports broader privacy, recognition, and anonymization research but has important scope and population limitations.
- All tested sensing technologies have high potential for identity inference, challenging claims that some are privacy-preserving.The authors call for mitigation approaches such as anonymization.
- Existing gait-focused recognition systems do not yet reliably generalize across recording sessions.The limitation may reflect current systems, insufficiently robust biometric data, or both; radar, CSI, and BFI systems appear particularly unsuited.
- The dataset supports attribute and activity inference studies and anonymization evaluation using participant identities and multiple activities.It is also among the largest available datasets for some included sensors, including radar and NIR.
- Synchronized recordings across sensors and perspectives enable multi-sensor, cross-sensor, and multi-perspective recognition research.The combination of information sources remains important for evaluating privacy-utility trade-offs.
- Deploying these sensors in smart cities can introduce significant privacy risks, so they should receive scrutiny and legal treatment comparable to video cameras.
- The dataset is restricted to a lab environment, has fewer subjects than some video gait datasets, and does not represent the entire population.Its sample is limited to able-bodied individuals and biased in ethnicity and age; the authors caution that results must be contextualized accordingly.
- The benchmark evaluates representative state-of-the-art recognition systems rather than every available system.The authors consider the impact of this selection negligible because the approaches are diverse.
7 Conclusion
MultiGait provides a synchronized benchmark across eight sensing technologies, perspectives, and sessions for comparing smart-city sensing privacy risks. It finds high identification risk for many supposedly privacy-friendly sensors and poor multi-session generalization by current recognition systems.
- MultiGait records eight sensing technologies across multiple perspectives and sessions in a synchronized, full-factorial design.
- The benchmark finds high identification risk for many sensors claimed to be privacy-friendly, while current recognition systems fail to generalize across sessions.The dataset also supports future evaluation of robustness, human motion, activity modeling, and temporal generalization.
A Demographics
The demographics appendix directs readers to Table 6 for categorical participant information.
- Table 6 contains categorical demographic information on nationality, gender, hair color, and skin color.
B Additional Results
The appendix reports where the full identity-inference and demographic results are located. It also lists the skin-color category distribution shown in the supplied passage.
- Tables 7 and 8 report full results for the single-session and multi-session identity-inference experiments, respectively.Table 9 contains the full attribute-inference validation results.
- Table 6 presents the categorical demographics of the participants.
- 47.2% of participants are in the light skin-color category, followed by 34.2% light intermediate and 9.5% dark intermediate.The remaining reported categories are dark 4.5%, very light 3.0%, and very dark 0.5%.
- Table 7 reports measured accuracies for all sensors, recognition systems, and perspectives in the single-session experiment.
C Dataset Release
MultiGait access is governed by researcher agreements, administrator approval, and restrictions intended to balance reproducibility with the sensitivity of biometric data. Academic users must follow limits on redistribution, modification, commercial use, and publication of dataset materials.
- Access and consent: Researchers must sign a release agreement and obtain administrator approval; non-research-institution applicants must also briefly describe their research intention.Students and postdoctoral researchers without permanent institutional contracts additionally need a permanent member’s signature.
- Usage restrictions: The dataset may only support academic research and may not be redistributed, copied, disseminated, or modified.All users must sign the access document, and commercial use is strictly prohibited.
- Publication requirements: Publications containing more than 10 still frames or a clip require written administrator approval and must black out subjects’ faces.Researchers must avoid presenting material in ways that could cause embarrassment or mental anguish, cite the dataset, and send publication copies to administrators.