Source-linked AI summary
Active User Authentication for Smartphones: A Challenge Data Set and Benchmark Results
Upal Mahbub, Sayantan Sarkar, Vishal M. Patel, Rama Chellappa
TL;DR
Smartphone authentication must address the security of personal information while password entry can be burdensome. This paper introduces UMDAA-02 and benchmarks face, touch, and location tasks; its face-verification findings indicate that more robust, fast methods are needed for satisfactory performance on this dataset.
Problem
Password entry can be cumbersome even as smartphones contain personal information, motivating investigation of automated user verification.
Source: §1 · p. 1Method
The paper introduces UMDAA-02, a passively collected smartphone dataset, and benchmarks face detection and verification, swipe identification, and next-place prediction.
Source: §1 · p. 2Results
Takeaways & Limitations
UMDAA-02 provides a broad sensor dataset for studying multimodal fusion in practical active-authentication scenarios.
Source: §7 · p. 7The face-verification results indicate that more robust yet fast methods are needed to achieve satisfactory performance on this dataset.
Source: §4 · p. 5
Abstract
from arXiv · showhide
In this paper, automated user verification techniques for smartphones are investigated. A unique non-commercial dataset, the University of Maryland Active Authentication Dataset 02 (UMDAA-02) for multi-modal user authentication research is introduced. This paper focuses on three sensors - front camera, touch sensor and location service while providing a general description for other modalities. Benchmark results for face detection, face verification, touch-based user identification and location-based next-place prediction are presented, which indicate that more robust methods fine-tuned to the mobile platform are needed to achieve satisfactory verification accuracy. The dataset will be made available to the research community for promoting additional research.
1. Introduction
Smartphone password entry can be burdensome even as phones hold personal information, motivating continuous authentication in the background. The paper introduces UMDAA-02, a passively collected smartphone dataset, and reports benchmarks spanning face, touch, and location tasks.
1. Introduction
- Password entry can be time-consuming, cumbersome, and error-prone, while phones store personal information and are often lost or stolen.The passage reports that over 40% of users in major U.S. cities have lost phones or experienced phone theft, while 34% of U.S. smartphone users do not use password locks.Source: §1 · p. 1
- Active authentication compares a user’s ongoing usage patterns with an enrolled pattern in the background, granting full phone functionality when they are sufficiently similar.Otherwise, the system blocks the current user.Source: §1 · p. 1
- UMDAA-02 is introduced as a non-commercial smartphone dataset with a wide range of sensor data, collected passively to represent volunteers’ natural, regular phone use.The collection application ran in the background on Nexus 5 devices and periodically uploaded data to a secure online location.Source: §1 · pp. 1–2
- The paper reports four benchmarks: face detection, face verification, swipe-based user identification, and next-place prediction using geolocation.The location experiment is described as potentially useful to active-authentication research when fusing multiple modalities.Source: §1 · p. 2
2. Previous Works
Prior active-authentication work explores face, touch, typing, sensor, and multimodal approaches, along with several fusion strategies. Existing datasets vary in modality coverage and realism, while the large Project Abacus and Move datasets were unavailable to researchers.
2. Previous Works
- Active-authentication approaches include face, touch or swipe, multimodal fusion, gait, and device-movement signals, with trade-offs in accuracy, computation, and discriminative power.The passage says face-based authentication is most accurate but computationally demanding, while swipe or accelerometer data alone are not sufficiently discriminative.Source: §2 · p. 2
- Suggested protocols include progressive or risk-based authentication and context-aware authentication, with fusion at feature, score, or decision level.The passage lists k-nearest-neighbor, SVM, decision-tree, Bayesian, and GMM-based fusion algorithms.Source: §2 · p. 2
- MOBIO provides audiovisual face-authentication data, but its required head positioning inside an elliptical box does not represent real-life image acquisition.The dataset contains 61 hours of data from 150 participants across 12 sessions.Source: §2 · p. 2
- Other datasets cover smartphone behavior but have modality gaps: MIT Reality lacks face and touch biometrics, while UMDAA-01 includes unconstrained faces and touch data.UMDAA-01 face images include partially visible, frontal, and non-frontal faces under varied illumination.Source: §2 · p. 2
- Project Abacus and Project Move were passively collected, but neither dataset was available to the research community.Abacus contains 27.62 TB from approximately 1,500 users; Move contains 114 GB of inertial signals from 80 volunteers.Source: §2 · p. 2
3. Description of the UMDAA-02 Dataset
UMDAA-02 contains sensor data from 48 volunteers collected on Nexus 5 phones over two months, alongside phone-use and session information. The section describes sensor coverage and sampling, session organization, and variability in collected face images.
3. Description of the UMDAA-02 Dataset
- UMDAA-02 contains 141.14 GB of sensor signals from 48 volunteers, collected on Nexus 5 phones over two months across a broad set of smartphone sensors.The listed sensors include the front-facing camera, touchscreen, GPS, Bluetooth, WiFi, motion sensors, light, proximity, temperature, and pressure sensors.Source: §3 · p. 2
- Collection also recorded screen-lock and unlock timing, call timestamps, and the foreground application; volunteers used the research phone as their primary device for a week and could stop collection or review data before sharing.The supplied passages describe the option to stop collection and review stored data prior to sharing.Source: §3 · p. 3
- Sensor recording was generally triggered by significant modality changes; GPS sampling rate varied with movement speed, and camera images were captured at 3 fps for each session’s first 60 seconds.Table 1 presents significant information for each modality per session.Source: §3 · p. 3, Table 1 · p. 3
- A session begins when a user unlocks the phone and ends when it is locked, with data organized in nested folders named by year, month, day, and session start time.These folders encode chronology and session information.Source: §3 · p. 3
- Figure 2 presents histograms of image and session counts per user; Figure 3 provides one user’s sample images exhibiting variation in pose, illumination, occlusion, and expression.Source: Figure 2 · p. 3, Figure 3 · p. 3
4. Face Detection and User Verification
UMDAA-02 face images challenge detection and verification because they include partial faces, occlusion, pose, and illumination variation. Benchmarks show that stronger detection scores can come with substantial runtime, while face-verification error remains high.
Dataset and Challenge
- The annotated UMDAA-02-FD subset contains 33,209 images from 43 users, with partial faces and wide variation in pose, illumination, and occlusion.Images were selected at seven-second intervals; five users were excluded because their phone front cameras malfunctioned.Source: §4 · p. 3
Face Detection
- DP2MFD achieved the best face-detection Accuracy and F1-Score, but ran almost 100 times slower than Viola-Jones, limiting real-time smartphone suitability.The authors conclude that the best detection scores remain unsatisfactory and that better detectors for active authentication are needed.Source: Table 3 · p. 4, §4 · p. 4
Face Verification
- Verification extracts six feature types from aligned, centered, and cropped faces, then compares enrollment templates with averaged features from a moving window of M faces.The six representations include pixel intensity, LBP, MEEN, and Alexnet and DCNN features; the experiment evaluates four distance measures.Source: §4 · pp. 4–5
- 18.44% EER was the best face-verification result, using DCNN features (F6) with CosD at N = 20 and M = 30.The authors consider this unsatisfactory against an expected EER below 5%; partially visible faces can undermine landmark alignment and matching.Source: §4 · p. 5
- DCNN features performed best among the tested feature-distance combinations, while CNN feature extraction is generally much slower than non-CNN extraction without a GPU.The authors call for faster, more robust verification methods for satisfactory performance on this dataset.Source: §4 · p. 5
5. User Identification Using Swipe Dynamics
The swipe experiment evaluates user identification from naturally collected, single-finger touch sequences. Random Forest performs best among the tested classifiers, but its error rate remains unsatisfactory for practical use.
Data and Evaluation
- Swipes with at least five touch data points were represented by 24-dimensional features, with each user's chronologically first 70% used for training and validation and the remainder for testing.The data include finger-down, in-touch, and finger-up events; the area covered by a finger is not available in UMDAA-02.Source: §5 · pp. 5–6
Results
- 22.1% EER was achieved by Random Forest, which outperformed the other tested swipe-identification methods.The authors state that this performance is not satisfactory for practical use.Source: §5 · p. 6
Evaluation
- The experiment averaged scores across consecutive swipes for robustness and compared seven classifiers using EER.Methods included KNN, RBF-SVM, Naive Bayes, Linear Regression, RT+LR, Random Forest, and Gradient Boosting.Source: §5 · p. 6
6. Geo-location Data and Next Place Prediction
The location experiment predicts a user's next place from geolocation histories using Mobility Markov Chains. Accuracy improves with more prior observations and larger clustering radii, with larger radii trading localization capability for prediction accuracy.
State Construction
- DBSCAN clusters each user's location history into places, with additional Transit and Unknown states; observations are also divided by weekday or weekend and three time zones.Transit is assigned when location changes rapidly at 2 m/s or faster; other data points are labeled Unknown.Source: §6 · p. 6
Model and Evaluation
- The Mobility Markov Chain predicts a next state from the last visited state, and its training and testing traces are divided chronologically at 70%.Increasing the maximum cluster radius from 20 m to 500 m reduces the average number of states per user from 144 to 35.Source: §6 · p. 7
Results
- At R = 500 meters and n = 8, next-location Accuracy was 65.3% and Accuracy3 was 96.6%.Accuracy3 counts a prediction as correct when the actual next location is among the three most probable locations; the larger radius improves accuracy at the cost of localization capability.Source: §6 · p. 7
7. Conclusion
The paper presents UMDAA-02 as a multimodal challenge dataset for active authentication, with benchmark results for face and touch authentication and preliminary next-location prediction. Its broad smartphone-sensor data is intended to support research on multimodal fusion in practical authentication settings.
7. Conclusion
- UMDAA-02 provides benchmark results for face- and touch-based active authentication and preliminary next-location prediction.Source: §7 · p. 7
- UMDAA-02 is the first non-commercial smartphone-usage dataset containing data from a wide variety of smartphone sensors.Source: §7 · p. 7
- The dataset can provide resources for studying multimodal fusion across modalities in practical active-authentication scenarios.Source: §7 · p. 7
- The authors state that the dataset will be released to the research community.Source: §7 · p. 7