Source-linked AI summary
UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones
Daniela Micucci, Marco Mobilio, Paolo Napoletano
TL;DR
Public smartphone activity datasets often lack subject and activity metadata needed for targeted evaluation, especially for fall detection. The paper introduces UniMiB SHAR and benchmarks it across activity and fall classification tasks, finding that activity recognition is easier than fall-type classification and that subject-dependent evaluation performs better.
Problem
Most public smartphone datasets lack detailed subject and activity metadata, limiting selection of samples for targeted human-activity and fall-detection evaluations.
Method
The authors construct UniMiB SHAR with labeled smartphone accelerometer samples covering nine ADLs and eight falls, then benchmark classifiers across four classification tasks.
Results
Activity-type classification is easier than fall-type classification, fall-versus-no-fall discrimination is easy, and subject-dependent evaluation outperforms subject-independent evaluation.
Takeaways & Limitations
UniMiB SHAR supports research on robust features and classification schemes for fall detection and personalization.
Takeaways & Limitations
Broader cross-dataset, personalization, and smartwatch–smartphone evaluations were identified as planned future work.
Abstract
from arXiv · showhide
Smartphones, smartwatches, fitness trackers, and ad-hoc wearable devices are being increasingly used to monitor human activities. Data acquired by the hosted sensors are usually processed by machine-learning-based algorithms to classify human activities. The success of those algorithms mostly depends on the availability of training (labeled) data that, if made publicly available, would allow researchers to make objective comparisons between techniques. Nowadays, publicly available data sets are few, often contain samples from subjects with too similar characteristics, and very often lack of specific information so that is not possible to select subsets of samples according to specific criteria. In this article, we present a new dataset of acceleration samples acquired with an Android smartphone designed for human activity recognition and fall detection. The dataset includes 11,771 samples of both human activities and falls performed by 30 subjects of ages ranging from 18 to 60 years. Samples are divided in 17 fine grained classes grouped in two coarse grained classes: one containing samples of 9 types of activities of daily living (ADL) and the other containing samples of 8 types of falls. The dataset has been stored to include all the information useful to select samples according to different criteria, such as the type of ADL, the age, the gender, and so on. Finally, the dataset has been benchmarked with four different classifiers and with two different feature vectors. We evaluated four different classification tasks: fall vs no fall, 9 activities, 8 falls, 17 activities and falls. For each classification task we performed a subject-dependent and independent evaluation. The major findings of the evaluation are the following: i) it is more difficult to distinguish between types of falls than types of activities; ii) subject-dependent evaluation outperforms the subject-independent one
1. Introduction
The introduction motivates smartphone-based human-activity and fall recognition by limitations in existing public datasets and presents UniMiB SHAR as a more flexible benchmark dataset. It emphasizes detailed subject and activity metadata, including 11,771 smartphone-accelerometer samples from 30 subjects.
- Motivation: Insufficient physical activity is associated with a 20% to 30% higher risk of all-cause mortality than performing at least 150 minutes of moderate-intensity activity weekly.
- Dataset gap: Public datasets are scarce because researchers usually build their own datasets and rarely make them publicly available, hindering objective comparisons.
- Existing datasets: Only 5 datasets out of 11 contain both falls and activities of daily living, limiting coverage across recognition tasks.
- Existing datasets: Only MobiAct, RealWorld (HAR), and UMA Fall support selection across several dimensions such as age, sex, weight, or ADL type.
- UniMiB SHAR contribution: UniMiB SHAR contains 11,771 samples from 30 subjects, covering 9 ADL types and 8 fall types, with 151 accelerometer values for each axis per sample.The samples comprise 7,579 activities of daily living and 4,192 falls; each sample retains the identity of the subject who generated it.
2. Dataset Description
UniMiB SHAR was designed by comparing public datasets to select representative movement-related ADLs and falls, then recording smartphone acceleration from 30 healthy subjects. Each sample contains 3 acceleration vectors of 151 values, yielding 11,771 samples split between ADLs and falls.
- Data acquisition: The Samsung Galaxy Nexus recorded triaxial acceleration at a maximum frequency of 50Hz, with Android limiting the range to ±2g and resolution to 0.004g.Subjects carried the smartphone in their front trouser pocket, alternating between right and left positions across protocols.
- ADL selection: The dataset selects 9 movement-related ADLs from commonly represented activities in 24 analyzed public datasets, emphasizing low-order daily activities and excluding postural, housekeeping, cooking, and personal-care activities.Selected ADLs include walking, going upstairs, going downstairs, sitting down, running, standing up from sitting, standing up from laying, lying down from standing, and jumping.
- Fall selection: Falls were selected from 9 datasets by choosing two representative, popular falls from each of four categories while excluding datasets that did not specify fall types.The selection also targeted a fall count comparable to other datasets, and the chosen falls were confirmed as common in real life.
- Participants: 30 healthy subjects participated, including 24 women and 6 men aged 18 to 60 years (27 ±12 years), and each performed both ADLs and falls.Subjects provided written informed consent, and the study followed the WMA Declaration of Helsinki.
- Dataset representation: 11,771 samples comprise 7,759 ADLs and 4,192 falls, represented for each activity by 3 vectors of 151 values, or a 1x453 acceleration vector.The ADL and fall samples are not equally distributed across activity types because running and walking were performed for longer durations.
3. Dataset Evaluation
The evaluation compared four activity/fall classification tasks using subject-dependent and leave-subject-out validation, two accelerometer feature representations, and four classifiers. Because activity classes were imbalanced, performance was measured with macro average accuracy.
- Classification tasks: The dataset defined AF-17, AF-2, A-9, and A-8 subsets to assess discrimination among all classes, ADLs versus FALLs, ADL types, and FALL types.AF-17 combines 9 ADL and 8 FALL classes, while AF-2 merges each group into one class.
- Validation protocols: Traditional 5-fold cross-validation mixed samples from the same subject across training and test folds, whereas leave-subject-out validation tested on one user while training on all others.The 5-fold results were averaged across stratified random test folds; leave-subject-out evaluation reduced personalization effects.
- Feature representations: The experiments compared 453-dimensional raw-data vectors with 151-dimensional magnitude vectors derived from the accelerometer signal.Raw vectors concatenate 151 acceleration values from each Cartesian direction.
- Classifiers: Four classifiers were evaluated: 1-nearest neighbour, radial-basis-kernel SVM, a three-layer feed-forward ANN, and a Random Forest with 300 bagged classification trees.The ANN used back propagation and a softmax output layer; its hidden-neuron count was set to m × k.
- Evaluation metric: Macro average accuracy was used to address class imbalance by giving each activity’s partial accuracy equal contribution to the evaluation.MAA is the arithmetic average of the accuracy Acca for each activity.
4. Results and Discussion
Five-fold evaluation generally performed better with raw data than with signal magnitude, but fine-grained activity and fall recognition remained challenging. Leave-subject-out evaluation caused an evident performance drop, motivating more robust features and online learning methods.
- 5-fold evaluation: Raw-data features outperformed magnitude features in the 5-fold evaluation overall.The k-fold scheme used activity samples from all subjects in training and testing, producing subject-dependent classifiers.
- 5-fold evaluation: 83% MAA with raw data and KNN versus 66% with magnitude and RF was reported for AF-17, while F-8 reached 78% and 57%, respectively.These results indicate that distinguishing activity types and especially fall types was challenging.
- 5-fold evaluation: 88% MAA was obtained for A-9 with raw data and RF, compared with 79% using magnitude and SVM.The most misclassified activity pairs included standing transitions, lying or sitting transitions, stair movement, walking, and jumping.
- 5-fold evaluation: 99% MAA was achieved for AF-2 with raw data and SVM, making fall-versus-no-fall recognition very easy.The task was reported as easy for all classifiers and both feature representations.
- Leave-subject-out evaluation: 56.58% MAA was the best AF-17 result in leave-subject-out evaluation, using RF and magnitude.Performance dropped relative to 5-fold evaluation, particularly for fine-grained AF-17, A-9, and F-8 tasks because subjects perform activities differently.
- Leave-subject-out evaluation: More robust feature vectors and online learning schemes were proposed to reduce the subject-dependent versus subject-independent gap and improve performance using a few subject-dependent samples.The proposed directions also included robust features for accelerometer samples from different fall types.
5. Conclusion
The conclusion emphasizes UniMiB SHAR’s value for studying smartphone-based activity and fall recognition, personalization, and robust features. Results show that feature-vector performance depends on evaluation setting, while activities are easier to classify than fall types.
- Dataset value: Most smartphone datasets do not support selecting samples by subjects’ physical characteristics or performed activities and falls.Only MobiAct, RealWorld (HAR), and UMA Fall are identified as exceptions among 11 datasets with smartphone measurements.
- Classification findings: Raw data performs better than magnitude in subject-dependent evaluation, whereas magnitude performs better than raw data in subject-independent evaluation.
- Classification findings: Different activities are easier to classify than different types of falls, while distinguishing falls from no falls is very easy for both feature vectors.
- Research opportunities: UniMiB SHAR enables research on robust features for falls and robust features and classification schemes for personalization.
- Future work: Planned work includes evaluating state-of-the-art ADL recognition techniques across UniMiB SHAR and publicly available smartphone accelerometer datasets.The authors also plan personalization experiments using datasets containing subject characteristics.