Source-linked AI summary
Deep Activity Recognition Models with Triaxial Accelerometers
Mohammad Abu Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, Hwee-Pink Tan
TL;DR
Activity recognition from triaxial accelerometers remains limited by poor accuracy and scarce labeled data. The paper applies deep models that learn hierarchical features from acceleration signals and combines them with HMMs for sequential recognition, reporting improved recognition over shallow and state-of-the-art methods while avoiding handcrafted feature design.
Problem
Activity recognition using accelerometers faces poor recognition accuracy and a scarcity of labeled training data.
Method
The paper uses deep activity recognition models for triaxial accelerometers and combines their hierarchical representations with HMM stochastic temporal modeling in a DL-HMM.
Results
Deep activity recognition models improve recognition accuracy over conventional shallow models, while DL-HMM outperforms HMM-based methods for temporal activity recognition.
Takeaways & Limitations
The approach automatically learns acceleration features, avoids handcrafted feature design, and uses unlabeled samples for unsupervised feature extraction.
Takeaways & Limitations
Handcrafted-feature approaches remain laborious because they require human selection of effective features and decision thresholds from sensory data.
Abstract
from arXiv · showhide
Despite the widespread installation of accelerometers in almost all mobile phones and wearable devices, activity recognition using accelerometers is still immature due to the poor recognition accuracy of existing recognition methods and the scarcity of labeled training data. We consider the problem of human activity recognition using triaxial accelerometers and deep learning paradigms. This paper shows that deep activity recognition models (a) provide better recognition accuracy of human activities, (b) avoid the expensive design of handcrafted features in existing systems, and (c) utilize the massive unlabeled acceleration samples for unsupervised feature extraction. Moreover, a hybrid approach of deep learning and hidden Markov models (DL-HMM) is presented for sequential activity recognition. This hybrid approach integrates the hierarchical representations of deep activity recognition models with the stochastic modeling of temporal sequences in the hidden Markov models. We show substantial recognition improvement on real world datasets over state-of-the-art methods of human activity recognition using triaxial accelerometers.
Introduction
The paper addresses immature triaxial-accelerometer activity recognition by applying deep learning to improve accuracy, learn features automatically, exploit unlabeled data, and model temporal sequences.
- Deep learning extracts high-level feature hierarchies from high-dimensional acceleration data and automatically learns meaningful representations.
- Deep generative models use abundant unlabeled activity samples during unsupervised pre-training, addressing scarce labeled datasets.
- Deep activity recognition models significantly enhance recognition accuracy compared with conventional shallow models.
- A DL-HMM combines deep hierarchical representations with HMM stochastic modeling for temporal activity recognition.
Related Work
Prior activity-recognition systems rely on shallow classifiers and handcrafted features, while this paper motivates data-driven deep models as more automatic and better suited to temporal sequences.
- Limitations of Shallow Classifiers: Existing systems use shallow supervised algorithms such as multilayer perceptrons, support vector machines, and decision trees.
- Limitations of Shallow Classifiers: Conventional neural networks can suffer local optima and vanishing gradients when many hidden layers are added.
- Limitations of Shallow Classifiers: The paper reports significant recognition-accuracy improvement on real-world datasets over state-of-the-art triaxial-accelerometer methods.
- Limitations of Handcrafted Features: Handcrafted statistical features are problem-specific, generalize poorly across domains, and require laborious human intervention.
- Limitations of Handcrafted Features: Deep learning learns discriminative features from historical data systematically and automatically, supporting self-configurable recognition frameworks.
- Limitations of Handcrafted Features: The proposed probabilistic framework produces activity-membership probabilities that serve as an HMM emission matrix for ordered activity sequences.
Problem Statement
The problem is to infer human activities from windowed triaxial accelerometer measurements, represented as noisy three-axis signals and transformed into spectrogram inputs for recognition models.
- Data Acquisition: An accelerometer records a three-dimensional point at each time t containing x-, y-, and z-axis acceleration components.
- Data Acquisition: The system assumes bounded per-axis proper acceleration, with an accelerometer capacity represented by a known constant B > 0.
- Data Acquisition: Each axis measurement is modeled as noiseless proper acceleration plus independent zero-mean Gaussian noise with variance σ^2.
- Windowing and Recognition: Sliding windows form channel frames, and the recognition method infers an activity y_t from each windowed excerpt.
- Data Preprocessing: A spectrogram represents acceleration-energy changes across frequency and time and is used as the deep models’ input.
- Data Preprocessing: The concatenated triaxial spectrogram has length L = 3(N/2 + 1), reducing dimensionality relative to the time-domain signal length 3N.
System and Model
The system learns intrinsic activity features from acceleration data with a deep generative model, then predicts activities either independently or with HMM-based temporal modeling. Its DBN architecture uses layer-wise RBM training, followed by supervised fine-tuning and optional sequential inference.
- Deep activity recognition models: The model learns informative activity features through unsupervised generative pre-training and supervised discriminative fine-tuning, reducing reliance on hand-engineered features.The pre-training stage generates intrinsic features, while fine-tuning adapts the model using available labels.
- System pipeline: The recognition pipeline converts triaxial acceleration into spectrograms, computes deep intrinsic features, and estimates posterior probabilities for candidate activities.The final prediction is based on the posterior distribution P(ai|xt).
- Deep activity recognition models: Deep belief networks are built by stacking restricted Boltzmann machines trained greedily layer by layer, with acceleration spectrograms used as model input.After unsupervised pre-training, the learned weights are fine-tuned using available activity labels.
- Temporal activity recognition: The deep model supplies HMM emission probabilities so temporal activity sequences can be decoded using posterior inference and the most probable path.HMMs represent first-order temporal transitions between hidden activities and emissions from observations.
- Deployment and complexity: Offline training is computationally heavy and recommended on a capable server, whereas online recognition on the wearable device has linear complexity O(L × D).Here, L is input length and D is the number of deep-model layers.
Baselines and Result Summary
The paper evaluates deep activity recognition against existing approaches on three public datasets spanning mobile, healthcare, and industrial activity-recognition settings. Across these datasets, deep models improve recognition accuracy while avoiding hand-engineered features.
- Datasets: The evaluation uses three public datasets covering mobile-phone activities, Parkinson’s freezing-of-gait detection, and car-maintenance activities.WISDM contains triaxial mobile-phone data; Daphnet uses three accelerometers on patients; Skoda uses one node from a 20-node setup.
- Evaluation Metrics: Binary Daphnet classification uses sensitivity, specificity, and accuracy, whereas WISDM and Skoda use average recognition accuracy for multiclass activities.The binary metrics are defined using TP, TN, FP, and FN; multiclass evaluation averages per-activity recognition accuracy.
- Results: 6.53% higher accuracy than MLPs and 3.93% higher than ensemble learning are reported on the WISDM Actitracker dataset.The paper also reports significant improvements on the Daphnet and Skoda datasets.
- Results: Deep activity recognition models improve recognition accuracy over conventional methods across the three datasets and avoid hand-engineering features.The reported comparison includes the paper’s proposed method and previous solutions.
Experiments on Real Datasets
Experiments examine signal representations, DBN architecture, generative pre-training, and temporal modeling on real accelerometer datasets. Deeper and overcomplete models benefit from pre-training, while DL-HMM improves sequential recognition on Skoda.
- Signal Representation: Active motions such as jogging and walking produce high-frequency acceleration components, while sitting and standing produce low-frequency components tied to gravity.The Figure 3 caption presents time-series and spectrogram representations for six WISDM activities.
- DBN Structure: Using more layers improves recognition accuracy: four layers of 500 neurons outperform two layers of 1000, which outperform one layer of 2000.These configurations have comparable total neuron counts but different depth.
- DBN Structure: A 4x overcomplete architecture is recommended because undercomplete deep models are harder to optimize and show degrading training accuracy as layers are added.Overcomplete means each layer has more neurons than the input length; the experiments use an input length of 303 for 10-second frames.
- Pre-training Effects: Generative pre-training becomes more important as activity-recognition networks deepen, while it is almost ineffective for one-layer networks.Table 2 compares accuracy with and without pre-training for layers containing 1000 neurons.
- Temporal Modeling: 89.38% accuracy is achieved on 10 Skoda activities with a three-layer deep model, improving 3.38% over the referenced HMM method.The model uses three layers of 1000 neurons each and one accelerometer node.
- Temporal Modeling: 99.13% near-perfect recognition is achieved by the hybrid DL-HMM, which exploits temporal correlation in the sequential Skoda activity task.The HMM parameters represent transitions among activities and their initial prior probabilities.
Conclusions and Future Work
The proposed deep activity recognition approach improves recognition over traditional shallow methods with handcrafted features. It also uses model probabilities as emissions in an HMM to temporally model human-activity sequences.
- Deep activity recognition models produce significant recognition-accuracy improvements over shallow networks with handcrafted features.They extract hierarchical features directly from triaxial acceleration data.
- Recognition probabilities from the deep models serve as an HMM emission matrix for temporal activity modeling.This extends the approach from activity classification to sequential human-activity recognition.
- The approach addresses activity recognition using triaxial accelerometers.