Source-linked AI summary
Human Activity Recognition using Recurrent Neural Networks
Deepika Singh, Erinc Merdivan, Ismini Psychoula, Johannes Kropf, Sten Hanke, Matthieu Geist, Andreas Holzinger
TL;DR
Smart-home activity recognition is important for ambient assisted living but is challenged by ambiguous and noisy sensor data and limitations in existing temporal models. The paper applies an LSTM recurrent neural network to three real-world smart-home datasets and compares it with established probabilistic models. The LSTM matches or outperforms those models on raw inputs, improving the best result by approximately 40% in one case.
Problem
Activity recognition must identify simple and complex activities from ambiguous, noisy sensor data, while available real-world datasets for training data-driven models are limited.
Method
The paper applies an LSTM recurrent neural network to classify activities from smart-home sensor data and compares it with Naive Bayes, HMM, HSMM, and CRF.
Results
The LSTM matched or outperformed existing probabilistic models on raw inputs, improving the best result by approximately 40% in one case.
Takeaways & Limitations
Raw-input activity prediction can reduce the human effort required for preprocessing and handcrafted feature construction in ambient assisted living.
Takeaways & Limitations
Deep learning badly captures model uncertainty, motivating future investigation of Bayesian approaches for uncertainty reasoning.
Abstract
from arXiv · showhide
Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living. The increasingly large amount of data sets calls for machine learning methods. In this paper, we introduce a deep learning model that learns to classify human activities without using any prior knowledge. For this purpose, a Long Short Term Memory (LSTM) Recurrent Neural Network was applied to three real world smart home datasets. The results of these experiments show that the proposed approach outperforms the existing ones in terms of accuracy and performance.
1 Introduction
Human activity recognition in smart homes supports ambient assisted living but remains difficult because sensor data can be ambiguous and noisy. The paper introduces an LSTM recurrent neural network and evaluates it against established machine-learning models on benchmark datasets.
- Smart-home activity recognition is studied for home automation, convenience services, and improving residents’ quality of life in ambient assisted living.
- Sensor data can be ambiguous, noisy, and uncertain, making it challenging to identify simple and complex activities in real-world settings.Noise may arise from human behavior or incorrect sensor readings caused by network errors.
- Existing approaches detect activity patterns through sensor-derived features and classification or through Hidden Markov Modeling.
- Deep learning learns multilayer nonlinear representations directly through successive feature-extraction and transformation stages.The paper describes deep learning as neural-network methods with more than one hidden layer.
- The paper applies an LSTM recurrent neural network to classify activities such as cooking, bathing, and sleeping, comparing it with Naive Bayes, HMM, HSMM, and CRF.
- The paper evaluates the models using publicly available benchmark datasets and reports results in later sections.
2 Related work
Prior activity-recognition research includes data-driven and knowledge-driven approaches, with established classifiers addressing uncertainty or temporal information incompletely. The paper motivates recurrent neural networks because temporal distance and long-term dependencies may preserve useful activity patterns.
- Data-driven approaches handle uncertainty and temporal information but require large training datasets, whose availability is a major ambient-assisted-living challenge.
- Knowledge-driven approaches model relationships between sensor data and activities descriptively but cannot handle uncertainty and temporal information.
- Activity recognition has used classifiers including Naive Bayes, decision trees, HMM, CRF, nearest neighbor, SVM, and boosting methods.
- Naive Bayes can achieve good accuracy with large sample sets but does not model temporal information.
- HMM, HSMM, and CRF incorporate temporal information but may discard pattern sequences conveyed by intervals between events.
- LSTM is designed to model temporal sequences and learn long-term dependencies, motivating its use for activity-recognition patterns defined by temporal distance.
3 LSTM Model
LSTM models temporal sequences through memory cells controlled by input, output, and forget gates. Unrolled over time, the network maps binary sensor inputs to activity-label predictions.
- LSTM architecture: LSTM models temporal sequences and learns long-term dependencies through a memory cell controlled by three gates.The input, output, and forget gates regulate write, read, and reset operations on the cell state.
- LSTM architecture: The input, output, and forget gates control modifications to the cell state.The cell state carries information between LSTM blocks.
- LSTM equations: The gate equations use weight matrices, sensor inputs, sigmoid and hyperbolic tangent activations, and memory-cell terms.The symbols i, f, and o denote the corresponding gates, while c denotes the memory cell.
- Temporal network: Unrolling LSTM cells in time constructs a layer whose hidden state and output evolve at each time step.Figure 2 represents x as the binary sensor-input vector and y as the predicted activity label.
4 Experiments
The experiments evaluate separate house-specific LSTM models on three annotated smart-home datasets using raw and last-fired sensor representations. On raw data, LSTM performs best across all houses, while on last-fired data it matches the best performance for two datasets but drops slightly in House C.
- 4.1 Dataset: Three single-user houses supplied publicly available annotated sensor datasets with different sensor deployments and activity labels.Activities were recorded through diaries and a Bluetooth headset with speech recognition software.
- 4.1 Dataset: Each house used leave-one-day-out evaluation, with the remaining days for training and separate models trained per house.The experiments used both raw sensor data and last-fired sensor data representations.
- Experimental setup: The LSTM used a time slice of (70), hidden state size (300), Adam with learning rate 0.0004, and TensorFlow implementation.Training used a Titan X GPU, with reported training time measured for one day and one house.
- 4.2 Results: LSTM achieved the best performance for all three houses on raw sensor data.The comparison included Naive Bayes, HMM, HSMM, and CRF.
- 4.2 Results: Approximately 40% improvement occurred on House C over the best result for raw sensor data.The paper also reports significant improvement in House B.
- 4.2 Results: On last-fired sensor data, LSTM did not improve results, matched the best performance for two datasets, and dropped slightly in House C.These results were compared using the same models as in the raw-data evaluation.
5 Discussion
The LSTM activity predictor matched or outperformed established probabilistic models on raw sensory inputs, improving the best result by 40% in one case while reducing preprocessing effort.
- The LSTM activity predictor matched or outperformed Naive Bayes, HMM, HSMM, and CRF models on raw input.The comparison covered existing probabilistic models without requiring handcrafted feature preprocessing.
- 40% was the improvement over the best existing result in one case.
- Raw-input prediction reduces human effort spent on preprocessing and handcrafted feature construction in ambient assisted living.The paper identifies feature preparation as potentially time-consuming in AAL environments.
6 Future Work
Future work targets prediction variance, training controls, hyperparameter effects, alternative deep learning models, and uncertainty representation.
- Future work will reduce prediction variance and investigate early stopping while training across different days.
- Different optimization and hyperparameter-search techniques could address LSTM hyperparameters that significantly affect performance.
- Applying other deep learning models is proposed because the LSTM model performed superiorly on raw data.
- Bayesian approaches are discussed as a way to represent model uncertainty in deep learning without sacrificing computational complexity or test accuracy.The paper points to dropout training as approximate Bayesian inference in deep Gaussian processes.