Source-linked AI summary
Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey
Florenc Demrozi, Graziano Pravadelli, Azra Bihorac, Parisa Rashidi
TL;DR
HAR research needs methods that can recognize activities across varied settings while balancing accuracy, privacy, computational demands, and data limitations. This survey reviews machine-learning-based HAR using inertial, physiological, and environmental sensors, and finds that DL achieves strong recognition performance while CML remains widely used; reproducibility and dataset availability remain major constraints.
Problem
HAR systems face varied daily activities, subject variability, privacy-performance trade-offs, computational constraints, annotation difficulty, and limited heterogeneous datasets.
Method
The survey synthesizes HAR literature by reviewing sensors, devices, activities, datasets, models, and evaluation metrics, with particular attention to accelerometers and combined sensor modalities.
Results
78% of proposed HAR methodologies were not fully reproducible because of proprietary datasets; among 142 datasets, only 30 were publicly available.
Takeaways & Limitations
CML remains widely used because it requires less training data and computation, whereas DL can recognize more complex activities while maintaining high accuracy.
Takeaways & Limitations
Most evaluated models rely on controlled environments and limited activities, restricting assessment of generalization across subjects and settings.
Abstract
from arXiv · showhide
In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly's daily life by monitoring their cognitive and physical function through daily activities. This survey focuses on critical role of machine learning in developing HAR applications based on inertial sensors in conjunction with physiological and environmental sensors.
1 INTRODUCTION
HAR is increasingly used across health care, sports, gaming, and monitoring, but practical systems must address activity variability, privacy, computation, and annotation challenges. The survey presents a sensor-driven workflow linking data collection, model development, and evaluation.
- HAR supports monitoring physical, functional, and cognitive health, particularly as the population aged over 65 is projected to grow from 461 million to 2 billion by 2050.
- HAR algorithms must handle diverse activities, intra-subject and inter-subject variability, privacy-performance trade-offs, embedded-device efficiency, and difficult data annotation.
- Video cameras provide rich context, but privacy and computational requirements motivate using depth images and other ambient or embedded sensors.
- Deep learning has increased recognition accuracy, while classic machine learning can suit small datasets, lower-dimensional inputs, and problems with available expert knowledge.
- The standard workflow identifies sensors, collects and annotates data with preprocessing, selects and trains a model, and evaluates recognition metrics, with model choice potentially influencing preprocessing.
2 EXISTING SURVEYS
Existing HAR surveys commonly organize research by data source or algorithm, while this survey combines a meta-review with analysis of sensors, devices, datasets, activities, and performance. Its scope emphasizes inertial, physiological, and environmental sensing rather than exclusively video-based methods.
- Among 293 identified papers, 46 were surveys published since 2015, commonly categorized by data sources or activity-recognition algorithms.
- The survey excluded 23 exclusively video-based papers and instead examined sensor sources, especially accelerometers used alone or with physiological and environmental sensors.
- Figure 1 compares DL and CML by published-paper distribution and average recognition accuracy, while Figure 2 shows their yearly publication distributions.
- The standard HAR workflow includes device identification, data collection, model selection and training, and model evaluation.
- The review contributes a comprehensive meta-review, sensor overview, performance comparison, and account of dataset availability and popularity.
3 SELECTION CRITERIA
The survey searched Google Scholar for HAR studies published from January 2015 to September 2019 using combinations of HAR, learning, wearable-sensor, and sensor-specific terms. From 249110 records, 293 articles were selected based on publication-venue quality.
- Google Scholar searches covered January 2015 to September 2019 and combined HAR terms with deep learning, machine learning, wearable sensors, and specific sensor names.
- The search produced 249110 records, from which 293 articles were selected based on publication-venue quality.
- Selected articles came from IEEE, ACM, Elsevier, and Sensors, with an average citation count of 46.
4 BACKGROUND
HAR systems use machine learning with wearable and environmental sensor data, combining classical and deep learning approaches with sensor fusion. Model choice reflects dataset size, computational constraints, interpretability, and the temporal or spatial structure of the data.
- Machine Learning and Deep Learning: HAR research applies CML and DL algorithms to data gathered from wearable and environmental sensors.The survey introduces machine learning concepts, sensor types, and sensor fusion techniques for HAR applications.
- Machine Learning and Deep Learning: DL methods learn feature representations automatically from raw input data, but are difficult to interpret and require large datasets and high computational cost.The survey identifies black-box behavior, data requirements, and computational cost as key DL limitations.
- Machine Learning and Deep Learning: CML methods remain suitable for small datasets, lower-dimensional inputs, expert knowledge, and fast training requirements.They generally require less training data and lower computational resources than DL methods.
- Deep Learning Architectures: CNNs extract local patterns from images or one-dimensional signal segments, whereas RNNs model sequential time-series dependencies.RNN hidden states incorporate prior sequence elements, while CNNs impose local connectivity through convolution and pooling.
- Sensors and Wearable Devices: Accelerometers are prevalent in HAR because they are small and inexpensive, and wearable devices increasingly combine them with other sensing and communication modules.Examples include gyroscopes, magnetometers, GPS, BLE, WLAN, and physiological sensors.
- Sensor Fusion: Sensor fusion combines measurements to reduce noise and uncertainty and improve robustness, although increasing sensor count makes fusion more challenging.Common fusion approaches include Bayesian estimation, Kalman filters, and particle filtering.
5 HUMAN ACTIVITY
Activities of Daily Life encompass routine physical activities such as eating, bathing, dressing, work, homemaking, and leisure. HAR studies most often recognize basic locomotion and posture, while also examining increasingly complex and context-specific activities.
- Activities of Daily Life: Activities of Daily Life include routine activities involving physical movement, such as eating, bathing, dressing, working, homemaking, and leisure.The survey uses ADL as a broad category for activities performed daily.
- Common Activities: Walking, running, standing, sitting, and stair walking are the most frequently studied ADL activities in HAR research.Walking upstairs and downstairs are listed separately among the most popular activities.
- Complex Activities: Recent HAR studies also examine complex activities including cooking phases, house cleaning, driving, smoking, swimming, and biking.Some studies focus on activities performed in specific locations.
6 DATA SOURCE DEVICES IN HAR
Accelerometers dominate sensor-based HAR surveys, often alongside gyroscopes and magnetometers, while physiological, environmental, radio, and embedded sensors broaden the available data sources. Smartphones and smartwatches offer comparable recognition accuracy to standalone devices and can execute HAR models directly.
- Inertial Sensors: 149 reviewed papers used accelerometers, 83 used gyroscopes with accelerometers, and 27 used magnetometers with accelerometers.All selected papers used at least one accelerometer, alone or combined with other sensors.
- Inertial Sensors: Small, low-cost, and non-invasive accelerometers, gyroscopes, and magnetometers are the most commonly used HAR sensors.Figure 7(a) summarizes their prevalence across the reviewed literature.
- Physiological and Environmental Sensors: HAR also uses environmental sensors, radio signals, medical equipment, and embedded sensors such as temperature, humidity, WiFi, ECG, EMG, GPS, heart-rate, barometer, and audio sensors.These sources extend inertial measurements with physiological and environmental information.
- Derived Sensor Information: Indirect sensor metrics such as energy harvesting and RSSI can support activity recognition related to body measurements or environmental variations.The survey identifies these smart metrics as promising additional inputs.
- Device Types: Smartphone- and smartwatch-based methods achieve recognition accuracy in line with standalone-device methods while providing on-device computational capabilities.The survey reports that their total number exceeds standalone-device methods, although proposed methods are mostly standalone-device based.
7 DATA
HAR data are collected from inertial, physiological, and environmental sensors, then preprocessed and evaluated across datasets whose coverage and public availability remain limited.
- Data types: HAR data comprise inertial, physiological, and environmental sensor streams, including motion, bodily signals, and environmental conditions.
- Inertial sensors: Accelerometers, gyroscopes, and magnetometers are inexpensive, widely used sources for characterizing human activity.
- Dataset challenges: Controlled or scripted activity collection makes generalization difficult because human movement varies within and across subjects, while large subject samples remain scarce.
- Datasets: Open HAR datasets primarily use smartphone- and smartwatch-embedded inertial sensors, with activity counts ranging from two to thirty-three.
- Preprocessing and features: 93% average accuracy was reported across 34 reviewed DL articles, while DL methods commonly omit manual feature extraction but require more data and longer training.
8 CLASSIFICATION MODEL AND EVALUATION
The survey compares classical and deep learning models for HAR using accuracy as the common metric, documenting model popularity, recognition accuracy, and activity coverage.
- Model comparison: CML models remained more popular than DL models, and accuracy was used because it was the only metric common across reviewed articles.
- Deep learning: DL coverage included 54 of 149 reviewed papers, with CNNs the most frequently used model among the surveyed DL methodologies.
- Deep learning: CNNs achieved an average accuracy of 93.7% across an average of 11 daily-life activities, while LSTMs achieved 91.5% across 17 activities.
- Classical machine learning: CML coverage included 95 of 149 reviewed papers, with SVM used in 35 papers and kNN in 23 papers.
- Classical machine learning: Decision Trees achieved 94.2% average accuracy across eight activities, while Random Forests achieved 93.3% across ten activities.
9 DISCUSSION
The discussion links model choice to data and computation, while emphasizing accelerometer use, limited dataset openness, and challenges in reproducibility and generalization.
- Reproducibility: 78% of proposed HAR methodologies were not fully reproducible because they used proprietary datasets.
- Reproducibility: Only 30 of 142 datasets were publicly available, creating barriers to identifying superior methodologies and benchmarking results.
- Generalization: Limited heterogeneous datasets and predominantly controlled, single-environment evaluations reduce the possibility of developing models with better generalization capabilities.
- Model selection: Model selection depends primarily on the number of activities, available labeled data, and computational requirements.
- Model trade-offs: CML models require less training data and computation, whereas DL models can recognize more complex activities without preprocessing but are harder to interpret.
- Sensors: Accelerometers are the most widely used, and often indispensable, sensors, frequently combined with gyroscopes or magnetometers.
10 FUTURE RESEARCH DIRECTION
The survey identifies limited standardization as a central HAR limitation and highlights transfer learning, sensor fusion, and fine-grained recognition as future directions.
- HAR lacks standardized methodologies that generalize across heterogeneous activities and diverse users.
- Transfer learning could reuse knowledge across sensor positions or different inertial sensor types, but its usefulness across scenarios remains insufficiently studied.
- Sensor fusion could improve reliability and accuracy by combining modalities and switching sensors when one modality is unreliable.
- Fine-grained recognition of daily object interactions could identify sub-actions and action sequences, providing richer context information.
11 CONCLUSION
The survey reports substantial progress in sensor-based HAR and reviews research from 2015 to 2019 across activities, sensors, devices, preprocessing, datasets, and recognition models.
- Sensor-based HAR offers advantages over vision-based methods because vision approaches raise privacy concerns and face computational constraints.
- The survey analyzes HAR literature by studied activities, sensor sources, and devices while excluding video-based methodologies.
- The review focuses on physiological, inertial, and environmental sensors and categorizes devices as standalone, smartphone, or smartwatch.
- For each device category, the survey reports average recognized activities, average datasets used for testing, and average accuracy.
- The survey also examines accelerometers, gyroscopes, magnetometers, preprocessing techniques, publicly available datasets, and commonly used recognition models.