PaperlayerView exact revision on arXiv

Source-linked AI summary

Different Approaches for Human Activity Recognition: A Survey

Zawar Hussain, Michael Sheng, Wei Emma Zhang

arXiv:1906.05074v1cs.CV

TL;DR

Human activity recognition lacks a unified view across sensing approaches and activity types despite applications in health, security, entertainment, and intelligent environments. This survey reviews 2010–2018 research, emphasizing device-free and RFID solutions, and organizes it into three categories and 10 sub-topics. It compares techniques and analyzes design attributes, metrics, trends, applications, open issues, and future directions.

  • Problem

    Existing surveys often focus on limited activity types or approaches, while comprehensive analysis of device-free RFID activity recognition is lacking.

  • Method

    The survey synthesizes 2010–2018 human activity-recognition research using an action-, motion-, and interaction-based taxonomy with 10 sub-areas, emphasizing device-free RFID approaches.

  • Results

    The survey covers nearly all activity-recognition sub-fields and provides comparative analysis of device-free techniques, including RFID-based gesture recognition achieving 91% accuracy in one reviewed system.

  • Takeaways & Limitations

    The review identifies device-free sensing as a major research direction and highlights applications across healthcare, tracking, interaction, and activity recognition.

Abstract

from arXiv · show

Human activity recognition has gained importance in recent years due to its applications in various fields such as health, security and surveillance, entertainment, and intelligent environments. A significant amount of work has been done on human activity recognition and researchers have leveraged different approaches, such as wearable, object-tagged, and device-free, to recognize human activities. In this article, we present a comprehensive survey of the work conducted over the period 2010-2018 in various areas of human activity recognition with main focus on device-free solutions. The device-free approach is becoming very popular due to the fact that the subject is not required to carry anything, instead, the environment is tagged with devices to capture the required information. We propose a new taxonomy for categorizing the research work conducted in the field of activity recognition and divide the existing literature into three sub-areas: action-based, motion-based, and interaction-based. We further divide these areas into ten different sub-topics and present the latest research work in these sub-topics. Unlike previous surveys which focus only on one type of activities, to the best of our knowledge, we cover all the sub-areas in activity recognition and provide a comparison of the latest research work in these sub-areas. Specifically, we discuss the key attributes and design approaches for the work presented. Then we provide extensive analysis based on 10 important metrics, to give the reader, a complete overview of the state-of-the-art techniques and trends in different sub-areas of human activity recognition. In the end, we discuss open research issues and provide future research directions in the field of human activity recognition.

I. INTRODUCTION

Human activity recognition research spans vision-based and sensor-based approaches, with growing interest in device-free sensing. This survey organizes 2010–2018 research into action-, motion-, and interaction-based categories, emphasizing device-free and RFID solutions.

  • Approach Motivation: Vision-based recognition can provide good results but raises privacy concerns and depends on adequate lighting.Because prior surveys already covered vision-based techniques, this survey excludes them from its main scope.
  • Sensor Deployment: Sensor-based approaches are divided by deployment into wearable, object-tagged, and dense-sensing categories.Wearable systems require users to carry sensors, while object-tagged systems require interaction with specific instrumented objects.
  • Sensor Deployment: Device-free sensing places sensors in the environment so users can perform activities without carrying tags or devices.The approach is presented as more practical, although environmental interference can introduce noise into captured data.
  • Survey Scope: The survey classifies activity recognition literature into action-based, motion-based, and interaction-based activities, divided into 10 sub-areas.Action-based areas include gestures, posture, falls, daily living, behavior, and ambient assisted living; motion-based areas include tracking, motion detection, and people counting; interaction-based work covers human-object interaction.
  • Related Surveys: Earlier surveys commonly focused on a single approach, including device-free radio, RFID, radio-based, Wi-Fi-based, or broader RF activity recognition.Their emphases included categorization, system steps, comparisons using selected metrics, and future research directions.

B. Sensor Based

Previous surveys organize sensor-based activity recognition by sensing modality, learning strategy, application, body placement, and activity type. The surveyed literature is broad, but earlier reviews often lacked comprehensive cross-approach comparison and detailed analysis of RFID device-free methods.

  • Sensor-Based Surveys: Sensor-based surveys classified research by vision versus sensing, data-driven versus knowledge-driven methods, and modalities including wearable and dense sensing.Deep-learning surveys additionally organized work by sensor modality, deep model, and application area.
  • Wearable Device Surveys: Wearable-sensor surveys examined design issues including sensor selection, data collection, recognition performance, processing, and energy consumption.They also categorized systems as supervised online, supervised offline, and semisupervised offline.
  • Wearable Device Surveys: Other reviews distinguished global body motion from local interaction activity and summarized mobile-phone-based recognition using built-in sensors.These classifications emphasized activity scope and sensor placement or device platform.
  • Survey Coverage: Earlier surveys primarily focused on one approach, such as sensor-based, vision-based, wearable, mobile-phone, or RF-based recognition.The survey identifies this specialization as a limitation of prior coverage.
  • Survey Gap: The survey targets a gap by comprehensively analyzing device-free RFID activity-recognition research while comparing advantages and disadvantages across techniques.It presents this focus as absent from previous comprehensive survey coverage.

III. TECHNICAL BACKGROUND

Human activity recognition combines sensor deployment, data collection, preprocessing and feature selection, and machine-learning inference. The technical background contrasts cameras, depth cameras, wireless and wearable sensors, and RFID-based sensing.

  • Recognition Process: Human activity recognition consists of sensor selection and deployment, data collection, preprocessing and feature selection, and machine-learning inference.These four phases form the general recognition pipeline.
  • Sensing Technologies: Sensors may be attached to objects, worn by users, or deployed in the environment, with common examples including accelerometers, motion, biosensors, gyroscopes, pressure, proximity, and RFID sensors.Cheap, portable sensors can sense and communicate information wirelessly.
  • Vision-Based Sensing: Surveillance-camera systems process videos and images manually or automatically to recognize human activities.Traditional cameras are limited by darkness, whereas depth cameras such as Kinect can operate in total darkness and provide RGB, depth, and audio streams.
  • Wireless Sensing: Wireless sensing uses properties such as Wi-Fi Channel State Information for applications including localization, tracking, and fall detection without requiring users to carry devices.The passage identifies unobtrusive operation as a major Wi-Fi advantage.
  • Sensor Examples: A multi-axis accelerometer measures acceleration simultaneously along the x, y, and z directions and supports recognition tasks ranging from gestures and posture to falls and tracking.It is also used in ambient assisted living and activities-of-daily-living solutions.

2) Magnetometer:

The supplied passages introduce sensing technologies and RFID components used to capture information for human activity recognition. They also frame activity recognition as covering bodily actions, object interactions, presence, motion, and tracking.

  • Sensor types: Magnetometers detect magnetic-field changes caused by human activity, while motion sensors detect movement or presence in an observed area.Proximity sensors detect nearby objects without physical contact and are widely used for gesture recognition.
  • RFID sensing: RFID readers emit radio waves, receive tag backscatter, and collect tag information through an antenna.Tags may be active, with batteries and longer range, or passive, harvesting energy from reader radio waves.
  • Recognition scope: Activity recognition includes whole-body actions, body-part gestures, object interactions, presence or motion detection, and human trajectory tracking.Examples include walking, running, sitting, cooking, intrusion detection, and movement tracking.

approach.

This section establishes comparison criteria for human activity-recognition techniques and surveys action-based activities, especially gesture recognition. It contrasts sensing approaches and highlights device-free RFID methods alongside their practical constraints.

  • Comparison metrics: The survey compares techniques by approach, technology, input information, learning algorithm, supervision, application, cost, accuracy, latency, and real-time capability.Cost distinguishes expensive device-per-person systems from cheap single-device systems, while latency is especially important for real-time applications.
  • Gesture recognition: Gesture recognition supports human-machine interaction where mice, keyboards, or touchscreens may be infeasible, including for elderly or disabled users and infection-sensitive environments.The survey presents approaches, technologies, advantages, disadvantages, and applications for gesture recognition.
  • Gesture recognition: Device-free RFID gesture systems infer gestures from changes in tag counts, RSSI, or phase values measured by passive RFID tags.LSI uses successful-read counts relative to an unobstructed reference, while Smart surface clusters RSSI disturbances with K-means.
  • Gesture recognition: The included gesture-recognition table organizes techniques by approach, technology, advantages, disadvantages, and applications.The supplied passages identify the table’s scope but do not provide its individual entries.
  • Gesture recognition: Ding et al.’s RFID system recognizes touchscreen gestures and airborne English letters with 91% accuracy, without prior training and in real time.Performance degrades for high-speed gestures, and users must remain within ≤5 cm of the tagged plate.

2) Posture Recognition:

Posture-recognition research includes device-free RFID and broader activity-recognition systems using learned signal patterns. The surveyed examples span feature selection, multiple classifiers, denoising, sequence matching, and RF-radar sensing.

  • Posture recognition: RF-Care uses environmental passive RFID tag arrays to capture posture-induced RSSI disturbances in indoor office and home scenarios.It applies feature-selection methods, SVM for steady postures, and HMM for posture-transition detection.
  • Posture recognition: The posture-recognition table organizes techniques by approach, technology, advantages, disadvantages, and applications.The supplied passages identify the table’s scope but do not provide its individual entries.
  • Posture recognition: Yao et al.’s dictionary-based RFID system segments continuous data into activities, learns one dictionary per activity without supervision, and reports better accuracy than comparison approaches.Its reported latency is around 4.5 seconds per recognized activity, which may be too slow for some applications.
  • Posture recognition: R&P combines RFID phase and RSSI values, denoises them separately, and uses modified T-DTW matching that reduces matching time by 60%.The technique is evaluated across different realistic settings.
  • Posture recognition: RF-radar achieved 95.3% accuracy for office desk-work and 34.9% for checkout-counter activities, with performance reported as better than IMU in the experiments.Combining RF-radar and IMU further improved accuracy, and increasing RF-radar projections could increase recognition accuracy.

3) Behavior Recognition:

Behavior recognition covers monitoring people and customers, while fall detection targets abnormal activity important in healthcare and security. The surveyed methods use RFID, wireless signals, floor tags, and wearable sensors for detection and classification.

  • Behavior recognition: Behavior recognition supports remote monitoring in elderly-care settings and customer analysis in shopping centers, including interests, preferences, and item relationships.RFID-based systems analyze wireless-signal changes around tagged store items to identify shopping behavior.
  • Behavior recognition: Customer behavior systems detect browsing, item picking, and item correlations from disturbances in RFID phase readings around tagged merchandise.CBID additionally identifies popular items and explicit or implicit relationships between items.
  • Fall detection: Falls are abnormal transitions from standing, sitting, or walking to reclining without control, with heightened risks for older people.The survey motivates fall detection through healthcare and security applications where abnormal activity should be identified.
  • Comparison tables: The fall-detection table organizes techniques by approach, technology, advantages, disadvantages, and applications, while the daily-living table covers the same comparison dimensions.The supplied passages identify both table scopes but do not provide their individual entries.
  • Fall detection: Device-free fall-detection methods use wireless CSI, floor sensors, or passive RFID tags hidden in carpet grids to detect signal or observation changes.RFID carpet systems represent blocked and readable tags as binary images, select a likely fall region, and classify activities using extracted features.
  • Fall detection: TagFall detects falls and their direction from abrupt RSSI changes by modeling normal-activity clusters and identifying anomalous patterns.The method uses Angle Based Outlier Detection for mining RSSI clustering patterns.

5) Activities of Daily Living:

Activities of daily living are recognized with wearable, object-tagged, dense-sensing, and hybrid systems, with device-free approaches prominent in the surveyed work. Applications include smart homes, elder care, monitoring, and detecting abnormal routines.

  • 5) Activities of Daily Living:: ADL systems support smart-home adaptation, independent living, remote monitoring, and detection of abnormal routines in older adults and patients.Examples include monitoring Alzheimer’s patients and detecting nycthemeral shifts that may help identify dementia-related disease earlier.
  • 5) Activities of Daily Living:: Wearable and object-tagged approaches can constrain users because they require carrying sensors or interacting with tagged objects.These constraints motivate interest in device-free sensing, although device-free approaches also have challenges.
  • 5) Activities of Daily Living:: A hybrid wearable-and-RFID system performed better than using either accelerometer or RFID data separately.The comparison evaluated accelerometer-only, RFID-only, and combined inputs.
  • 5) Activities of Daily Living:: Device-free dense sensing captures activities through sensors deployed in the environment, avoiding the need for users to carry devices.The surveyed work describes dense sensing with motion, pressure, temperature, proximity, and infrared sensors.
  • 5) Activities of Daily Living:: ADL recognition spans wearable, object-tagged, dense-sensing, and hybrid sensor deployments, with accelerometers, RFID, proximity, pressure, temperature, and infrared sensors used.Hybrid systems combine multiple technologies, such as wearable accelerometers with RFID-tagged objects.

C. Motion Based Activities

Motion-based activity recognition covers presence, motion detection, tracking, localization, and trajectory mining, with RFID central to tracking and indoor localization. The surveyed systems use signal changes or interference patterns but remain sensitive to environmental conditions and deployment constraints.

  • C. Motion Based Activities: RFID technology leads motion-based tracking and indoor localization, where environmental tags and fixed readers commonly provide low-cost, high-accuracy sensing.The survey identifies tracking and localization as substantially researched motion-based sub-areas.
  • C. Motion Based Activities: Device-free RFID tracking is affected by environmental interference, and some systems lack deployment details or support only limited object configurations.Reported constraints include direction dependence, reflective metallic objects, approximately 4-meter concrete-wall range, and single-object tracking in Tadar.
  • C. Motion Based Activities: RFID-based device-free tracking uses tag-signal changes, RSSI patterns, interference, or reflected signals to detect movement, location, and trajectories.Methods include RSSI-based coordinate modeling, binary interference maps, and reflected-signal tracking beyond walls.
  • C. Motion Based Activities: Frequent-trajectory systems train on tag disturbances or RSSI-derived binary signals, then compare observed activity with learned normal trajectories.The trajectory-mining approach separates training from monitoring and labels activity normal or suspicious through comparison.

2) Motion Detection:

Motion detection and people counting use device-free RFID, Wi-Fi, other RF, and conventional sensors to infer presence, movement, or population size. These approaches support surveillance, smart homes, healthcare, and facility monitoring but retain scope and deployment limitations.

  • 2) Motion Detection:: Motion-based recognition supports surveillance, security, smart homes, health monitoring, crowd management, elder care, and traffic management.Presence detection is described as foundational for adapting smart-home environments and detecting intrusions.
  • 2) Motion Detection:: Device-free motion detection exploits changes in Wi-Fi or RFID signals to detect human presence, movement, and direction without requiring a carried device.Examples include Wi-Fi systems such as FIMD, RoMD, and MoSense, and RFID systems such as EMoD and RF-HMS.
  • 2) Motion Detection:: People counting combines image-based, binary-sensor, mechanical-barrier, Wi-Fi, and RF methods, with wireless approaches offering economical, privacy-preserving use of existing infrastructure.RSS is used because it varies with the number of people in an environment.
  • 2) Motion Detection:: RFID-based R# estimates crowd size from variance in backscattered-signal RSS, but it cannot count more than ten people and performs poorly when people move quickly.The study also leaves effects of metallic objects, participant height, and time complexity unaddressed.
  • 2) Motion Detection:: The survey characterizes people counting as the least-developed motion-based sub-area, whereas tracking and localization have received substantial research attention.The authors suggest that specialized sensors requiring minimal processing may partly explain the difference.

D. Interaction Based Activities

Interaction-based activity recognition interprets how people manipulate objects or surfaces, especially through RFID-based sensing. Passive RFID tags enable low-maintenance interactive interfaces that classify gestures, object states, and trajectories.

  • D. Interaction Based Activities: Interaction-based recognition is motivated by applications such as entertainment and alternative human-computer interfaces beyond conventional mouse-and-keyboard input.Users can control machines through body gestures or interactions with tagged objects.
  • D. Interaction Based Activities: Passive RFID tags are attractive for human-computer interaction because they are battery-free, inexpensive, maintenance-free, and easy to attach to objects.The surveyed systems use tags on toys, paper, surfaces, and other objects to provide machine inputs.
  • D. Interaction Based Activities: RFID interaction systems detect gestures and object interactions through changes in RSSI, phase, read rate, impedance, or tag trajectories.IDSense classifies object states such as touch, still, and swipe, while PaperID and Rio turn paper or surfaces into interactive inputs.
  • D. Interaction Based Activities: PaperID recognizes touch, swipe, cover, wave, slide, and free-air gestures, while Rio converts surfaces into touchpads without hardware modification.PaperID addresses interference from densely placed tags using a half-antenna design.
  • D. Interaction Based Activities: Pantomime recognizes gestures from the trajectories of objects carrying passive RFID tags using one antenna per location.It addresses reduced reading rates and tag coupling by selectively reading target tags and boosting the reading rate.

Summary:

Human activity recognition supports healthcare, intelligent environments, and human-object interaction, with device-free and RFID approaches enabling practical sensing and new interfaces.

  • Summary:: RFID-based interaction systems support smart surfaces, touchpads, and gesture recognition using interactions with ordinary objects.Passive RFID tags can be attached to objects, while research can convert common surfaces such as paper into touchpads or smart surfaces.
  • Summary:: Human activity recognition is applied across health, security, surveillance, entertainment, and human-computer interaction.The field interprets activities from information collected by sensors and supports several application areas.
  • A. Elder Health Care: Human activity recognition supports remote elder monitoring, fall detection, medication management, exercise management, and independent living.It can also assist with early detection and treatment support for patients with Alzheimer’s and dementia.
  • B. Intelligent Environment: Smart environments use activity recognition to automate homes and remotely monitor patients in context-aware care centers.Systems can adjust lighting and climate when residents enter or leave and support independent living for older adults.

C. Security and Surveillance

Security and surveillance have shifted from human guards toward automated monitoring, while activity-recognition research still faces challenges in complex, multi-person settings.

  • C. Security and Surveillance: Surveillance cameras can monitor continuously, but human effort is still needed to interpret security footage.This motivates activity-recognition methods for security and surveillance applications.
  • C. Security and Surveillance: Human activity recognition is important for interpreting behavior in security and surveillance, but the supplied discussion identifies unresolved complexity rather than a complete solution.The paper presents these issues as open research opportunities.
  • C. Security and Surveillance: Current systems mainly recognize simple activities performed by one subject, leaving composite activities as a major research challenge.Exercise is an example of a composite activity formed from sitting, standing, and running.
  • C. Security and Surveillance: Most existing solutions focus on a single subject, although real-world situations can involve multiple people simultaneously or jointly.Examples include people interacting in kitchens or living rooms and activities such as handshakes or hugs.

Concurrent Activities:

Human activity recognition remains limited by assumptions about one activity, known normal behaviors, stable users, and fixed environments, motivating more intelligent and robust systems.

  • Concurrent Activities:: Current systems generally assume that a person performs only one activity at a time, leaving concurrent activities such as reading while drinking insufficiently studied.This assumption is more plausible for ambulatory activities such as walking and running than for everyday multitasking.
  • Concurrent Activities:: Recognition performance can degrade when different people perform the same activity or when one person changes pace or style.This variability remains a limitation of current activity-recognition solutions.
  • B. Need for Intelligent Solutions: Existing systems often recognize only trained, normal activities and cannot reliably detect rare abnormal behaviors or predict what will happen next.Abnormal-activity recognition is important for security and healthcare but is hindered by ambiguous definitions and limited training data.
  • Concurrent Activities:: Most solutions require extensive training, may need retraining after environmental changes, and lack a common benchmark for evaluation.The paper calls for environment-independent systems and standardized data and experimental setups.
  • Concurrent Activities:: Device-free approaches avoid requiring users to carry devices but remain vulnerable to environmental interference.Environmental factors can affect the performance of many proposed systems.

Need for Standard Testing Setup:

The survey identifies major obstacles to evaluating human activity recognition techniques: heterogeneous goals, hardware, classifiers, environments, and incomplete reporting. It therefore highlights the need for standardized evaluation and more comprehensive technical documentation.

  • Need for Standard Testing Setup:: No standard setup or benchmark currently enables consistent evaluation of human activity recognition solutions.The survey calls for a system through which new and existing solutions can be evaluated.
  • Need for Standard Testing Setup:: Comparisons are difficult because sub-areas prioritize different objectives, such as real-time processing for gestures versus accuracy for activities of daily living.
  • Need for Standard Testing Setup:: Different hardware, sensing approaches, and machine-learning classifiers make cross-method comparisons challenging because each approach has distinct advantages and disadvantages.
  • Need for Standard Testing Setup:: Experimental results depend on the setting, since wearable, object-tagged, and other approaches may be tested in different environments such as rooms or kitchens.
  • Need for Standard Testing Setup:: The reviewed literature often omits latency, complexity, classifier, and online-versus-offline details, limiting reproducibility and interpretation.The survey recommends reporting latency, complexity, and limitations more explicitly.
  • Need for Standard Testing Setup:: The survey organizes nearly all activity-recognition sub-fields into action-based, motion-based, and interaction-based categories, with ten sub-categories and emphasis on device-free RFID approaches.It also reviews applications, compares techniques, and discusses open research issues and future directions.
Loading 1906.05074v1…