PaperlayerView exact revision on arXiv

Source-linked AI summary

Different Approaches for Human Activity Recognition: A Survey

Zawar Hussain, Michael Sheng, Wei Emma Zhang

arXiv:1906.05074v1cs.CV

TL;DR

Human activity recognition lacks a unified view across sensing approaches and activity types despite applications in health, security, entertainment, and intelligent environments. This survey reviews 2010–2018 research, emphasizing device-free and RFID solutions, and organizes it into three categories and 10 sub-topics. It compares techniques and analyzes design attributes, metrics, trends, applications, open issues, and future directions.

Abstract

from arXiv · show

Human activity recognition has gained importance in recent years due to its applications in various fields such as health, security and surveillance, entertainment, and intelligent environments. A significant amount of work has been done on human activity recognition and researchers have leveraged different approaches, such as wearable, object-tagged, and device-free, to recognize human activities. In this article, we present a comprehensive survey of the work conducted over the period 2010-2018 in various areas of human activity recognition with main focus on device-free solutions. The device-free approach is becoming very popular due to the fact that the subject is not required to carry anything, instead, the environment is tagged with devices to capture the required information. We propose a new taxonomy for categorizing the research work conducted in the field of activity recognition and divide the existing literature into three sub-areas: action-based, motion-based, and interaction-based. We further divide these areas into ten different sub-topics and present the latest research work in these sub-topics. Unlike previous surveys which focus only on one type of activities, to the best of our knowledge, we cover all the sub-areas in activity recognition and provide a comparison of the latest research work in these sub-areas. Specifically, we discuss the key attributes and design approaches for the work presented. Then we provide extensive analysis based on 10 important metrics, to give the reader, a complete overview of the state-of-the-art techniques and trends in different sub-areas of human activity recognition. In the end, we discuss open research issues and provide future research directions in the field of human activity recognition.

I. INTRODUCTION

Human activity recognition research spans vision-based and sensor-based approaches, with growing interest in device-free sensing. This survey organizes 2010–2018 research into action-, motion-, and interaction-based categories, emphasizing device-free and RFID solutions.

Approach Motivation

  • Vision-based recognition can provide good results but raises privacy concerns and depends on adequate lighting.Because prior surveys already covered vision-based techniques, this survey excludes them from its main scope.Source: §I · p. 1

Sensor Deployment

  • Sensor-based approaches are divided by deployment into wearable, object-tagged, and dense-sensing categories.Wearable systems require users to carry sensors, while object-tagged systems require interaction with specific instrumented objects.Source: §I · pp. 1–2
  • Device-free sensing places sensors in the environment so users can perform activities without carrying tags or devices.The approach is presented as more practical, although environmental interference can introduce noise into captured data.Source: §I · p. 2

Survey Scope

  • The survey classifies activity recognition literature into action-based, motion-based, and interaction-based activities, divided into 10 sub-areas.Action-based areas include gestures, posture, falls, daily living, behavior, and ambient assisted living; motion-based areas include tracking, motion detection, and people counting; interaction-based work covers human-object interaction.Source: §I · p. 2

Related Surveys

  • Earlier surveys commonly focused on a single approach, including device-free radio, RFID, radio-based, Wi-Fi-based, or broader RF activity recognition.Their emphases included categorization, system steps, comparisons using selected metrics, and future research directions.Source: A. Radio Frequency Based · pp. 2–3

B. Sensor Based

Previous surveys organize sensor-based activity recognition by sensing modality, learning strategy, application, body placement, and activity type. The surveyed literature is broad, but earlier reviews often lacked comprehensive cross-approach comparison and detailed analysis of RFID device-free methods.

Sensor-Based Surveys

  • Sensor-based surveys classified research by vision versus sensing, data-driven versus knowledge-driven methods, and modalities including wearable and dense sensing.Deep-learning surveys additionally organized work by sensor modality, deep model, and application area.Source: B. Sensor Based · p. 3

Wearable Device Surveys

  • Wearable-sensor surveys examined design issues including sensor selection, data collection, recognition performance, processing, and energy consumption.They also categorized systems as supervised online, supervised offline, and semisupervised offline.Source: C. Wearable Device Based · p. 3
  • Other reviews distinguished global body motion from local interaction activity and summarized mobile-phone-based recognition using built-in sensors.These classifications emphasized activity scope and sensor placement or device platform.Source: C. Wearable Device Based · p. 3

Survey Coverage

  • Earlier surveys primarily focused on one approach, such as sensor-based, vision-based, wearable, mobile-phone, or RF-based recognition.The survey identifies this specialization as a limitation of prior coverage.Source: D. Vision Based · pp. 3–4

Survey Gap

  • The survey targets a gap by comprehensively analyzing device-free RFID activity-recognition research while comparing advantages and disadvantages across techniques.It presents this focus as absent from previous comprehensive survey coverage.Source: D. Vision Based · p. 4

III. TECHNICAL BACKGROUND

Human activity recognition combines sensor deployment, data collection, preprocessing and feature selection, and machine-learning inference. The technical background contrasts cameras, depth cameras, wireless and wearable sensors, and RFID-based sensing.

Recognition Process

  • Human activity recognition consists of sensor selection and deployment, data collection, preprocessing and feature selection, and machine-learning inference.These four phases form the general recognition pipeline.Source: §III · p. 4, Figure 2 · p. 4

Sensing Technologies

  • Sensors may be attached to objects, worn by users, or deployed in the environment, with common examples including accelerometers, motion, biosensors, gyroscopes, pressure, proximity, and RFID sensors.Cheap, portable sensors can sense and communicate information wirelessly.Source: §III · p. 4

Vision-Based Sensing

  • Surveillance-camera systems process videos and images manually or automatically to recognize human activities.Traditional cameras are limited by darkness, whereas depth cameras such as Kinect can operate in total darkness and provide RGB, depth, and audio streams.Source: A. Surveillance Cameras · p. 4, B. Depth Cameras · p. 5

Wireless Sensing

  • Wireless sensing uses properties such as Wi-Fi Channel State Information for applications including localization, tracking, and fall detection without requiring users to carry devices.The passage identifies unobtrusive operation as a major Wi-Fi advantage.Source: C. Wi-Fi · p. 5

Sensor Examples

  • A multi-axis accelerometer measures acceleration simultaneously along the x, y, and z directions and supports recognition tasks ranging from gestures and posture to falls and tracking.It is also used in ambient assisted living and activities-of-daily-living solutions.Source: 1) Accelerometer · p. 5

2) Magnetometer:

The supplied passages introduce sensing technologies and RFID components used to capture information for human activity recognition. They also frame activity recognition as covering bodily actions, object interactions, presence, motion, and tracking.

Sensor types

  • Magnetometers detect magnetic-field changes caused by human activity, while motion sensors detect movement or presence in an observed area.Proximity sensors detect nearby objects without physical contact and are widely used for gesture recognition.Source: 2) Magnetometer · p. 5, 4) Proximity Sensor · p. 5

RFID sensing

  • RFID readers emit radio waves, receive tag backscatter, and collect tag information through an antenna.Tags may be active, with batteries and longer range, or passive, harvesting energy from reader radio waves.Source: E. RFID · p. 5

Recognition scope

  • Activity recognition includes whole-body actions, body-part gestures, object interactions, presence or motion detection, and human trajectory tracking.Examples include walking, running, sitting, cooking, intrusion detection, and movement tracking.Source: §IV · pp. 5–6

approach.

This section establishes comparison criteria for human activity-recognition techniques and surveys action-based activities, especially gesture recognition. It contrasts sensing approaches and highlights device-free RFID methods alongside their practical constraints.

Comparison metrics

  • The survey compares techniques by approach, technology, input information, learning algorithm, supervision, application, cost, accuracy, latency, and real-time capability.Cost distinguishes expensive device-per-person systems from cheap single-device systems, while latency is especially important for real-time applications.Source: approach · p. 6

Gesture recognition

  • Gesture recognition supports human-machine interaction where mice, keyboards, or touchscreens may be infeasible, including for elderly or disabled users and infection-sensitive environments.The survey presents approaches, technologies, advantages, disadvantages, and applications for gesture recognition.Source: 1) Gesture Recognition · p. 7
  • Device-free RFID gesture systems infer gestures from changes in tag counts, RSSI, or phase values measured by passive RFID tags.LSI uses successful-read counts relative to an unobstructed reference, while Smart surface clusters RSSI disturbances with K-means.Source: 1) Gesture Recognition · p. 7
  • The included gesture-recognition table organizes techniques by approach, technology, advantages, disadvantages, and applications.The supplied passages identify the table’s scope but do not provide its individual entries.Source: Table II · p. 8
  • Ding et al.’s RFID system recognizes touchscreen gestures and airborne English letters with 91% accuracy, without prior training and in real time.Performance degrades for high-speed gestures, and users must remain within ≤5 cm of the tagged plate.Source: 1) Gesture Recognition · p. 8

2) Posture Recognition:

Posture-recognition research includes device-free RFID and broader activity-recognition systems using learned signal patterns. The surveyed examples span feature selection, multiple classifiers, denoising, sequence matching, and RF-radar sensing.

Posture recognition

  • RF-Care uses environmental passive RFID tag arrays to capture posture-induced RSSI disturbances in indoor office and home scenarios.It applies feature-selection methods, SVM for steady postures, and HMM for posture-transition detection.Source: 2) Posture Recognition · pp. 8–9
  • The posture-recognition table organizes techniques by approach, technology, advantages, disadvantages, and applications.The supplied passages identify the table’s scope but do not provide its individual entries.Source: Table III · p. 9
  • Yao et al.’s dictionary-based RFID system segments continuous data into activities, learns one dictionary per activity without supervision, and reports better accuracy than comparison approaches.Its reported latency is around 4.5 seconds per recognized activity, which may be too slow for some applications.Source: 2) Posture Recognition · p. 9
  • R&P combines RFID phase and RSSI values, denoises them separately, and uses modified T-DTW matching that reduces matching time by 60%.The technique is evaluated across different realistic settings.Source: 2) Posture Recognition · p. 9
  • RF-radar achieved 95.3% accuracy for office desk-work and 34.9% for checkout-counter activities, with performance reported as better than IMU in the experiments.Combining RF-radar and IMU further improved accuracy, and increasing RF-radar projections could increase recognition accuracy.Source: 2) Posture Recognition · p. 9

3) Behavior Recognition:

Behavior recognition covers monitoring people and customers, while fall detection targets abnormal activity important in healthcare and security. The surveyed methods use RFID, wireless signals, floor tags, and wearable sensors for detection and classification.

Behavior recognition

  • Behavior recognition supports remote monitoring in elderly-care settings and customer analysis in shopping centers, including interests, preferences, and item relationships.RFID-based systems analyze wireless-signal changes around tagged store items to identify shopping behavior.Source: 3) Behavior Recognition · pp. 9–10
  • Customer behavior systems detect browsing, item picking, and item correlations from disturbances in RFID phase readings around tagged merchandise.CBID additionally identifies popular items and explicit or implicit relationships between items.Source: 3) Behavior Recognition · p. 10

Fall detection

  • Falls are abnormal transitions from standing, sitting, or walking to reclining without control, with heightened risks for older people.The survey motivates fall detection through healthcare and security applications where abnormal activity should be identified.Source: 4) Fall Detection · p. 10

Comparison tables

  • The fall-detection table organizes techniques by approach, technology, advantages, disadvantages, and applications, while the daily-living table covers the same comparison dimensions.The supplied passages identify both table scopes but do not provide their individual entries.Source: Table V · p. 11, Table VI · p. 12

Fall detection

  • Device-free fall-detection methods use wireless CSI, floor sensors, or passive RFID tags hidden in carpet grids to detect signal or observation changes.RFID carpet systems represent blocked and readable tags as binary images, select a likely fall region, and classify activities using extracted features.Source: 4) Fall Detection · p. 11
  • TagFall detects falls and their direction from abrupt RSSI changes by modeling normal-activity clusters and identifying anomalous patterns.The method uses Angle Based Outlier Detection for mining RSSI clustering patterns.Source: 4) Fall Detection · p. 11

5) Activities of Daily Living:

Activities of daily living are recognized with wearable, object-tagged, dense-sensing, and hybrid systems, with device-free approaches prominent in the surveyed work. Applications include smart homes, elder care, monitoring, and detecting abnormal routines.

5) Activities of Daily Living:

C. Motion Based Activities

Motion-based activity recognition covers presence, motion detection, tracking, localization, and trajectory mining, with RFID central to tracking and indoor localization. The surveyed systems use signal changes or interference patterns but remain sensitive to environmental conditions and deployment constraints.

C. Motion Based Activities

  • RFID technology leads motion-based tracking and indoor localization, where environmental tags and fixed readers commonly provide low-cost, high-accuracy sensing.The survey identifies tracking and localization as substantially researched motion-based sub-areas.Source: 1) Tracking · p. 15
  • Device-free RFID tracking is affected by environmental interference, and some systems lack deployment details or support only limited object configurations.Reported constraints include direction dependence, reflective metallic objects, approximately 4-meter concrete-wall range, and single-object tracking in Tadar.Source: 1) Tracking · pp. 15, 17
  • RFID-based device-free tracking uses tag-signal changes, RSSI patterns, interference, or reflected signals to detect movement, location, and trajectories.Methods include RSSI-based coordinate modeling, binary interference maps, and reflected-signal tracking beyond walls.Source: 1) Tracking · pp. 16–17
  • Frequent-trajectory systems train on tag disturbances or RSSI-derived binary signals, then compare observed activity with learned normal trajectories.The trajectory-mining approach separates training from monitoring and labels activity normal or suspicious through comparison.Source: 1) Tracking · p. 16

2) Motion Detection:

Motion detection and people counting use device-free RFID, Wi-Fi, other RF, and conventional sensors to infer presence, movement, or population size. These approaches support surveillance, smart homes, healthcare, and facility monitoring but retain scope and deployment limitations.

2) Motion Detection:

  • Motion-based recognition supports surveillance, security, smart homes, health monitoring, crowd management, elder care, and traffic management.Presence detection is described as foundational for adapting smart-home environments and detecting intrusions.Source: 2) Motion Detection · p. 17, 3) People Counting · p. 18
  • Device-free motion detection exploits changes in Wi-Fi or RFID signals to detect human presence, movement, and direction without requiring a carried device.Examples include Wi-Fi systems such as FIMD, RoMD, and MoSense, and RFID systems such as EMoD and RF-HMS.Source: 2) Motion Detection · pp. 17–18
  • People counting combines image-based, binary-sensor, mechanical-barrier, Wi-Fi, and RF methods, with wireless approaches offering economical, privacy-preserving use of existing infrastructure.RSS is used because it varies with the number of people in an environment.Source: 3) People Counting · p. 18
  • RFID-based R# estimates crowd size from variance in backscattered-signal RSS, but it cannot count more than ten people and performs poorly when people move quickly.The study also leaves effects of metallic objects, participant height, and time complexity unaddressed.Source: 3) People Counting · p. 19
  • The survey characterizes people counting as the least-developed motion-based sub-area, whereas tracking and localization have received substantial research attention.The authors suggest that specialized sensors requiring minimal processing may partly explain the difference.Source: 3) People Counting · p. 19

D. Interaction Based Activities

Interaction-based activity recognition interprets how people manipulate objects or surfaces, especially through RFID-based sensing. Passive RFID tags enable low-maintenance interactive interfaces that classify gestures, object states, and trajectories.

D. Interaction Based Activities

  • Interaction-based recognition is motivated by applications such as entertainment and alternative human-computer interfaces beyond conventional mouse-and-keyboard input.Users can control machines through body gestures or interactions with tagged objects.Source: D. Interaction Based Activities · p. 19, 1) Human Object Interaction · p. 19
  • Passive RFID tags are attractive for human-computer interaction because they are battery-free, inexpensive, maintenance-free, and easy to attach to objects.The surveyed systems use tags on toys, paper, surfaces, and other objects to provide machine inputs.Source: 1) Human Object Interaction · p. 19
  • RFID interaction systems detect gestures and object interactions through changes in RSSI, phase, read rate, impedance, or tag trajectories.IDSense classifies object states such as touch, still, and swipe, while PaperID and Rio turn paper or surfaces into interactive inputs.Source: 1) Human Object Interaction · pp. 19–20
  • PaperID recognizes touch, swipe, cover, wave, slide, and free-air gestures, while Rio converts surfaces into touchpads without hardware modification.PaperID addresses interference from densely placed tags using a half-antenna design.Source: 1) Human Object Interaction · pp. 19–20
  • Pantomime recognizes gestures from the trajectories of objects carrying passive RFID tags using one antenna per location.It addresses reduced reading rates and tag coupling by selectively reading target tags and boosting the reading rate.Source: 1) Human Object Interaction · pp. 20–21

Summary:

Human activity recognition supports healthcare, intelligent environments, and human-object interaction, with device-free and RFID approaches enabling practical sensing and new interfaces.

Summary:

  • RFID-based interaction systems support smart surfaces, touchpads, and gesture recognition using interactions with ordinary objects.Passive RFID tags can be attached to objects, while research can convert common surfaces such as paper into touchpads or smart surfaces.Source: Summary · p. 21
  • Human activity recognition is applied across health, security, surveillance, entertainment, and human-computer interaction.The field interprets activities from information collected by sensors and supports several application areas.Source: §V · p. 21

A. Elder Health Care

  • Human activity recognition supports remote elder monitoring, fall detection, medication management, exercise management, and independent living.It can also assist with early detection and treatment support for patients with Alzheimer’s and dementia.Source: A. Elder Health Care · p. 22

B. Intelligent Environment

  • Smart environments use activity recognition to automate homes and remotely monitor patients in context-aware care centers.Systems can adjust lighting and climate when residents enter or leave and support independent living for older adults.Source: B. Intelligent Environment · p. 22

C. Security and Surveillance

Security and surveillance have shifted from human guards toward automated monitoring, while activity-recognition research still faces challenges in complex, multi-person settings.

C. Security and Surveillance

  • Surveillance cameras can monitor continuously, but human effort is still needed to interpret security footage.This motivates activity-recognition methods for security and surveillance applications.Source: C. Security and Surveillance · p. 22
  • Human activity recognition is important for interpreting behavior in security and surveillance, but the supplied discussion identifies unresolved complexity rather than a complete solution.The paper presents these issues as open research opportunities.Source: §VI · p. 23
  • Current systems mainly recognize simple activities performed by one subject, leaving composite activities as a major research challenge.Exercise is an example of a composite activity formed from sitting, standing, and running.Source: Composite Activity · p. 23, A. Complex Activities · p. 23
  • Most existing solutions focus on a single subject, although real-world situations can involve multiple people simultaneously or jointly.Examples include people interacting in kitchens or living rooms and activities such as handshakes or hugs.Source: Multiple Subjects · p. 23

Concurrent Activities:

Human activity recognition remains limited by assumptions about one activity, known normal behaviors, stable users, and fixed environments, motivating more intelligent and robust systems.

Concurrent Activities:

  • Current systems generally assume that a person performs only one activity at a time, leaving concurrent activities such as reading while drinking insufficiently studied.This assumption is more plausible for ambulatory activities such as walking and running than for everyday multitasking.Source: Concurrent Activities · p. 23
  • Recognition performance can degrade when different people perform the same activity or when one person changes pace or style.This variability remains a limitation of current activity-recognition solutions.Source: Variability · p. 23

B. Need for Intelligent Solutions

Concurrent Activities:

Need for Standard Testing Setup:

The survey identifies major obstacles to evaluating human activity recognition techniques: heterogeneous goals, hardware, classifiers, environments, and incomplete reporting. It therefore highlights the need for standardized evaluation and more comprehensive technical documentation.

Need for Standard Testing Setup:

  • No standard setup or benchmark currently enables consistent evaluation of human activity recognition solutions.The survey calls for a system through which new and existing solutions can be evaluated.Source: Need for Standard Testing Setup · p. 24
  • Comparisons are difficult because sub-areas prioritize different objectives, such as real-time processing for gestures versus accuracy for activities of daily living.Source: Main Focus · pp. 24–25
  • Different hardware, sensing approaches, and machine-learning classifiers make cross-method comparisons challenging because each approach has distinct advantages and disadvantages.Source: Main Focus · p. 25
  • Experimental results depend on the setting, since wearable, object-tagged, and other approaches may be tested in different environments such as rooms or kitchens.Source: Experimental Setup · p. 25
  • The reviewed literature often omits latency, complexity, classifier, and online-versus-offline details, limiting reproducibility and interpretation.The survey recommends reporting latency, complexity, and limitations more explicitly.Source: Missing Details · p. 25
  • The survey organizes nearly all activity-recognition sub-fields into action-based, motion-based, and interaction-based categories, with ten sub-categories and emphasis on device-free RFID approaches.It also reviews applications, compares techniques, and discusses open research issues and future directions.Source: §VIII · p. 25

Related papers

Loading 1906.05074v1…