PaperlayerView exact revision on arXiv

Source-linked AI summary

WiGest: A Ubiquitous WiFi-based Gesture Recognition System

Heba Abdelnasser, Moustafa Youssef, Khaled A. Harras

arXiv:1501.04301v2cs.HC

TL;DR

WiGest addresses the gap between specialized or constrained gesture interfaces and deployable fine-grained hands-free control using standard WiFi equipment. It recognizes hand gestures from RSSI primitives and gesture families, then maps them to application actions without training. The system achieves 87.5% primitive accuracy with one AP, 96% with three overheard APs, and 96% application-action classification accuracy, including through-wall scenarios and interfering humans.

  • Problem

    Existing gesture systems have sensing, deployment, training, or control limitations, while WiFi-based approaches generally do not detect fine-grained hand gestures near a specific mobile device.

  • Method

    WiGest uses unmodified off-the-shelf WiFi devices to extract RSSI primitives, compose gesture families with attributes, and map them to application actions while filtering noise and interference.

  • Results

    87.5% basic-primitive accuracy with one AP increased to 96% with three overheard APs, while application-action classification reached 96% and remained robust to interfering humans.

  • Takeaways & Limitations

    WiGest supports ubiquitous hands-free gesture-based interaction with mobile devices using standard WiFi equipment, without extra sensors, equipment modification, or training.

Abstract

from arXiv · show

We present WiGest: a system that leverages changes in WiFi signal strength to sense in-air hand gestures around the user's mobile device. Compared to related work, WiGest is unique in using standard WiFi equipment, with no modi-fications, and no training for gesture recognition. The system identifies different signal change primitives, from which we construct mutually independent gesture families. These families can be mapped to distinguishable application actions. We address various challenges including cleaning the noisy signals, gesture type and attributes detection, reducing false positives due to interfering humans, and adapting to changing signal polarity. We implement a proof-of-concept prototype using off-the-shelf laptops and extensively evaluate the system in both an office environment and a typical apartment with standard WiFi access points. Our results show that WiGest detects the basic primitives with an accuracy of 87.5% using a single AP only, including through-the-wall non-line-of-sight scenarios. This accuracy in-creases to 96% using three overheard APs. In addition, when evaluating the system using a multi-media player application, we achieve a classification accuracy of 96%. This accuracy is robust to the presence of other interfering humans, highlighting WiGest's ability to enable future ubiquitous hands-free gesture-based interaction with mobile devices.

I. INTRODUCTION

WiGest addresses limitations of conventional gesture interfaces by using standard WiFi signal changes to recognize hands-free gestures around a specific mobile device. It processes signal primitives and gesture attributes while addressing noise, interference, variability, and energy efficiency, achieving strong evaluation results.

  • Motivation: Conventional gesture systems can be sensitive to lighting, require specialized instrumentation or sensors, demand device handling, or need line of sight.These limitations motivate WiFi-based hands-free interaction for mobile applications.
  • Motivation: WiFi-based systems analyze RSSI or CSI changes caused by human motion, but existing systems do not provide fine-grained control of a specific user device.Hands-free input is especially useful when touch interaction is difficult because hands are wet, dirty, busy, or gloved.
  • System contribution: WiGest uses hand-induced WiFi signal-strength changes at a specific device to identify rising, falling, and pause primitives, then maps gesture combinations and attributes to application actions.Recognized attributes include primitive speed and magnitude, while gesture families can be mapped to distinguishable actions.
  • Challenges and design: WiGest addresses noisy RSSI, gesture variability, interfering people, changing signal polarity, and mobile energy constraints through signal processing, multiple APs, a preamble, thresholding, and linear-time wavelets.The preamble identifies gesture engagement and helps reduce false positives from interfering humans.
  • Evaluation: 96% classification accuracy was achieved for a multimedia-player application, robust to interfering humans.The prototype was evaluated on off-the-shelf laptops in an apartment and an engineering-building environment using more than 1000 primitive actions and gestures in realistic scenarios.

II. RELATED WORK

Related gesture-recognition approaches use specialized sensing or training and often operate in controlled settings. WiGest instead targets fine-grained, deployable hand gestures with off-the-shelf WiFi components, accepting additional noise and interference challenges.

  • Conventional sensing: Vision, inertial, ultrasonic, and infrared systems can suffer from lighting sensitivity, installation overhead, dedicated sensors, or line-of-sight requirements.These constraints limit deployment across some mobile and hands-free scenarios.
  • WiFi-based recognition: RSSI- and CSI-based WiFi activity-recognition systems support presence detection, tracking, and activity recognition, but many require training or controlled environments.They generally do not detect fine-grained hand gestures near mobile devices.
  • RF-based gesture systems: WiVi and WiSee detect fine-grained RF gestures using radar-like or Doppler-based techniques, but these approaches require special hardware.WiGest is positioned as an off-the-shelf alternative for gesture recognition.
  • WiGest distinction: WiGest works with off-the-shelf WiFi components, making it more deployable while requiring it to handle greater noise and interference.Its design therefore addresses practical signal-quality challenges associated with commodity equipment.
  • System organization: WiGest organizes recognition into primitives, gestures, and applications, allowing application actions to share a gesture while differing in parameters.For example, play and fast-forward can use the same gesture with different repetition counts.

III. WIGEST CONCEPTUAL OVERVIEW

WiGest converts raw RSSI changes into primitives, combines primitive sequences into gesture families, and maps those families and attributes to application actions. This layered representation separates signal detection from gesture selection and application control.

  • Primitives: The primitives layer detects RSSI rising edges, falling edges, and pauses corresponding respectively to moving away, moving toward, and holding a hand still over the device.For rising and falling edges, WiGest extracts motion speed and signal-change magnitude.
  • Gestures: Higher-level gestures combine primitive sequences; an up-down gesture, for example, consists of a rising edge followed by a falling edge.The system uses raw signal primitives as the building blocks for gesture recognition.
  • Gesture Families: A gesture family groups gestures with the same primitive sequence, such as up-down, right-left, and left-right gestures sharing a falling-then-rising sequence.Families let developers choose a suitable physical gesture while preserving the same signal-level representation.
  • Gesture Families: Gesture attributes include repetition count and repetition frequency, enabling gestures in one family to express different application meanings.These attributes distinguish, for example, one repetition from multiple consecutive repetitions.
  • Application Actions: The application layer maps actions to gesture families and selects gestures or attributes such as movement speed to control action parameters.A media player can map a rightward movement to play and hand speed to the rate of volume change.

IV. THE WIGEST SYSTEM

WiGest processes noisy WiFi signals through three stages: primitive extraction, gesture identification, and action mapping. Its extraction pipeline uses wavelet analysis to denoise signals and identify time-localized gesture primitives.

  • WiGest's processing flow comprises Primitives Extraction, Gesture Identification, and Action Mapping stages.
  • The Primitives Extraction stage reduces noise with DWT, then extracts rising edges, falling edges, and pauses from raw signals.
  • DWT recursively splits signals into approximation and detail coefficient vectors across multiple levels for multiscale time-frequency analysis.
  • The system uses Haar wavelet basis functions, with coefficients computed from the input signal using dot products with discrete orthogonal functions.
  • Wavelet denoising decomposes the signal, thresholds detail coefficients, and reconstructs the signal to reduce false edges caused by wireless noise.

2) Noise Reduction:

WiGest combines computationally efficient wavelet denoising with adaptive frequency analysis and DWT-based edge detection. The resulting module translates RSSI signals into rising edges, falling edges, and pauses while fusing detections from multiple APs.

  • Wavelet denoising runs in linear time, makes no particular assumptions about signal nature, and permits signal discontinuities.
  • Wavelet analysis provides variable frequency resolution for detecting hand-motion edges despite speed differences across users or sessions.
  • The spectrogram localizes two motion frequencies through 2D local-maximum extraction, selecting DWT levels of interest for edge detection.
  • DWT detailed coefficients represent rising edges as local minima and falling edges as local maxima, while pauses require stable RSSI variance for at least 0.5 second.
  • Multiple overheard APs improve accuracy and noise resilience by combining primitive detections through majority voting.

4) Primitives attributes:

WiGest identifies gesture attributes and boundaries after extracting signal primitives. It uses speed, magnitude, frequency, and count features, with a preamble adapting interpretation to users and environments.

  • Each detected edge receives speed and magnitude labels based on edge duration and RSSI-derived distance thresholds.Speeds are high below 0.75 seconds, medium between 0.75 and 1.5 seconds, and low above 1.5 seconds; magnitude is high above 0.55 ft and low below it.
  • Gesture processing extracts gesture types and attributes through segmentation followed by identification and template matching.
  • A distinctive preamble marks gesture engagement, suppressing interpretation of environmental actions or noise when the preamble is absent.
  • The preamble determines whether signal polarity should be flipped and estimates magnitude and motion-frequency thresholds for adaptation.
  • WiGest encodes rising edges, falling edges, and pauses as positive signs, negative signs, and zeros before matching the resulting sequence to gesture templates.

C. Action Mapping

WiGest maps gesture families and their attributes to application actions, while evaluation spans varied environments, distances, orientations, and line-of-sight conditions. The system's reported accuracy improves with additional APs and supports realistic deployment scenarios.

  • C. Action Mapping: Application developers map gesture families and attributes to actions, such as distinguishing media-player play from fast-forward through gesture repetition.
  • Evaluation: Evaluation uses an apartment and an engineering-building environment with off-the-shelf laptops, standard APs, three users, over 1000 primitive actions, and interfering people.
  • Evaluation: The three evaluation scenarios are no-wall line-of-sight, through-one-wall, and through-two-walls configurations between the device and AP.

A. Primitives Layer Evaluation

WiGest’s primitive detection remains accurate across user orientations and improves substantially when multiple overheard access points are combined.

  • 90.5% overall primitive detection accuracy is achieved across East, West, North, and South orientations at 14 ft.The West orientation performs best, while East performs worst because the body blocks the line of sight and attenuates RSSI.
  • 96% primitive detection accuracy is reached with three APs, increasing to 100% when the device hears seven APs.A majority vote across overheard APs selects the correct primitive.

3) Impact of Distance:

WiGest maintains primitive and gesture-family recognition over practical distances, while detecting gesture attributes with high accuracy.

  • 3) Impact of Distance:: More than 87% primitive detection accuracy is maintained for AP distances up to 26 ft.Accuracy decreases with distance because weaker RSSI produces lower signal-to-noise ratio and smaller gesture-induced signal changes.
  • High true-positive and low false-positive rates are achieved while detecting primitive speed and distance attributes.The evaluation uses 350 samples of each primitive, totaling 1050 samples, at a 14 ft AP distance.
  • 92.6% gesture-family detection accuracy is achieved across 192 gestures performed at four orientations.This exceeds primitive detection accuracy because additional gesture processing is applied.
  • 96% exact count detection and 98% detection within a count error of one are achieved for gesture repetitions.Repetition frequency within one second is detected with 93% accuracy.

C. Whole-home Gesture Recognition Case Study

The apartment case study shows that WiGest can classify media-player actions across line-of-sight and through-wall locations, while its preamble and signal diversity address interference.

  • C. Whole-home Gesture Recognition Case Study: 96% overall classification accuracy is achieved for seven media-player actions across eight apartment locations.The locations cover line-of-sight, through-one-wall, and through-two-walls scenarios, totaling 1120 actions.
  • 89% gesture detection accuracy is maintained with one AP and four interfering users performing random actions.The authors caution that more interfering users in a small area may affect accuracy and require further investigation.
  • Interfering users more than 4 ft away have no effect on accuracy, while users closer than 3 ft reduce it.AP diversity and WiGest’s unique preamble mitigate the impact of nearby interference.

3) Distance from the interfering human:

WiGest combines RSSI-based gesture sensing with lightweight processing and reports robust performance across environments and interference conditions, while ongoing work seeks finer-grained recognition.

  • WiGest uses no equipment modifications, extra sensors, or training before deployment.The system uses WiFi RSSI changes to detect hand motions around a user device.
  • Signal denoising, primitive extraction, false-positive reduction, and polarity adaptation address core system challenges.A threshold-detected preamble and linear-time wavelet processing support energy efficiency.
  • VI. CONCLUSION: 87.5% primitive detection accuracy is achieved with one AP up to 26 ft, including through-the-wall non-line-of-sight scenarios.Using three overheard APs increases accuracy to 96%.
  • 96% classification accuracy is achieved for application actions, with results robust to interfering humans.The evaluation spans two environments and supports future hands-free gesture interaction with mobile devices.
  • Future extensions will use detailed CSI and other ubiquitous wireless technologies to improve accuracy and produce finer-grained gesture families.
Loading 1501.04301v2…