PaperlayerView exact revision on arXiv

Source-linked AI summary

RAPID: Retrofitting IEEE 802.11ay Access Points for Indoor Human Detection and Sensing

Jacopo Pegoraro, Jesus Omar Lacruz, Francesca Meneghello, Enver Bashirov, Michele Rossi, Joerg Widmer

arXiv:2109.04819v4eess.SP

TL;DR

Millimeter-wave human sensing often relies on dedicated radars, while standard WiFi packet sequences make micro-Doppler extraction difficult. RAPID retrofits IEEE 802.11ay access points to localize people and extract movement signatures, reporting strong tracking and recognition results against sub-6 GHz WiFi sensing.

  • Problem

    Millimeter-wave human sensing often depends on dedicated radar hardware, and extracting micro-Doppler from standard packet sequences is difficult because random phase offsets disrupt coherence.

    Source: §1 · pp. 1–2

  • Method

    RAPID uses IEEE 802.11ay beam-training fields to localize and track people, then analyzes reflected-packet CIR phase differences to extract micro-Doppler signatures.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    RAPID outperforms state-of-the-art sub-6 GHz WiFi sensing, tracks up to 5 concurrently moving subjects, and achieves 94% HAR accuracy and 90% person identification accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    RAPID combines radar-like human sensing with standard-compliant IEEE 802.11ay communication packets, avoiding a dedicated sensing infrastructure.

    Source: §1 · p. 2

    Lower-frequency systems can improve recognition of movements with little radial displacement relative to the receiver, such as sitting down.

    Source: §7 · p. 17

Abstract

from arXiv · show

In this work we present RAPID, the first joint communication and radar system based on next-generation IEEE 802.11ay WiFi networks operating in the 60 GHz band. Unlike existing approaches for human sensing at millimeter-wave frequencies, which rely on special-purpose radars, RAPID achieves radar-level sensing accuracy with IEEE 802.11ay access points, thus avoiding the burden of installing ad-hoc sensors. RAPID enables contactless human sensing applications, such as people tracking, Human Activity Recognition (HAR), and person identification without requiring modifications to the standard packet structure. Specifically, we leverage IEEE 802.11ay beam training to accurately localize and track multiple individuals within the same environment. Then, we propose a new way of using beam tracking to extract micro-Doppler signatures from the time-varying Channel Impulse Response (CIR) estimated from reflected packets. Such signatures are fed to a deep learning classifier to perform HAR and person identification. RAPID is implemented on a cutting-edge IEEE 802.11ay-compatible FPGA platform with phased antenna arrays, and evaluated on a large dataset of CIR measurements. It is robust across different environments and subjects, and outperforms state-of-the-art sub-6 GHz WiFi sensing techniques. Using two access points, RAPID reliably tracks multiple subjects, reaching HAR and person identification accuracies of 94% and 90%, respectively.

1 INTRODUCTION

RAPID retrofits IEEE 802.11ay access points for indoor sensing, addressing gaps in standard-compliant mmWave human sensing while reusing communication packets. It combines people tracking with micro-Doppler-based recognition and reports strong results against sub-6 GHz WiFi sensing.

1 INTRODUCTION

  • RAPID targets contactless sensing through existing 60 GHz WiFi infrastructure, avoiding dedicated sensing hardware while adding sensing alongside high-throughput communication.This supports the paper’s aim of indoor human and environment sensing with retrofitted IEEE 802.11ay hardware.Source: §1 · p. 1
  • Standard WiFi sensing faces limited bandwidth and multi-person tracking constraints, while extracting micro-Doppler from packet sequences is difficult because random phase offsets disrupt coherence.Prior 802.11ad CIR-based localization did not capture micro-Doppler information for HAR and person identification.Source: §1 · pp. 1–2
  • RAPID uses IEEE 802.11ay beam training to localize multiple people, then extracts their micro-Doppler signatures from reflected-packet CIR phase differences for HAR and person identification.Deep learning classifiers process the resulting spectrograms.Source: §1 · p. 2
  • The FPGA-based access points use phased arrays and full-duplex operation, sharing a local oscillator between transmit and receive chains to avoid random phase offsets.The receiver detects robust TRN-field Golay sequences without complex self-interference cancellation.Source: §1 · p. 2
  • RAPID must adapt sensing to standard Golay sequences, fixed sensing resolution, phased-array beam patterns, and the packet-rate and phase-coherence requirements of micro-Doppler estimation.Unlike radar chirps, Golay sequences cannot be tuned to the sensing scenario, requiring redesigned detection, range estimation, and angle-of-arrival methods.Source: §1 · p. 2
  • RAPID tracks up to 5 concurrently moving subjects, achieves 94% HAR accuracy and 90% person identification accuracy, and outperforms state-of-the-art sub-6 GHz WiFi sensing.It can combine information from multiple access points for improved performance.Source: §1 · p. 2

2 RELATED WORK

Prior WiFi and mmWave sensing approaches offer useful capabilities but face limitations in multi-person tracking, environmental robustness, deployment, or communication-standard compliance. RAPID uses standard-compliant IEEE 802.11ay packets to extract radar-like micro-Doppler signatures.

2 RELATED WORK

  • Sub-6 GHz WiFi systems are often limited to single-person scenarios and can be sensitive to environmental changes, while still falling short of radar-like sensing accuracy.Their relatively small bandwidth supports only coarse localization and tracking.Source: §2 · p. 3
  • Dedicated mmWave radars provide fine-grained sensing but have high deployment costs and limited working range, motivating reuse of existing mmWave communication links.The paper identifies reuse of communication links as a way to avoid additional hardware while maintaining radar-like sensing and detection performance.Source: §2 · p. 3
  • Earlier 802.11ad sensing work did not jointly preserve the communication packet structure and extract subjects’ micro-Doppler signatures for HAR and person identification.RAPID is presented as the first system to extract radar-like human movement signatures from standard-compliant 802.11ay packets.Source: §2 · p. 3

3 RAPID SENSING SYSTEM

RAPID uses IEEE 802.11ay beam-training and tracking fields to support indoor human sensing. Its processing blocks estimate location, tracking, and Doppler information from standard-compliant Golay-sequence CIR measurements.

3 RAPID SENSING SYSTEM

  • RAPID leverages in-packet beam training and tracking fields in IEEE 802.11ay networks for indoor human sensing.The system is organized around localization and tracking, HAR, and person identification.Source: §3 · p. 3
  • The system’s processing blocks estimate range, angle of arrival, and Doppler from CIR measurements obtained through standard-compliant Golay sequences.The paper introduces algorithms for these estimates along with CIR and CIR-phase models.Source: §3 · p. 3
  • The sensing pipeline supports people localization and tracking, human activity recognition, and person identification.The section details the processing blocks used for these tasks.Source: §3 · p. 3

3.1 System overview

RAPID’s overview links IEEE 802.11ay CIR estimation to human tracking, per-person micro-Doppler extraction, and deep-learning recognition. Tracking also enables identity assignment across activities, including when a person walks only later.

3.1 System overview

  • Each TRN unit provides a CIR estimate for a beam pattern, and combining these estimates gives an angular scan of surrounding people and objects.Candidate human reflection paths are detected from CIR amplitude using background subtraction and thresholding.Source: §3.1 · p. 4
  • Correlation-based angle estimation and an Extended Kalman Filter track detected people over time, while multiple access points can improve detection and address occlusions.The overview describes using estimated positions to select the CIR paths and beam patterns containing each person’s micro-Doppler information.Source: §3.1 · p. 4
  • RAPID computes each person’s micro-Doppler spectrum from selected complex CIR portions and feeds spectrograms to CNN modules for activity recognition and gait-based person identification.With multiple access points, classification decisions can use the AP with the most confident output.Source: §3.1 · p. 4
  • Tracking links gait-based identity to a person’s location across activities, so a person can be identified after beginning to walk and tracked during earlier non-walking activities.The overview states the reverse sequence also works: a person first sits or performs other activities, then later starts walking.Source: §3.1 · p. 4

3.2 CIR estimation

RAPID uses standard-compliant Golay sequences to estimate the channel impulse response (CIR) and interpret its paths as reflections from the environment. Repeated estimates across packets and beam patterns provide time- and direction-dependent information for sensing.

3.2 CIR estimation

  • The co-located transmitter and receiver use reflected signals to retrieve each reflector’s distance, angular position, moving velocity, and micro-Doppler information.The wide transmission bandwidth gives the CIR fine-grained environmental information.Source: §3.2 · p. 4
  • The CIR is a discrete vector of complex path gains obtained by correlating received signals with predefined Golay sequences; each path maps to a reflector distance using its delay.The delay grid is set by transmission bandwidth, with delay resolution 1/B.Source: §3.2 · p. 4
  • CIR estimates across packets sample the channel over time at interval Tc, while estimates across beam patterns amplify environmental reflections differently.Each beam pattern steers transmission toward a specific direction.Source: §3.2 · pp. 4–5
  • For a given path, its gain depends on the beam pattern and target reflectivity, while its phase depends on the path delay.The path quantity hℓ,p(k) varies with path index and packet time.Source: §3.2 · p. 5

3.3 People localization and tracking

RAPID localizes and tracks people by processing CIR estimates in stages: removing static reflections, estimating distance and angle, and updating trajectories over time. Localization updates can be slower than packet-level CIR readings, while activity and identity recognition use the higher-rate readings.

3.3 People localization and tracking

  • RAPID localizes and tracks people through background subtraction, distance estimation, angular-position estimation, and Kalman-filter processing of distance and angle over time.Localization and tracking updates occur every Δt > Tc, while HAR and identification use CIR readings at rate 1/Tc.Source: §3.3 · p. 5

3.3.1 Background subtraction

  • Static background CIR is estimated by time-averaging amplitudes, then removed to isolate foreground paths; changing beam patterns sweeps the field of view.Static reflections can be stronger than human reflections and may affect localization accuracy.Source: §3.3.1 · p. 5

3.3.2 Distance estimation

  • Candidate distances come from strong local maxima in path strength that exceed a dynamic threshold based on the current CIR’s average and maximum power.The selected paths’ distances are calculated from their delays.Source: §3.3.2 · p. 5

3.3.3 Angular position estimation

  • For each candidate path, RAPID estimates angle by correlating its normalized squared CIR amplitudes across beams with the beam-pattern gains across azimuth.The estimated direction is the one with the highest correlation, since each beam pattern amplifies the path differently.Source: §3.3.3 · pp. 5–6

3.3.4 People tracking - extended Kalman filter

  • An extended Kalman filter tracks each person’s Cartesian position and velocity under a constant-velocity model, associating new observations with prior states using NN-JPDA.The resulting state estimates recover each person’s path and the CIR beam patterns containing their micro-Doppler signature.Source: §3.3.4 · p. 6

3.4 micro-Doppler extraction

RAPID derives micro-Doppler information from the phase evolution of reflected CIR paths, then uses tracking to isolate the relevant beams and paths for each person. The resulting time-frequency signatures capture movement, subject to the system’s packet sampling and Doppler-resolution settings.

3.4.1 CIR phase model

  • A moving reflector’s radial velocity modulates CIR phase across slow-time; velocity is treated as constant during a micro-Doppler processing interval, with motion away from the AP positive.Fast-time indexes CIR samples within an estimate; slow-time indexes samples collected across packets.Source: §3.4.1 · p. 6

3.4.2 micro-Doppler spectrum

  • The STFT power spectrum of each CIR path across slow-time contains phase-modulation information related to reflector velocity, while mmWave micro-Doppler distinguishes moving body parts.The STFT uses a fixed window and Hann weighting; its power is |Hℓ,p(n, i)|2.Source: §3.4.2 · p. 6
  • Tracking restricts micro-Doppler processing to a person’s directionally strongest beam and CIR paths near the estimated range, rather than all paths and beams.A fast-time window centered on the best-matching path is transformed into spectrum components and concatenated across time into the person’s spectrogram.Source: §3.4.2 · pp. 6–7

3.4.4 Human µD range and resolution

  • The Doppler resolution and maximum measurable velocity depend on STFT window length M and packet interval Tc, which must allow capture of the velocities of interest.The stated Doppler-frequency resolution is ΔfD = 1/(MTc), and the maximum measurable Doppler frequency is fDmax = 1/(2Tc).Source: §3.4.4 · p. 7

3.5 Activity recognition and person identification

RAPID classifies each tracked person’s micro-Doppler spectrogram for activity and, during walking, identity. It uses separate CNN classifiers and can combine predictions from multiple access points by selecting the most confident AP.

3.5 Activity recognition and person identification

  • RAPID classifies walking, running, sitting down, waving, and standing still, and identifies people from a known set during walking.The identity classifier is applied only when the HAR classifier labels the activity as walking.Source: §3.5 · p. 7

3.5.2 Deep learning classifier

  • Separate HAR and identity CNNs process sliding windows of each person’s micro-Doppler spectrogram, sharing an architecture but using separate weights.The identity CNN is trained on walking spectrograms; Figure 2 shows four-second examples from four subjects, with color indicating power across Doppler velocity bins.Source: §3.5 · p. 7, Figure 2 · p. 8
  • Before classification, RAPID removes Doppler bins from −0.28 to 0.28 m/s and normalizes each spectrogram column to [0, 1].The removed bins correspond to very low, non-informative velocities; normalization compensates for reflection-strength differences when subjects are far from APs.Source: §3.5 · p. 7

3.5.3 Combining multiple APs

  • When multiple APs detect a person, RAPID uses the classification from the AP assigning the highest probability to its predicted class.If only one AP detects the person, its classification is used.Source: §3.5.3 · p. 8

4 ENABLING SENSING IN IEEE 802.11AY

RAPID uses IEEE 802.11ay beam training to obtain range and angle, then in-packet beam tracking to sample the CIR frequently enough to extract human micro-Doppler signatures.

4 ENABLING SENSING IN IEEE 802.11AY

  • At 1.76 GHz bandwidth, CIR estimates provide approximately 8.5 cm range resolution, and beaconing or beam training about every 100 ms can locate human targets.Range and angle are inferred from CIR multipath components illuminated by different beam patterns; their differing gains support angular estimation.Source: §4 · p. 8
  • The TRN Golay sequences support CIR estimation, while different beam patterns illuminate targets as multipath components whose gains support angle estimation.The 1.76 GHz bandwidth yields approximately 8.5 cm range resolution; a full scan of all beam patterns is not required, since a smaller subset may suffice.Source: §4 · p. 8
  • Beacon or beam-training CIRs sampled every 100 ms yield a maximum measurable Doppler velocity of only 6.25 · 10^-3 m/s, insufficient for micro-Doppler extraction.RAPID addresses this sampling limitation using the standard's beam-tracking procedure on data packets.Source: §4 · p. 8
  • After locating subjects, RAPID appends TRN units to data packets and points their beam patterns toward people, enabling more frequent CIR sampling with low additional packet overhead.Data packets arrive more frequently than beam-training packets, allowing the sampling interval needed for human movement analysis.Source: §4 · p. 8, Figure 4 · p. 9

5 IMPLEMENTATION

RAPID adapts an mmWave SDR platform for IEEE 802.11ay joint communication and radar, adding full-duplex and multi-AP capabilities through FPGA, RF, and timing design choices.

5 IMPLEMENTATION

  • RAPID uses the mm-FLEX SDR platform because available mmWave COTS devices support IEEE 802.11ad with limited PHY access, while IEEE 802.11ay COTS solutions were not available.The platform includes a Kintex Ultrascale FPGA baseband processor, high-speed converters and memory, and a co-located Core i7 processor card.Source: §5 · p. 9, §5.1 · p. 9

5.1 Hardware components

  • The RF front end operates at 60 GHz and uses separate 16-element transmit and receive arrays with freely configurable beam-pattern codebooks.The baseband supports 1.76 GHz bandwidth with 3.52 GSPS sampling; USB, SPI, and GPIO control the system and rapid beam changes.Source: §5.1 · p. 9

5.2 Full-duplex operation

  • Using transmit and receive arrays on the same RF front end shares a local oscillator, keeping carrier-frequency offset within [−40, 40] Hz for phase-coherent CIR estimates.Directional beams and robust TRN Golay sequences avoid the need for complex self-interference cancellation; transmit-power control prevents receiver saturation.Source: §5.2 · p. 9
  • An FPGA state machine reconfigures antennas during TRN fields and coordinates packet transmission and reception; clock-domain crossing and FIFO queues provide deterministic latency.The design accounts for independent transmit/receive clocks and variable DDR-controller latency, and the same procedure can be used at any network station.Source: §5.2 · p. 9

5.3 Multi-AP system

  • The multi-AP setup adds a second baseband processor and independent 60 GHz front end; APs are unsynchronized and use separate channels at 58.32 and 60.48 GHz.The channels can also be configured to share a frequency band by implementing carrier sensing.Source: §5.3 · pp. 9–10

6 EXPERIMENTAL RESULTS

The experimental results evaluate RAPID’s indoor tracking and micro-Doppler-based activity recognition and person identification, including the effects of occlusion and furniture and generalization across settings. They also examine communication overhead and sensing with standard-compliant packets and target-illuminating beams.

6 EXPERIMENTAL RESULTS

  • The experiments used 12 beam-training TRN units over a −45° to 45° field of view and achieved 2° mean angular accuracy.Measurements covered two multipath laboratories; two APs used for multi-AP tests were separated by 1.8 m. Micro-Doppler packets were spaced by 0.27 ms, capturing velocities from −4.62 to 4.62 m/s at 0.14 m/s resolution.Source: §6 · p. 10, §6.1 · p. 10

6.2 Baseline experiments

  • RAPID reconstructed a walking trajectory and its torso and limb micro-Doppler contributions, and localized a sitting-and-standing subject with 26 cm median error.Across around 2,000 position estimates, the probability of error below 40 cm was close to 1; the experiments also showed that IEEE 802.11ay Golay sequences support indoor tracking and micro-Doppler extraction.Source: §6.2 · pp. 10–11

6.3 Multi-person multi-AP tracking scenario

  • Adding AP2 increased subject detection probability by 11%, 16%, 16%, and 11% for 2, 3, 4, and 5 subjects, respectively.For a subject at P4, averaging two APs reduced average localization error from 0.35 m to 0.08 m.Source: Figure 11 · p. 11, §6.3 · p. 12, Figure 15 · p. 13

6.4 Impact of furniture and detection parameters

  • Furniture slightly degraded tracking accuracy, while complete occlusion caused a more noticeable but contained loss as the EKF predicted positions until detection resumed.The paper attributes the limited furniture effect to continued detection of the torso reflection and EKF smoothing; it cautions that nonlinear movement during complete occlusion could worsen performance.Source: §6.4 · p. 13, Table 1 · p. 13

6.5 Human activity recognition

  • RAPID outperformed sub-6 GHz sensing in generalizing to unfamiliar rooms and subjects, and remained effective under interference from another person.The sub-6 GHz system had very low F1-score in the different-room, different-subject scenario; its approximately 4 m ranging resolution also made it more vulnerable to interference.Source: §6.5 · p. 14, Figure 20 · p. 15
  • RAPID’s HAR F1-score decreased only slightly from 2 to 5 concurrent subjects, while combining two APs brought slight improvement in some cases.The evaluation used 34 sequences, and the two-AP system used decision fusion.Source: §6.5 · p. 15, Table 4 · p. 15
  • During furniture occlusion, RAPID misclassified walking when body-part reflections disappeared, then resumed correct classification after the subject became visible.The model had not been trained on occlusion samples; the paper attributes the errors to millimeter-wave signals’ inability to provide movement information when body parts are blocked.Source: §6.5 · p. 15, Figure 19 · p. 15

6.6 Person identification

  • RAPID evaluated gait-based identification using data from 7 subjects and demonstrated simultaneous tracking, activity recognition, and associating activities with the walking person’s identity.The dataset included 3–5 minutes of training data per subject, collected across 10 days; the supplied passage does not report identification accuracy values.Source: §6.6 · p. 15
  • RAPID detects activity changes and identifies S0 and S1 among seven training subjects from spectrograms recorded while they walk.The identification CNN is applied after successful subject tracking.Source: §6.6 · p. 16

6.7 Overhead considerations

  • TRN fields add 0.6·ξ% overhead for a 20 kB MCS 8 packet, and RAPID appends them only to one out of every 3–4 PHY PDUs.With Tc = 0.27 ms and the 1.76 Gsps sample rate, this further reduces overhead; ξ can equal the number of tracked subjects.Source: §6.7 · p. 16
  • RAPID senses reflections of standard-compliant packets, using an arbitrary beam pattern for appended TRN fields to illuminate targets without otherwise interfering with communications.The stated sensing overhead comes from adding TRN fields; channel access for sensing is discussed separately in prior work.Source: §6.7 · p. 16

7 CONCLUDING REMARKS

RAPID uses standard-compliant IEEE 802.11ay packets for indoor multi-person tracking and human micro-Doppler sensing. Two combined access points tracked up to five subjects, with up to 94% HAR and 90% person identification accuracy.

7 CONCLUDING REMARKS

  • Two combined RAPID-APs tracked up to 5 subjects concurrently indoors, with up to 94% HAR and 90% person identification accuracy.Source: §7 · p. 16
  • For HAR, RAPID significantly outperformed standard sub-6 GHz sensing in distinguishing similar activities and generalizing to new environments and unknown subjects.Source: §7 · p. 16
  • RAPID estimates the CIR from standard-compliant 802.11ay TRN fields and extracts human micro-Doppler signatures from phase differences between subsequent packets.These signatures support joint tracking and localization, HAR, and person identification.Source: §7 · p. 16
  • Two full-duplex FPGA-based SDR access points with phased arrays were evaluated in an extensive measurement campaign.The system combines mmWave sensing with the scalability and ease of deployment of existing communication hardware.Source: §7 · p. 16
  • Future work includes combining mmWave with sub-6 GHz radios, whose richer multipath may improve recognition of movements with little radial displacement.The paper gives sitting down as an example.Source: §7 · pp. 16–17
  • Other research directions include bistatic configurations, multi-access-point data fusion, and assessing sensing limits across deployment conditions.These conditions include AP number and location, indoor space, people density, and environmental objects that may cause occlusions or spurious reflections.Source: §7 · p. 17
Loading 2109.04819v4…