Source-linked AI summary
Joint Activity Recognition and Indoor Localization with WiFi Fingerprints
Fei Wang, Jianwei Feng, Yinliang Zhao, Xiaobin Zhang, Shiyuan Zhang, Jinsong Han
TL;DR
The paper targets joint activity recognition and indoor localization from WiFi CSI fingerprints, a task motivated by the distinct but interacting effects of activity and location. It builds a dual-branch 1D convolutional network and evaluates it on a dataset spanning 6 activities and 16 locations, achieving 88.13% activity-recognition accuracy and 95.68% indoor-localization accuracy.
Problem
The paper addresses the limited prior work on jointly recognizing activities and localizing users from WiFi CSI fingerprints.
Method
The authors collect IEEE 802.11n CSI fingerprints and apply a dual-branch 1D convolutional network based on ResNet to predict activity and location in parallel.
Results
88.13% activity-recognition accuracy and 95.68% indoor-localization accuracy are achieved on the evaluated CSI-fingerprint dataset.
Takeaways & Limitations
The results support joint activity recognition and indoor localization with WiFi fingerprints for human-computer interaction applications.
Abstract
from arXiv · showhide
Recent years have witnessed the rapid development in the research topic of WiFi sensing that automatically senses human with commercial WiFi devices. This work falls into two major categories, i.e., the activity recognition and the indoor localization. The former work utilizes WiFi devices to recognize human daily activities such as smoking, walking, and dancing. The latter one, indoor localization, can be used for indoor navigation, location-based services, and through-wall surveillance. The key rationale behind this type of work is that people behaviors can influence the WiFi signal propagation and introduce specific patterns into WiFi signals, called WiFi fingerprints, which can be further explored to identify human activities and locations. In this paper, we propose a novel deep learning framework for joint activity recognition and indoor localization task using WiFi Channel State Information~(CSI) fingerprints. More precisely, we develop a system running standard IEEE 802.11n WiFi protocol, and collect more than 1400 CSI fingerprints on 6 activities at 16 indoor locations. Then we propose a dual-task convolutional neural network with 1-dimensional convolutional layers for the joint task of activity recognition and indoor localization. Experimental results and ablation study show that our approach achieves good performances in this joint WiFi sensing task. Data and code have been made publicly available at https://github.com/geekfeiw/apl.
I. INTRODUCTION
The paper addresses the underexplored joint task of recognizing activities and localizing users from WiFi CSI fingerprints. It proposes a two-branch 1D convolutional network and evaluates it on a purpose-built IEEE 802.11n dataset.
- Motivation: Joint activity recognition and indoor localization remain relatively underexplored despite substantial prior work on each task separately.The paper motivates joint sensing through location-dependent responses to identical gesture commands in smart-home applications.
- Challenges: CSI fingerprints vary across locations for the same activity and across activities at the same location, creating a need for shared activity representations and location-distinguishing features.These two sources of variation are identified as the major challenges of the joint task.
- Problem formulation: The joint task learns from WiFi fingerprint W(A, L) to predict both activity category A and user location L simultaneously.The formulation captures the two factors represented in each fingerprint.
- Approach: The proposed C1D network uses two branches, one for activity recognition and one for indoor localization, and processes temporal WiFi fingerprints.The network is based on ResNet and applies 1D convolutions to temporal data.
- Evaluation: The authors implement IEEE 802.11n with two USRP sets and collect 1394 valid samples from 6 hand gestures repeated across 16 indoor locations.The dataset uses one volunteer and excludes invalid data from the collected samples.
- Evaluation: The work reports a dataset and evaluation specifically designed for the joint activity-recognition and localization task.The proposed deep network is evaluated and its results are analyzed on this dataset.
II. RELATED WORK
Prior CSI-fingerprint research covers activity recognition, localization, and other sensing applications, but commonly treats classification through handcrafted features or temporal matching. The paper’s related-work context also distinguishes conventional 2D CNNs from models suited to temporal data.
- CSI sensing applications: CSI fingerprints have been used for activity recognition, gesture recognition, indoor localization, and health-care applications.Reported examples include falling detection, keystroke inference, typing recognition, and hand-sign recognition.
- CSI classification methods: Handcrafted CSI statistics combined with SVM require expertise in feature design, making the approach harder to apply to joint activity recognition and localization.Examples of statistics include the mean, maximum, minimum, and entropy of CSI time series.
- CSI classification methods: DTW combined with kNN classifies CSI fingerprints through temporal matching against samples in a training dataset.The passage presents this as another popular approach to CSI-fingerprint classification.
- Neural-network approaches: Conventional C2D networks are designed for 2D inputs such as images, while C3D networks process spatial and temporal dimensions in video data.These architectures use convolutional kernels across the dimensions of their respective inputs.
- System hardware: The hardware context includes USRP components such as a motherboard, daughterboard, and WiFi antenna, with sync and Ethernet cables shown in the system materials.The USRPs broadcast or receive WiFi signals under GNU Radio control.
III. DATA COLLECTION
The study collects CSI fingerprints for six smart-home gesture commands performed by one volunteer at 16 indoor locations, using a standardized USRP-based IEEE 802.11n setup. The preprocessing pipeline annotates action duration, splits the CSI samples, and upsamples the resulting series for analysis.
- Data collection: The experiment uses two Ettus N210 USRPs to broadcast WiFi signals and parse CSI fingerprints under the IEEE 802.11n protocol.The hardware is controlled through GNU Radio, with synchronization cables used to avoid phase shifting from clock differences.
- Activities: Six gestures—hand up, hand down, hand left, hand right, hand circle, and hand cross—represent smart-home human-computer interaction commands.The gestures correspond to volume control, channel switching, confirmation, and cancellation in the illustrated television example.
- Preprocessing: Signal preprocessing manually annotates each action duration, splits CSI samples during the action, and upsamples each CSI series to length 192.The figure illustrates the procedure using the 29th subcarrier for a circle action at position #2.
- Locations: One volunteer performs the six activities at 16 evenly selected locations in one room to cover most of the central area.The setup fixes the USRPs beside the selected locations and combines activity performance with location variation.
C. CSI FINGERPRINT ANALYSIS
CSI fingerprints vary across activities even at one location and also vary across repetitions or locations for the same activity. These variations make shared activity features and location-discriminative features difficult to identify jointly.
- Activity variation: CSI fingerprints for six activities at location #10 have different profiles across the 52 sub-carriers.The visualization uses sampling index as time, CSI amplitude as the y-axis, and 52 colors for the OFDM sub-carriers.
- Repetition variation: Repeated circle actions at location #3 produce different time-series profiles and different activity start points.The caption notes that late action starts can result in only partial CSI capture.
- Location variation: Circle actions performed at locations #6, #9, and #15 show substantial CSI variation across locations.This variation makes it challenging to find shared features for one activity across all locations.
IV. METHODOLOGY
The methodology represents CSI fingerprints as multichannel time series and applies a ResNet1D-based dual-branch network to predict activity and location in parallel.
- CSI representation: CSI fingerprints are modeled as time series with 52 sub-carriers and a sampled-time dimension.The fingerprint is represented as C ∈R52×t, where t denotes sampled time.
- 1D convolution: C1D sweeps along the time axis while treating the 52 OFDM channels as the input channels, capturing temporal values and shapes.This design differs from applying 2D convolutions to CSI fingerprints viewed as images.
- ResNet1D: The paper re-implements ResNet for temporal CSI fingerprints as ResNet1D.The basic residual block combines a convolutional branch with a shortcut branch.
- Residual block: The residual implementation uses two 3 × 1 C1Ds in the main branch and a 1 × 1 C1D in the shortcut branch, with batch normalization.ReLU follows the first main-branch convolution, and the shortcut projection addresses branch-size differences.
- Network framework: ResNet1D takes CSI time series as input and predicts user activity and location in parallel through shared and task-specific C1D layers.The described framework contains 11 C1D layers, including 9 shared layers and one independent layer for each sub-task.
- Network framework: The framework is expandable by cascading residual blocks, with evaluations comparing ResNet1D-, ResNet1D-, and ResNet1D-.The latter two residual-block settings are identified as defaults in the referenced ResNet.
D. LOSS FUNCTION
The network optimizes a combined loss for activity recognition and indoor localization, balancing the two task losses with λ.
- Joint objective: The total loss is the sum of activity-recognition and indoor-localization losses, weighted by λ.The formulation is L = L_activity + λL_location.
- Score normalization: Before loss computation, prediction scores are normalized with the SoftMax function.The normalized scores are converted to the 0-1 range.
- Task outputs: The activity and localization tasks use 6 and 16 categories, respectively.These category counts determine the corresponding loss outputs.
- Task losses: Cross Entropy Loss is applied to the normalized activity scores, and the localization loss is computed analogously.The paper sets λ = 1, treating activity recognition and indoor localization as equally important in the experiment.
E. IMPLEMENTATION
The ResNet1D implementation uses PyTorch and trains for 200 epochs with Adam, mini-batches of 128, and a decaying learning rate.
- Software and hardware: The network is implemented with PyTorch 1.0.0 on Windows 7 using one Nvidia Titan Xp GPU.Training uses the Adam optimizer with default β1 = 0.9 and β2 = 0.999 settings.
- Training configuration: Training runs for 200 epochs with mini-batch size 128 and initial learning rate 0.005.The learning rate decays by 0.5 every 10 epochs, and training data are shuffled before each epoch.
V. EVALUATION
Evaluation uses a 1,394-sample dataset spanning six activities and 16 locations, with learning curves tracking both tasks and showing faster convergence and better performance for localization.
- Dataset and split: 1,394 valid samples remain after discarding low-quality recordings from an original 1,440 samples.The dataset contains six activities repeated 15 times at each of 16 locations.
- Dataset and split: One out of every five samples forms a 278-sample test set, selected evenly across the collected data.The reported test condition is the same as the training condition because the test samples are evenly selected.
- Learning curves: Figure 12 reports loss and accuracy learning curves for activity recognition and indoor localization.The evaluation tracks training and test behavior for both subtasks.
- Learning curves: Training activity-recognition loss reaches a relatively low state around epoch 50, while test loss steadies around epoch 75.The test loss initially swings substantially during the first 45 epochs.
- Learning curves: Shuffling training data before each epoch is attributed to continued test-loss decreases after training loss becomes steady.Different mini-batch combinations continue updating the network after the training loss stabilizes.
- Task comparison: Indoor localization converges faster and achieves better performance than activity recognition in the reported learning-curve comparison.Activity-recognition accuracy steadies around epoch 100, while the localization curves are reported as faster-converging.
C. QUANTITATIVE RESULTS
ResNet1D- performs both activity recognition and indoor localization, with stronger localization accuracy and measurable spatial-error performance. Errors concentrate on particular gestures and locations.
- 88.13% accuracy for activity recognition and 95.68% for indoor localization are achieved by ResNet1D-.The reported confusion matrices show higher accuracy for indoor localization than activity recognition.
- 8% of hand cross samples are misclassified as hand left, while 6% are misclassified as hand circle.
- 15% of #16 location samples are wrongly predicted as #4 location.
- 0.97 precision and 0.77 recall for hand circle produce an F1 score of 0.82.The precision–recall gap indicates that hand circle recognition is uneven across the evaluated samples.
- 0.81 is the lowest F1 score for #4 location prediction, attributed to low recall.
- 0.0904m average localization error is reported, while the average mis-classified error is 2.0943m across incorrectly predicted samples.The average mis-classified error corresponds to two to three experimental locations.
D. DATA VISUALIZATION
t-SNE visualizations trace how ResNet1D transforms CSI inputs into increasingly discriminative representations for activity recognition and indoor localization. The two tasks separate at different network stages, motivating an activity-specific extension.
- D. DATA VISUALIZATION: CSI fingerprints are reshaped from C ∈R52×192 into C′ ∈R1×9984 vectors for t-SNE visualization.The original 52-subcarrier time series is cut and interpolated to t = 192 before reshaping.
- D. DATA VISUALIZATION: ResNet1D feature maps are visualized after pooling, each residual block, before fully connected layers, and after fully connected outputs.
- D. DATA VISUALIZATION: Activity-recognition features become discriminative after the fully connected layer, whereas indoor-localization features become discriminative beginning at RB4.
- D. DATA VISUALIZATION: The activity branch enhances features beyond the shared RB4 representation, motivating an added C1D3 × 1, 512 layer in ResNet1D-[1,1,1,1]+.
- D. DATA VISUALIZATION: ResNet1D-[1,1,1,1]+ outperforms ResNet1D- on activity recognition.
E. EXPANSIBLE STUDY AND BASELINES
The expansible ResNet1D family performs well across the joint task, but depth affects the two tasks differently. All proposed networks outperform DTW+kNN and SVM-RBF, with a runtime trade-off against SVM-RBF.
- Expansible study: ResNet1D- and ResNet1D- extend the architecture by customizing residual-block counts.
- Expansible study: Deeper ResNet1D networks tend to perform better on indoor localization but worse on activity recognition.
- Baselines: All proposed ResNet1D models outperform the DTW+kNN and SVM-RBF baselines.
- Baselines: SVM-RBF has the lowest time cost but the worst performance, while DTW+kNN is time-consuming despite being a strong time-series classification baseline.
- Conclusion: The proposed network achieves 88.13% activity-recognition accuracy and 95.68% indoor-localization accuracy, with higher computational cost than SVM-RBF.The conclusion states that larger-batch training can reduce computational time.