Source-linked AI summary
WiFlow: Estimating Optical Flow using WiFi Channel State Information
Thomas Weigel, Simon Kiefhaber, Fabian Portner, Matthias Hollick, Simone Schaub-Meyer
TL;DR
The paper asks whether optical flow can be estimated from WiFi CSI instead of camera frames, which can raise privacy concerns and fail in darkness. WiFlow combines CSI preprocessing, three flow-estimation architectures, and a paired CSI–pseudo-flow dataset, demonstrating feasibility in a fixed scene with generalization to unseen subjects. Its main limitations are environmental scope, imperfect motion supervision, and lower output detail than camera-based models.
Problem
The paper addresses whether optical flow, which captures general motion, can be recovered from CSI when camera frames are undesirable or unreliable.
Method
WiFlow maps preprocessed CSI sequences to 2D optical-flow fields using three architectures and a dataset pairing synchronized CSI with optical-flow pseudo ground truth.
Results
WiFlow demonstrates feasible optical-flow estimation in a fixed indoor scene, with WiFlowCombo achieving the best accuracies and generalizing to subjects absent from training.
Takeaways & Limitations
CSI-based optical flow is a candidate motion-sensing technology for applications where camera systems are unavailable or undesired.
Takeaways & Limitations
The method is limited to the trained environment, uses motion pseudo ground truth with shadow inconsistencies, and produces lower-detail outputs than camera-based models.
Abstract
from arXiv · showhide
Knowing where and how fast objects are moving within a scene is important across various domains. Usually, cameras are used to capture the data necessary for this task, but adding cameras often raises privacy concerns, and the quality of captured frames is heavily influenced by lighting conditions. In this work, we explore using WiFi channel state information (CSI) instead of camera frames for optical flow estimation. We propose WiFlow, a CSI based flow estimator, a preprocessor evaluation for CSI, and three model architectures that offer different trade-offs between accuracy and complexity. Further, we create the first dataset for training and evaluating CSI-based optical flow estimators, and our experiments provide insights into key design elements for this task. Code and data are available at https://visinf.github.io/wiflow.
1 Introduction
WiFlow investigates whether WiFi CSI can recover optical flow without camera frames, targeting motion sensing where cameras raise privacy or lighting concerns. The work demonstrates feasibility in a fixed indoor setting and introduces architectures and a supervised dataset for this task.
- Optical flow captures apparent motion between consecutive frames and supports downstream tasks including tracking, navigation, and action recognition.
- CSI reflects environmental changes caused by moving subjects, making it a potential input for device-free indoor motion sensing.
- WiFlow seeks to recover general motion patterns from CSI directly rather than producing task-specific outputs such as classes, locations, or poses.
- CSI-based flow could support camera-free or privacy-sensitive indoor applications because WiFi works in darkness and uses existing wireless infrastructure.
- The work contributes evidence of generalization across unseen subjects, three accuracy–complexity architectures, and a dataset pairing CSI with optical-flow pseudo ground truth.
2 Related Work
Prior optical-flow research includes classical, convolutional, and transformer-based approaches, while prior WiFi sensing predicts reconstructed content, human-centric outputs, or metric-space motion. WiFlow instead learns dense image-plane flow from CSI, using preprocessing to make raw channel measurements more learnable.
- Optical Flow: Optical flow estimates apparent pixel-wise motion between consecutive image frames using classical or deep-learning methods.
- Optical Flow: Convolutional optical-flow models evolved from FlowNet and coarse-to-fine pipelines toward RAFT-style global correlation with iterative refinement.
- WiFi Sensing: CSI-based prior work reconstructs RGB, depth, or meshes and estimates human-centric outputs such as segmentation, pose, skeletons, and 3D shape.
- WiFi Sensing: Other CSI systems infer physical-space trajectories, walking direction, speed, acceleration, or localization rather than dense image-plane motion.
- Optical Flow: WiFlow predicts dense image-plane motion from CSI alone at inference, while video supplies pseudo-flow supervision during training.
- Preprocessing: CSI preprocessing reduces environmental and hardware artifacts, using alternatives such as Fourier, wavelet, or PCA-based transformations.
3 WiFi Sensing
WiFi channel estimation produces CSI by measuring how the wireless environment transforms transmitted symbols. WiFi sensing repurposes these estimates because motion changes the channel, while the dataset comparison evaluates resources against optical-flow requirements.
- Channel State Information: The transmitted symbol x is transformed by the wireless channel H and noise n, so the receiver measures a distorted observation y.
- Channel State Information: CSI consists of noisy per-packet estimates of the wireless channel across subcarriers, obtained to undo propagation distortion during communication.
- WiFi Sensing: WiFi sensing uses channel estimates already computed for communication as observations of the physical environment.
- WiFi Sensing: Motion induces structured temporal changes in CSI that sensing models associate with properties of the underlying scene.
- Dataset: Table 1 compares existing and proposed datasets using key requirements for optical-flow estimation, including whether accessible RGB frames enable optical-flow pseudo ground truth.
4 WiFlow Dataset
WiFlow introduces a CSI dataset designed to support optical-flow supervision in a fixed, device-free indoor setting. The dataset aligns high-rate CSI with synchronized camera views, includes off-area motion, and provides multiple temporal-context variants and evaluation splits.
- Dataset Requirements: The dataset targets fixed indoor device-free sensing, with static transmitters, receivers, cameras, and walls and subjects carrying no WiFi devices.Multiple receive antennas provide complementary observations because single-antenna CSI can leave motion location and direction ambiguous.
- Dataset Requirements: Existing optical-flow datasets lack CSI, while existing CSI datasets generally lack synchronized video or the multi-device, multi-antenna measurements needed for evaluation.The dataset addresses the need for paired CSI and optical-flow supervision.
- Capture Setup: The capture setup uses one transmitter, four corner-mounted receivers, and two cameras covering sideview and birdview perspectives, while explicitly recording off-area motion.The dataset includes seven predefined actions, one or two moving people, void sequences, and off-area-only motion.
- Capture Setup: CSI is sampled at 1 kHz over 80 MHz using one single-antenna transmitter and four four-antenna receivers, producing high-frequency, high-bandwidth measurements.The setup uses channel 157 and NexmonCSI extraction.
- Synchronizing Training Data: Video is subsampled to 6 Hz for sideview and 10 Hz for birdview, then paired with the K closest preceding CSI samples by timestamp.The resulting variants use K = 10 for sideview and birdview, and K = 100 for birdview+.
- Pseudo Ground Truth: Pseudo ground truth is generated by an ensemble of state-of-the-art optical-flow methods on subsampled camera frames at 168×128 resolution.The aligned data supports training and evaluation with camera-derived flow supervision.
- Pseudo Ground Truth: Sideview motion is mostly horizontal, whereas birdview directions are more uniformly distributed; slower motions are more common in both views.The motion-angle distributions differ substantially between the two perspectives.
- Evaluation Splits: Time and subject splits separately evaluate temporal generalization and cross-subject generalization.The time split retains subjects across training and testing, whereas the subject split separates subjects.
5 WiFlow
WiFlow maps structured CSI sequences to dense image-plane optical flow using RAFT-inspired building blocks. Its three architectures separate or combine motion estimation and localization to trade computational complexity against flow-prediction capability.
- WiFlow: The model maps a structured CSI tensor to a dense 2D optical-flow field representing motion between consecutive camera frames.CSI is stacked across receiver antennas and subcarriers over a temporal snapshot window.
- WiFlow Building Block: All three models use a CSI preprocessor followed by a RAFT-like architecture that predicts either optical flow or a motion mask.The flow and mask variants are called Flow Block and Mask Block.
- WiFlow Building Block: The RAFT-inspired block uses feature, context, and refinement subnetworks, with the refinement network iteratively updating predictions.The feature and context networks are based on modified ResNet architectures.
- Model Architectures: WiFlowSimple directly predicts flow, WiFlowRoI first localizes motion and estimates flow within predicted regions, and WiFlowCombo performs both tasks in parallel.WiFlowRoI uses mask-derived regions and RoIAlign, while WiFlowCombo combines flow- and mask-branch outputs.
- Training Objective: Because static cameras produce mostly zero-flow pixels, the training loss up-weights errors on non-zero-flow pixels to avoid an all-zero prediction.The loss aggregates errors across refinement iterations using a decay factor.
6 Experiments
Experiments evaluate CSI preprocessing, receiver count, architecture accuracy, computational cost, generalization, resolution, and limitations. Quotient preprocessing and WiFlowCombo generally perform best, while performance remains constrained by scene complexity and output detail.
- Evaluation Metrics: EPE can be minimized by predicting zero motion, so the study reports EPEA alongside separate moving- and static-pixel errors.EPEM measures moving pixels, EPES static pixels, and EPEA amplifies all-area errors by 4.
- CSI Preprocessing: Quotient preprocessing achieves the lowest moving-pixel error and EPEA, motivating its use for all subsequent models.Raw achieves the lowest overall EPE but can perform poorly on actual moving pixels.
- Device Count: Adding receiver devices improves all metrics through three devices, with only minor gains beyond three.The experiments compare one to four receivers and also evaluate a two-antenna single-receiver variant.
- Computational Requirements: All models use less than 1GB of VRAM and run inference in the order of milliseconds, while WiFlowCombo requires roughly twice the compute of the other architectures.WiFlowSimple and WiFlowRoI have similar computational requirements.
- Architecture Comparison: WiFlowCombo performs best across most metrics, while WiFlowSimple achieves the best EPEM but produces more static-background noise.The qualitative results indicate that WiFlowRoI and WiFlowCombo localize motion better.
- Generalization and Limitations: Accuracy is similar between time and subject splits, but errors roughly double at doubled resolution and multi-person predictions are less precise.The resolution result indicates that error is dominated by CSI-based prediction capability; multi-person performance is lower than single-person performance.
7 Conclusion
The paper demonstrates optical-flow estimation from CSI in fixed scenes using three architectures and a new supervised dataset. WiFlowCombo provides the best accuracies, with generalization to unseen subjects, while the released dataset supports further CSI motion-estimation research.
- Conclusion: WiFlow estimates optical flow from CSI sequences within a fixed scene and compares three model architectures.The dataset incorporates optical-flow supervision alongside synchronized CSI for training and evaluation.
- Conclusion: WiFlowCombo achieves the best accuracies by using a dedicated motion mask predictor for sharpening.The result is reported as consistent across different perspectives and generalizes to subjects absent from training.
- Conclusion: The dataset is the first to incorporate optical flow as a modality and is publicly released to encourage CSI motion-estimation research.The paper identifies CSI-based optical flow as a candidate for scenarios where camera systems are unavailable or undesired.
A WiFlow Dataset
The WiFlow dataset contains repeated human-action recordings from multiple people, with camera and CSI data organized into complementary evaluation splits. Optical-flow pseudo-ground truth is generated from synchronized, downsampled video using an ensemble of flow methods.
- Dataset Splits: The time split uses 75% of sequences for training, 3% for validation, and 22% for testing.The subject split trains on seven subjects, validates on one, and tests on two.
- Dataset Overview: The dataset records seven actions from up to ten people, plus void and off-area conditions, with camera frames and CSI measurements.The dataset derives sideview, birdview, and birdview+ variants.
- Pseudo Ground Truth: Pseudo ground truths are produced by downsampling captured video and averaging predictions from an ensemble of optical-flow methods.The resulting PseudoGT is generated at 168 × 128 pixels, with thresholding used to reduce background noise.
C Architecture Details
WiFlowRoI and WiFlowCombo separate motion localization from motion estimation through a pretrained mask block.
- Architecture Details: WiFlowRoI and WiFlowCombo use a mask block that predicts moving regions without predicting motion direction or magnitude.The mask block is pretrained with mean squared error against a binary motion target before the remaining modules are trained.
D Qualitative Results
The supplementary materials provide qualitative videos, split-protocol details, and the optical-flow checkpoints used to generate pseudo ground truth.
- Supplemental files include videos showcasing qualitative results for all models across split protocols and actions.
- Table A.3 documents the sequences and subjects assigned to training, validation, and testing under the time and subject splits.
- Table A.4 lists the optical-flow model names and PTLFlow checkpoints used in the ensemble generating pseudo-ground-truth predictions.
E Detailed Results
Detailed evaluations show that WiFlowCombo is generally the most accurate architecture, masking suppresses background errors, and CSI-based flow remains robust under reduced brightness and noise.
- WiFlowCombo is the most accurate model across almost all actions and metrics, followed by WiFlowRoI.
- For the combined together action, model EPE differs from simpler actions such as fast-walk, supporting the findings in Section 6.2 and Figure 10.
- Mask-based WiFlowRoI and WiFlowCombo handle off-area and void actions almost without error, unlike WiFlowSimple.The results attribute this difference to masking strategies eliminating most background noise in this setting.
- EPEM cannot be evaluated for off-area and void because those actions contain no moving pixels, while the zero baseline has EPES of 0 on static pixels.
- 3% noise and reduced brightness lower MS-RAFT accuracy, whereas WiFlowCombo is unaffected because it does not use input images.