PaperlayerView exact revision on arXiv

Source-linked AI summary

KOALA: Koopman Operator Learning for WiFi-Based Anticipatory Hum

Quang-Anh N. D., Duc Pham Minh, Thao Phuong Pham, Minh Anh Nguyen, Huan X. Nguyen, Tuan Dang

arXiv:2608.15815v1cs.LG

TL;DR

Future human motion prediction from WiFi CSI is challenging because existing CSI methods focus on instantaneous pose estimation, while applying autoregressive vision predictors can amplify noise. KOALA lifts CSI-derived poses into a learned Koopman space for non-autoregressive multi-horizon prediction, and experiments on MM-Fi and WiPose show consistent improvements over baselines across short- and long-term horizons.

  • Problem

    Existing WiFi-CSI approaches focus on instantaneous pose estimation rather than modeling temporal dynamics for future motion prediction.

  • Method

    KOALA lifts CSI-derived pose sequences into a learned Koopman latent space and uses a residual CSI-conditioned operator with anchor-delta decoding for multi-horizon prediction.

  • Results

    KOALA consistently outperforms existing baselines across short- and long-term prediction horizons on MM-Fi and WiPose.

  • Takeaways & Limitations

    KOALA provides a unified framework for anticipatory human motion prediction directly from WiFi CSI without autoregressive iteration.

  • Takeaways & Limitations

    The Koopman invariance condition is learned only approximately from finite data, with the residual invariance defect reported empirically rather than guaranteed architecturally.

Abstract

from arXiv · show

WiFi Channel State Information (CSI) has emerged as a privacy-preserving alternative to cameras for human pose estimation. However, existing approaches treat pose inference as an instantaneous regression problem and do not model temporal dynamics, making future motion prediction infeasible. Naively applying vision-based prediction methods compounds the estimation noise already present in CSI-derived poses, as autoregressive rollouts amplify errors at every step. We propose KOALA, the framework for human motion prediction directly from WiFi CSI, by lifting noisy CSI-derived pose sequences into a learned Koopman latent space where nonlinear dynamics become linear, enabling multi-horizon prediction via simple matrix-vector products without autoregressive iteration or error accumulation. A residual CSI-conditioned operator resolves the identity attractor problem inherent from Koopman formulations, and an anchor-delta prediction head eliminates the degenerate shortcut of copying the current pose across all horizons. To regularise the lifting and operator jointly, we introduce a Koopman Anchored Latent (KAL) loss that operates in the temporal-encoder feature space, enforcing dynamical consistency across prediction horizons without requiring contrastive, spectral, or auxiliary losses. Experiments on MM-Fi and WiPose show that KOALA achieves robust, consistent performance across both short- and long-term prediction horizons, outperforming all baselines by a substantial margin.

1 Introduction

KOALA introduces WiFi-CSI-based multi-horizon human motion prediction by combining noisy CSI-derived poses with learned Koopman dynamics. Its residual operator, anchor-delta decoding, and KAL loss target key failure modes while experiments show consistent gains across horizons.

  • Motivation and framework: KOALA is presented as a unified framework for predicting human motion directly from WiFi CSI rather than only estimating instantaneous poses.It targets a privacy-preserving sensing modality that avoids line-of-sight requirements and visual access.
  • Motivation and framework: CSI and observed skeleton poses are lifted into a Koopman latent space, where one CSI-conditioned linear operator propagates multiple future horizons without autoregressive iteration.This uses matrix-vector propagation instead of repeatedly feeding predictions back into the model.
  • Design contributions: The residual CSI-conditioned operator addresses the identity-attractor problem, while anchor-delta decoding prevents copying the current pose across all horizons.These components are designed to avoid uninformative dynamics and degenerate predictions.
  • Design contributions: The Koopman Anchored Latent loss regularizes lifting and dynamics in temporal-encoder feature space without separate contrastive, spectral, or auxiliary losses.It combines reconstruction, linearity, and stability objectives into one anchored prediction term.
  • Evaluation: KOALA consistently outperforms existing baselines on MM-Fi and WiPose across short- and long-term prediction horizons.The introduction reports this as the central experimental finding without specifying a single headline metric.

2 Related Work

Prior work developed camera- and CSI-based pose estimation alongside diverse skeleton-motion predictors, but these lines of research leave noisy WiFi-CSI motion forecasting largely unaddressed. KOALA is positioned as the first method to tackle that setting.

  • WiFi-based pose estimation: WiFi pose estimation evolved from camera-supervised 2D joint recovery to 3D reconstruction and richer spatiotemporal architectures.Examples include residual CNN-recurrent models, Transformer encoders, and multi-person extensions.
  • Skeleton-based motion prediction: Skeleton-based motion prediction progressed from recurrent models to feed-forward, frequency-domain, graph-convolutional, Transformer, and MLP approaches.These methods addressed error accumulation, mean-pose collapse, frequency modeling, spatial structure, or representation strength in different ways.
  • Research gap: Existing motion-prediction methods operate on clean skeleton observations, whereas future-pose prediction from noisy WiFi CSI remains a harder and previously unaddressed setting.The distinction is between forecasting from already observed skeleton sequences and forecasting through a noisy sensing modality.
  • Research gap: KOALA is described as the first framework to address human motion prediction from WiFi CSI.This connects CSI-based pose estimation with anticipatory motion forecasting.

3 Methodology

KOALA processes WiFi CSI into structured pose and motion features, then lifts them into a Koopman latent space for non-autoregressive multi-horizon prediction. Its residual CSI-conditioned dynamics, anchor-delta decoding, and KAL loss address identity-attractor, degenerate-copying, and noisy-lifting challenges.

  • CSI Representation: CSI records time-varying amplitude and phase changes across subcarriers and antenna pairs as body segments move.These measurements form a CSI tensor over time, subcarriers, and antenna pairs.
  • Koopman Dynamics: Koopman linearization propagates lifted states across horizons with matrix powers, while KOALA conditions a residual operator on global CSI context.The residual form embeds identity while allowing a learned residual matrix and low-rank CSI adaptation factors to model dynamics.
  • Problem Formulation: KOALA estimates future 3D poses from CSI alone across multiple horizons while addressing noisy mapping, temporal consistency, and cascaded pose-estimation errors.The prediction horizons are H = {1, 3, 5, 10, 15, 20} frames.
  • Architecture: The pipeline encodes CSI, estimates and skeleton-encodes poses, fuses CSI and pose streams, and lifts contextual motion features into a Koopman latent space.A temporal encoder produces contextual features before lifting; estimated poses are detached before skeleton encoding.
  • Prediction Head: Anchor-delta decoding adds predicted pose changes to an anchor pose, preventing current-pose copying from producing low loss during slow motion.At long horizons, zero deltas incur loss when the true pose drifts from the anchor, forcing the operator to encode temporal dynamics.
  • Loss Design: The KAL loss regularizes lifting and operator learning in temporal-encoder feature space, combining reconstruction, linearity, and stability roles without several auxiliary losses.The joint invariance condition is approximate in finite-data training, with its residual defect ε reported empirically rather than guaranteed architecturally.

4 Experiments

KOALA is evaluated on MM-Fi and WiPose across diverse protocols, settings, and horizons. It consistently outperforms existing baselines, with moderate degradation over longer horizons.

  • MM-Fi includes 40 subjects, 27 actions, and 4 environments evaluated under three protocols and three split settings.
  • 26.14 mm MPJPE at 100 ms and 27.28 mm at 1000 ms on WiPose represent approximately 19% improvement over VMRNN.KOALA achieves the lowest MPJPE and PA-MPJPE at every reported horizon.
  • On MM-Fi, PA-MPJPE remains below 65.3 mm and PCK@20 above 82.3% across all configurations.Protocol 3 maintains PCK@20 above 85.4% at 1000 ms despite covering all 27 action categories.
  • 52.1 mm MPJPE at 100 ms is roughly 6.4× lower than SwinLSTM, while KOALA reaches 73.9% PCK@10 and no baseline exceeds 18.1%.KOALA outperforms nine baselines across every metric and horizon on MM-Fi.

5 Conclusion

The paper presents KOALA for anticipatory human motion prediction directly from WiFi CSI. Its Koopman formulation supports multi-horizon forecasts with substantial improvements over baselines, while important deployment boundaries remain.

  • KOALA lifts noisy CSI-derived pose sequences into a Koopman latent space for multi-horizon prediction without autoregressive iteration.
  • Residual parametrization, anchor-delta decoding, and KAL loss address identity attraction, mean-pose collapse, and dynamical consistency respectively.
  • Experiments on MM-Fi and WiPose show substantial improvements over all baselines, with MPJPE growing moderately across horizons.
  • The current formulation is single-person, lacks zero-shot cross-site validation, and produces one deterministic prediction per horizon.These boundaries limit multi-occupant coverage, transfer claims, and representation of future-motion distributions.
  • The approach has potential applications in healthcare monitoring and intelligent environments while offering a more privacy-conscious alternative to vision-based sensing.
  • Misuse, inadequate consent, and surveillance remain concerns broadly shared with wireless and machine-learning sensing systems.

A Proof of Remark 1

The proof shows that low reconstruction error makes the lifting approximately injective on the training manifold. Features separated beyond the reconstruction-error threshold cannot collapse to the same latent representation.

  • Triangle inequality bounds the two round-trip reconstruction terms by √ϵr when Lrec ≤ ϵr.
  • Feature pairs separated by more than 2√ϵr must have distinct lifted representations.
  • Therefore, the lifting is approximately injective on the training manifold, with guaranteed separation above that threshold.

B Proof of Theorem 1

The proof decomposes future pose error into Koopman invariance, lifting, decoder, and anchor-alignment terms. It also clarifies that the stability condition is empirical and that decoder error is not controlled by Koopman dynamics.

  • Latent error decomposition: The latent prediction error after h transitions is decomposed into accumulated Koopman invariance and lifting approximation components.
  • Pose-space mapping: The anchor-based delta head maps latent predictions into pose space, while an ideal decoder comparison isolates pose reconstruction behavior.
  • Decoder contribution: Decoder approximation error remains an explicit term because the learned MLP need not exactly reconstruct future poses even from ideal latent inputs.
  • Error propagation: The anchor term cancels in the latent comparison, and triangle-inequality steps yield the latent-to-pose propagation term Lδh.
  • Stability condition: κ ≤ 1 + η is an empirical property of the trained operator, not a hard architectural guarantee.KOALA monitors ∥B∥F and relies on KAL loss and residual initialization to keep κ near 1.

C Proof of Proposition 1

The proof shows that removing the KAL loss permits encoder collapse and unconstrained operator growth, whereas the anchored target prevents collapse and bounds the operator norm.

  • Degenerate joint minimizer: Without KAL reconstruction regularization, encoder and decoder can co-adapt to approximate the identity for any encoder scale ε.The estimation loss does not directly constrain either lifting map.
  • Prediction invariance: Prediction remains invariant to ε, so the objective provides no gradient opposing ε →0 and leaves ∥B∥F unconstrained.The proof identifies this as the source of the collapse direction.
  • Gradient bound: As ∥B∥F grows, the upper bound on the rollout gradient increases with operator powers, while no remaining objective term constrains ∥B∥F.The resulting rollout magnitudes can grow geometrically across the 20-step operator chain.
  • KAL prevents collapse: When ε →0, bounded decoding and operator dynamics produce nearly horizon-independent outputs, but the anchored KAL targets vary with subject motion velocity.This creates a nonzero KAL loss determined by minimum inter-frame feature variation.
  • Implementation: The training setup uses an observation window of T = 10 frames and evaluates horizons from 100 to 2000 ms, with a maximum horizon of 20 frames.The model uses d = 128, Koopman latent dimension Dz = 256, and low-rank rank r = 16.

F Qualitative Results

Qualitative sequences show that KOALA preserves accurate short-horizon motion and coherent skeleton structure through +2000 ms, despite increasing articulation complexity and minor distal-joint deviations.

  • Figure 3: KOALA predictions closely align with ground truth during gradual arm raising from +100 ms to +500 ms.Agreement holds for both global body configuration and individual joint positions.
  • Figure 3: At +1000 ms to +2000 ms, minor wrist and ankle deviations appear, but the overall body structure remains anatomically plausible without structural collapse.The qualitative result attributes this to coherent latent dynamics throughout the rollout.
  • Figure 4: The more dynamic Figure 4 sequence is a substantially harder target because of wider limb displacement and greater inter-frame variability.The figure compares predicted poses with ground truth across different time steps.
  • Figure 4: Despite greater articulation complexity, KOALA tracks the global pose trajectory at short horizons and maintains structurally coherent skeletons through +2000 ms.Predictions preserve joint connectivity and limb proportions without catastrophic drift or physically implausible configurations.

G KOALA Efficiency Metrics

KOALA’s reported computational footprint is compatible with real-time WiFi sensing, and 8-bit quantization reduces the model size to 5 MB for feasible edge deployment.

  • Real-time operation: KOALA operates within real-time constraints for WiFi sensing at CSI sampling rates of 100–200 Hz.The efficiency measurements use batch size 1, observation window T = 10, and six prediction horizons.
  • Edge deployment: 8-bit quantization reduces the model size to 5 MB, making edge deployment feasible.The paper presents this reduction as part of its computational-footprint analysis.

H Per-Joint Error Analysis

Error analysis shows larger errors for distal joints, substantial long-horizon latent divergence despite empirical performance, and clear dependence on residual operators, anchor-delta prediction, fusion, pretraining, and latent capacity.

  • Per-joint errors: KOALA has the lowest MM-Fi errors on pelvis and hip joints, while elbows and wrists are harder because they undergo larger motion amplitudes.WiPose shows a similar pattern, with most torso and lower-body joints near 20–30 mm and wrists remaining more difficult.
  • Koopman invariance: The reported empirical Koopman invariance defect compares lifted ground-truth future features with operator rollouts across prediction horizons.Both absolute defect and defect relative to target norm are reported.
  • Koopman invariance: 7444.4× at h=20 is the reported relative defect, so the authors do not claim approximate Koopman invariance at long horizons.The defect grows from 0.68× at h=1 and is described as divergence relative to the target scale.
  • Architecture ablations: Removing the residual operator increases MPJPE by 3–6 mm across horizons, while removing anchor-delta prediction yields a flat 64.6–67.7 mm error profile.The latter behavior is consistent with near-constant pose outputs rather than modeled motion.
  • Architecture ablations: Removing dual-stream fusion worsens MPJPE from 52.7 mm at 100 ms to 92.8 mm at 1000 ms, showing the value of CSI dynamics beyond skeletal coordinates.Removing HPE pretraining causes approximately 106 mm error throughout.
  • Oracle comparison: The oracle HPE comparison narrows the MM-Fi gap from 26.8 mm at 100 ms to 7.1 mm at 1500 ms, while the WiPose gap reaches 5.2 mm MPJPE and 1.7 mm PA-MPJPE.These results quantify performance relative to replacing HPE estimates with ground-truth poses.
  • Hyperparameter sensitivity: Reducing model dimension to d = 64 increases MPJPE by 2–2.5 mm, whereas Dz = 64 reaches 74.6 mm at 1000 ms versus 65.7 mm for Dz = 256.The study identifies d = 128 as a parameter-efficiency trade-off and latent dimension as the strongest factor.

J.4 Loss Ablations.

The ablations show that direct pose supervision is necessary for meaningful latent dynamics, while latent consistency alone produces non-degenerate but poorly calibrated predictions. Across predictor formulations, KOALA maintains lower and more stable error than the alternatives.

  • Loss ablations: Removing Lest yields a flat MPJPE profile of 132.75-132.92 mm across horizons, indicating that the model lacks meaningful pose-space dynamics without direct pose supervision.The accompanying analysis attributes this to uninformative HPE estimates and near-static latent propagation.
  • Loss ablations: Removing Lpred produces horizon-varying MPJPE from 141.09 mm at 100 ms to 168.06 mm at 1000 ms, showing that LKAL alone induces non-degenerate dynamics.However, PCK@10 remains approximately 13%, versus 65-74% for the full model, so feature-space consistency does not sufficiently calibrate pose-space predictions.
  • Zero-velocity baseline: The zero-velocity baseline worsens from 56.3 mm at 100 ms to 91.9 mm at 1000 ms, whereas KOALA spans only 52.1-61.9 mm over the same horizons.This comparison uses the same CSI-estimated anchor pose and no learned delta.
  • Predictor formulations: KOALA outperforms DMD, LRU, and Transformer predictors at every horizon when all operate on the same fused CSI-skeleton features.DMD degrades sharply with horizon, while LRU is the strongest baseline but still trails KOALA by 2-8 mm.
Loading 2608.15815v1…