PaperlayerView exact revision on arXiv

Source-linked AI summary

DiFF: Doppler-informed Flow Matching for Human Motion Flow

Kai Wang, Mingle Zhao

arXiv:2609.39098v1cs.ROcs.CVcs.HC

TL;DR

Sparse, noisy radar point clouds make non-rigid human motion-flow estimation severely ill-posed, and prior work has not used Doppler cues available in 4D radar. DiFF combines Doppler-informed initialization with KAN-based conditional flow matching; on mmBody it reports 2.1 mm EPE3D, an order-of-magnitude improvement over milliFlow.

  • Problem

    Sparse, noisy radar point clouds and non-rigid human articulation make point correspondences difficult, while prior work could not use Doppler velocity unavailable on the Vayyar radar in milliFlow.

    Source: §I · p. 1

  • Method

    DiFF combines a Doppler-informed initial motion prior with KAN-based conditional flow matching and a KAN-attention feature extractor.

    Source: §I · pp. 1–2, B. Framework Overview · p. 3

  • Results

    2.1 mm EPE3D on mmBody, an order-of-magnitude improvement over milliFlow; 63.36% of samples fall within the 0–1 mm range.

    Source: B. Comparison with State-of-the-Art… · p. 6

  • Takeaways & Limitations

    DiFF achieves state-of-the-art human motion scene-flow performance on mmBody, with ablations confirming the significance of the Doppler prior, KAN architecture, and dynamic correlation mechanism.

    Source: §V · p. 7

    The paper identifies replacing the linear flow path with learned nonlinear transport and extending to multi-person scenarios with occlusion handling as future directions.

    Source: §V · p. 7

Abstract

from arXiv · show

Perceiving human motion via privacy-preserving 4D millimeter-wave (mmWave) radar is critical for next-generation human-robot interaction (HRI), where point cloud scene flow serves as a foundational motion representation. Yet the extreme sparsity and noise of 4D radar point clouds make non-rigid motion flow estimation severely ill-posed--a challenge that existing rigid-centric methods and prior works fail to adequately address, largely because they neglect the rich Doppler velocity cues inherent in 4D radar. We propose DiFF, a generative framework that marries Doppler-informed motion priors with a Kolmogorov-Arnold Network (KAN)-based conditional flow matching model. At its core, a KAN-attention mechanism enables expressive feature extraction, while a prior-guided generative process harnesses Doppler cues to regularize the ill-posed solution space. Extensive experiments show that DiFF achieves state-of-the-art (SOTA) performance across diverse real-world datasets, reducing 3D endpoint error to the millimeter scale on the mmBody benchmark.

I. INTRODUCTION

Sparse, noisy radar point clouds make non-rigid human motion flow difficult to estimate, while prior scene-flow work largely assumes rigid or piecewise-rigid objects. DiFF combines Doppler-informed motion priors with KAN-based flow matching to address this challenge.

I. INTRODUCTION

  • Sparse, noisy radar returns and complex human articulation make reliable point correspondences difficult, leaving human motion-flow estimation severely ill-posed.Prior scene-flow approaches often assume rigid or piecewise-rigid objects, an assumption that breaks down for articulated human motion.Source: §I · p. 1, A. Scene Flow in Rigid and Deformable… · p. 2
  • DiFF combines a Doppler-informed motion prior with KAN-conditioned flow matching to estimate high-fidelity human motion flow.Its KAN-attention feature extractor uses learnable rational polynomial functions and global attention to capture geometric and temporal correlations in sparse point clouds.Source: §I · p. 1
  • The proposed Doppler prior estimates an initial radial flow that guides flow matching, while dynamic conditioning is reported to improve training stability.The architecture is also described as capturing local geometry and global context while improving inference efficiency.Source: §I · p. 2

A. Scene Flow in Rigid and Deformable Scenes

  • Scene-flow methods developed for autonomous driving include cost-volume, correlation, and diffusion approaches, alongside self-supervised methods that often use local rigidity.These methods commonly address rigid or piecewise-rigid scenes rather than highly articulated human motion.Source: A. Scene Flow in Rigid and Deformable… · p. 2

B. Motion Sensing and Estimation via Vision and Radar

Vision methods can capture human deformation, but camera privacy and environmental constraints motivate non-invasive sensing. Radar-based human sensing also uses richer signal features, including Doppler, beyond sparse point-cloud geometry.

B. Motion Sensing and Estimation via Vision and Radar

  • Camera-based methods can capture human-body deformation, but their privacy risks and sensitivity to poor lighting, smoke, or fog constrain use in homes, hospitals, and care facilities.mmWave radar is presented as preserving anonymity by capturing sparse points and operating through darkness.Source: B. Motion Sensing and Estimation via… · p. 2

C. Radar-based Human Motion Sensing and Doppler Prior

  • Radar human-sensing methods use features beyond point-cloud geometry, including Range-Doppler heatmaps and micro-Doppler signatures for activity recognition.Radar-based mesh reconstruction work is also cited as recovering dynamic, high-fidelity human meshes from radar signals.Source: C. Radar-based Human Motion Sensing… · p. 2
  • Prior radar work on human sensing motivates using Doppler information in a generative network for human motion flow.The paper states that its experiments validate the advantageous role of Doppler information from 4D radar/LiDAR in estimating human motion flow.Source: C. Radar-based Human Motion Sensing… · p. 2

A. Problem Formulation

The task is to estimate forward motion for source radar points across consecutive scans, without requiring persistent target returns at their displaced locations. DiFF encodes both scans, uses an intermediate flow to form conditioning features, and trains a flow field along a path from Doppler-informed initialization to ground truth.

A. Problem Formulation

  • The source and target inputs are consecutive radar point clouds, and each source return may carry intensity, amplitude, and signed Doppler.The flow estimate is defined over source points rather than requiring persistent detections across frames.Source: A. Problem Formulation · p. 2
  • The forward ground-truth flow assigns each source point a displacement to its target-time location, even when no target radar return exists there.The formulation estimates flow on source points from the two point sets and their available attributes.Source: A. Problem Formulation · p. 2

B. Framework Overview

  • The overview depicts source and target point-cloud encoding, state-dependent conditioning from warped points, and KAN decoding of the predicted velocity field.During inference, the condition computed at t = 0 is cached while the ODE updates St.Source: Figure 2 · p. 3
  • DiFF uses Doppler to initialize radial motion, then state-dependent conditioning to guide recovery of unobserved tangential components.Its architecture combines a KAN-attention geometric encoder with conditional flow matching.Source: B. Framework Overview · p. 3
  • A weight-shared encoder processes both point clouds, refining local neighborhoods with three KAN blocks and adding frame context through self-attention.The encoder preserves input point cardinality.Source: B. Framework Overview · p. 3
  • At sampled time t, intermediate flow St warps source points; cross-frame neighbor aggregation around them forms condition ct for a KAN decoder.Dynamic conditioning uses St, whereas static conditioning uses the initial state S0.Source: B. Framework Overview · p. 3
  • The training target is the derivative of a straight path from Doppler-informed initialization S0 to ground-truth flow Sgt.At inference, the model integrates the learned velocity field from t = 0 to t = 1.Source: B. Framework Overview · p. 3

C. Network Architecture

DiFF pairs KAN-based local feature refinement with global attention to represent sparse, articulated human radar returns. Its feature extractor preserves full point-cloud resolution while combining fine geometric detail and whole-frame context.

C. Network Architecture

1) KAN-Attention Feature Extractor:

  • The single-scale extractor maintains full point-cloud resolution to avoid losing critical information, unlike hierarchical methods.Shared-weight convolutional extractors first produce coarse spatial features for the two input clouds; KAN blocks then refine local geometry.Source: 1) KAN-Attention Feature Extractor · p. 3
  • KAN learns nonlinear functions on edges using adaptive local bases, unlike an MLP with fixed node activations.The paper links this design to sparse radar neighborhoods where fine-scale geometry, amplitude changes, and Doppler cues may be smoothed by a standard MLP.Source: 1) KAN-Attention Feature Extractor · p. 4

2) Flow Matching Backbone:

The flow-matching backbone conditions velocity prediction on cross-frame point-cloud features and the current flow state. DiFF initializes its trajectory with a Doppler-informed prior and uses a straight-line path toward the target flow.

2) Flow Matching Backbone:

  • The backbone dynamically warps target points using the evolving flow state, builds a cross-frame cost volume, and conditions velocity decoding on that signal.The decoder combines projected flow, time, and conditional context; at inference, the condition is computed once at t = 0 and cached.Source: 2) Flow Matching Backbone · p. 4

D. Doppler-informed Flow Matching

  • Doppler-informed initialization provides an inductive bias that the paper says accelerates training and improves final scene-flow accuracy.The trajectory starts from a motion-informed distribution rather than standard Gaussian noise; inference caches the initial condition while the flow state and time evolve through the ODE solver.Source: D. Doppler-informed Flow Matching · p. 4
  • The flow path is a straight-line interpolation between the Doppler-informed start and the target ground-truth flow.The starting point is defined from Doppler velocity; the paper identifies C as a frame-rate constant and α as a scaling constant, typically set to 1.Source: D. Doppler-informed Flow Matching · p. 4
  • Fig. 3 examines how the number of inference steps affects EPE3D and average inference time.The supplied caption identifies these quantities but does not state the direction or magnitude of their changes.Source: Figure 3 · p. 5

A. Experimental Setup

Experiments use milliFlow and mmBody, with different point-cloud density and sensor features. The mmBody evaluation uses non-occluded sequences and holds out test subjects from training.

A. Experimental Setup

  • Experiments use milliFlow and mmBody, which differ in point-cloud sparsity and available radar features.milliFlow has sparser, noisier clouds with intensity only; mmBody provides higher-quality multimodal radar points, including Doppler velocity and amplitude.Source: 1) Implementation Details · p. 5
  • The mmBody experiments use only non-occluded sequences and a 4:1 train-test split with test subjects unseen during training.Training uses sequences 0–5 and 7–16, while testing uses sequences 6 and 17–19; a subset is used to evaluate generalization.Source: 1) Implementation Details · p. 5
  • For mmBody, points are filtered near the SMPL-X skeleton, assigned body-part labels, and given flow ground truth from per-part transformations.Training pairs are sampled or padded to 512 points; inference estimates flow directly on variable-sized point clouds.Source: 1) Implementation Details · p. 5

2) Training and Inference Details:

The training setup constructs radial priors differently for milliFlow and mmBody, while conditioning may be recomputed during training or cached at inference. The model is trained for 200 epochs and inference uses 10 ODE steps.

2) Training and Inference Details:

  • For milliFlow, which lacks Doppler measurements, radial priors are formed by projecting available flow annotations onto radial directions.This setting analyzes the backbone on sparser radar clouds; the passage says a deployable no-Doppler setting could instead use learned or uninformed initialization.Source: 2) Training and Inference Details · p. 5
  • mmBody priors preserve measured radial motion while allowing exploration of unobserved tangential components.At inference, the radial component is computed from measured Doppler velocity and frame interval, with the perpendicular random component set to zero.Source: 2) Training and Inference Details · p. 5
  • Training uses dynamic conditioning recomputed from the intermediate flow state, while inference computes and caches the condition once at t = 0.The decoder still receives the evolving state and time during inference; the model uses 64-dimensional encoder features, three KAN blocks, and a two-layer KAN decoder, and inference uses 10 ODE steps.Source: 2) Training and Inference Details · p. 5
  • The model is trained end-to-end for 200 epochs on one NVIDIA RTX 4080 GPU with batch size 16 and AdamW.Training uses an initial learning rate of 1 × 10−3 with cosine annealing.Source: 2) Training and Inference Details · p. 5

B. Comparison with State-of-the-Art Methods

DiFF is compared with radar- and LiDAR-based methods on milliFlow and mmBody. It reports millimeter-scale mmBody error and stronger robustness across both datasets, attributing its performance to Doppler priors and an efficient KAN-attention architecture.

B. Comparison with State-of-the-Art Methods

C. Ablation Experiment

Ablations examine DiFF's Doppler prior, KAN components, and dynamic conditioning, while a variance study finds an EPE3D optimum that does not coincide with the lowest training loss.

a) Effect of Core Components

  • Replacing Doppler-informed initialization with Gaussian noise causes a sharp performance drop, supporting the prior's strong inductive bias.Source: C. Ablation Experiment · p. 7
  • Replacing KAN components with standard MLPs degrades performance, indicating KAN's adaptive activations better capture complex geometry in sparse point clouds.Source: C. Ablation Experiment · p. 7
  • Dynamic conditioning provides more robust guidance during ODE solving than static conditioning.Source: C. Ablation Experiment · p. 7

b) Effect of Initial Noise Variance

  • EPE3D is optimal at σ^2 = 0.012; larger variance adds noise, while smaller variance limits correction of prior errors.The best-EPE3D setting does not produce the lowest training loss.Source: C. Ablation Experiment · p. 7, Table IV · p. 7

V. CONCLUSION

DiFF combines Doppler velocity priors and a KAN-based feature extractor in a flow-matching framework for sparse mmWave radar scene flow. It achieves state-of-the-art performance on mmBody, while the conclusion identifies nonlinear transport and multi-person occlusion handling as future directions.

V. CONCLUSION

  • DiFF achieves state-of-the-art performance on the mmBody dataset for human motion scene flow estimation from sparse mmWave radar point clouds.The conclusion also reports that ablations confirm the significance of the Doppler prior, KAN architecture, and dynamic correlation mechanism.Source: §V · p. 7
  • DiFF integrates Doppler velocity priors into flow matching and uses a KAN-based feature extractor.Ablations confirm the significance of the Doppler prior, KAN architecture, and dynamic correlation mechanism.Source: §V · p. 7
  • Future directions include replacing the linear flow path with learned nonlinear transport and extending the framework to multi-person scenarios with occlusion handling.The nonlinear transport is proposed to better exploit the anisotropic Doppler prior.Source: §V · p. 7

Related papers

Loading 2609.39098v1…