FloodDiffusion 2: Efficient and Path Controllable Streaming Motion Generation

1 INTRODUCTION

• FD1’s changing active window makes finalized history representations change during denoising, preventing standard KV caching and requiring repeated computation.FD1’s text-based control also cannot precisely specify a walking direction, while applications require specified routes across different actions.Source: §1 · p. 2
• Partial Attention makes history representations independent of subsequent frames, enabling KV-cached inference and shared-history training.History queries read causal prefixes, while active queries read complete history and interact bidirectionally within the active window.Source: §1 · p. 2
• A necessary-and-sufficient Bregman criterion identifies losses that preserve diffusion’s conditional-mean velocity field and guides an FK-induced quadratic for geometric supervision.The quadratic uses a dataset-averaged kinematic response and avoids online FK evaluation.Source: §1 · pp. 2–3
• Root-path conditioning jointly specifies a frame-aligned root trajectory and time-varying text prompts, separating commanded movement from body articulation.The representation reduces gait-cycle sway while preserving natural body motion.Source: §1 · pp. 2–3
• FD2 improves efficiency and motion quality, achieving state-of-the-art streaming-method FID scores on SEED (0.048) and HumanML3D (0.053).HumanML3D FID improves from FD1’s 0.057 to 0.053.Source: §1 · p. 2

Motion generation and streaming

• Prior motion-generation approaches include continuous diffusion, autoregression, retrieval augmentation, masked token modeling, and incremental methods such as MotionStreamer and ARDY.FD1 uses frame-staggered diffusion built on Diffusion Forcing, and FD2 retains FD1’s schedule and sampler.Source: §2 · p. 3

Efficient streaming diffusion

• Transformer-XL reuses history states, while Rolling Forcing combines causal history with a bidirectional active window for KV caching.Rolling Forcing and Self Forcing train on self-generated histories; FD2 instead uses direct velocity supervision and shares a clean reference encoding across denoising windows.Source: §2 · p. 3

Geometry-aware supervision

• Geometric losses supplementing feature regression motivate FK-based supervision, but nonlinear FK losses can shift diffusion’s conditional-mean velocity target.FD2 applies mean-elicitation theory to design an FK-induced quadratic that preserves the target without online FK evaluation.Source: §2 · p. 3

Spatially controlled motion generation

• Spatial-control methods impose joint or trajectory constraints; FD2 holds three relative root channels clean while diffusing explicit body features.Its SEED root is a mass-weighted center-of-mass ground projection, with separate pelvis motion preserving root-relative articulation.Source: §2 · p. 3

3.1 PRELIMINARIES

• FD2 retains FD1’s frame-staggered schedule, where denoising ramps separate finalized history, the active window, and future noise as overlapping windows advance frame by frame.The processed prefix is sufficient under FD1’s causal-conditioning assumption.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2 PARTIAL ATTENTION

• Partial Attention makes history representations independent of later frames while active queries retain access to the complete processed prefix.This structure supports KV-cached inference and shared-history training; finalized frames are re-encoded causally before their keys and values are appended.Source: §3.2 · p. 4, Figure 3 · p. 5

3.3 DIFFUSION-COMPATIBLE LOSSES

• Under stated regularity and integrability conditions, target-first Bregman divergences with strictly convex potentials uniquely recover the conditional-mean velocity.The FK-induced quadratic uses averaged, normalized kinematic response matrices plus an identity term, yielding a fixed positive-definite metric without online FK evaluations.Source: §3.3 · pp. 5–6, Figure 4 · p. 6
• Direct nonlinear FK supervision need not recover the conditional-mean velocity because averaging geometric outputs generally differs from decoding the mean state.The proposed fixed quadratic retains geometric sensitivity through a linear response in standardized coordinates.Source: §3.3 · p. 5

3.4 ROOT-PATH CONDITIONING

• FD2-Path conditions generation on a frame-aligned planar root trajectory and text, while keeping the supplied root channels fixed during sampling.Its center-of-mass root has 20.3% lower planar jerk and 9.6% lower curvature than a pelvis projection; relative pelvis motion preserves body articulation around the commanded path.Source: §3.4 · pp. 6–7

4.1 EXPERIMENTAL SETUP

• Evaluation follows HumanML3D’s standard split and evaluator with FD1 preprocessing and duration handling (Guo et al., 2022a), while SEED uses the public Kimodo training partition, category-held-out Content test split, and v1.1 evaluator.Source: §4.1 · p. 7

4.2 ATTENTION AND EFFICIENCY

• Partial Attention retains comparable FID to Full attention (0.058 vs. 0.057) and improves R@3 and MM-Dist, while fully causal attention gives FID 0.096.The ablation uses the same HumanML3D configuration, checkpoint budget, and K = 1; diversity closer to 9.503 is better.Source: §4.2 · p. 7, Table 1 · p. 7
• At 512 windows per step, K = 8 packing reduces training denoiser FLOPs by 4.6× and GPU count from four to one; one GPU supports 1,024 windows.The experiments use 300-frame windows and 16 text tokens.Source: §4.2 · p. 7, Table 2 · p. 7
• With 4,500 history frames, KV caching reduces backbone latency from 26.002 to 2.303 ms per update (11.29×) and computation by 137×.The reported measurements use an RTX 4090 and include the complete backbone update, causal re-encoding, and K/V writes for the newly finalized frame.Source: §4.2 · p. 7, Table 3 · p. 8

4.3 MOTION QUALITY AND PATH CONTROL

• On HumanML3D, FK lowers FD2 text-only FID from 0.058 (FD1: 0.057) to 0.053, best among compared streaming text-only methods; FD2-Path leads path-conditioned methods in R@3, FID, MM-Dist, and 3D pelvis error.Source: §4.3 · p. 8
• On SEED, FD2 has the lowest streaming text-only FID, while its R@3 remains below ARDY; FK supervision also reduces FD2-Path tracking error from 0.18 to 0.11 cm.FK supervision improves all three text-only metrics and reduces FD2-Path foot skate from 4.07 to 3.24 cm/s.Source: §4.3 · p. 8, Table 5 · p. 9

4.4 QUALITATIVE CONTROL

• Figures 5 and 6 show action and gait changes under text and path conditioning, including speed-profile following while the action prompt remains unchanged.Figure 5 depicts prompt switching along a commanded path; Figure 6 depicts walking and running under varying commanded speed profiles.Source: §4.4 · p. 9, Figure 5 · p. 9, Figure 6 · p. 10

5 LIMITATIONS AND CONCLUSIONS

• FD2 improves motion quality, efficiency, and controllability on HumanML3D and SEED through Partial Attention, an FK-induced quadratic, and root-path conditioning.The FK-induced quadratic is guided by a necessary-and-sufficient Bregman criterion.Source: §5 · p. 10
• Finalized errors persist in history, generation beyond the training horizon requires extrapolation, and cache memory and per-update attention cost grow linearly with history length.The fixed, dataset-averaged FK quadratic leaves room for more adaptive mean-consistent geometric weighting; rare actions and unfamiliar combinations are limited by training-data coverage.Source: §5 · p. 10

A PARTIAL ATTENTION

• The Partial Attention appendix examines streaming locality and active-prediction information, then develops conditions for reusing finalized-history keys and values and for shared-history packing.It also considers the equivalence between shared-history packing and isolated training windows.Source: A PARTIAL ATTENTION · p. 13

A.1 COMPLETE ACTIVE INFORMATION AND PROBABILITY FLOW

• Under causal dependency and the stated schedule, the velocity field vanishes outside the active window, whose predictions depend only on the processed prefix.Fully causal predictions within the active window can nevertheless be strictly worse than predictions using the complete active information.Source: §A.1 · p. 13

A.2 HISTORY-PREFIX INVARIANCE

• Partial Attention makes each finalized-history state and its keys and values invariant to later suffixes, allowing cache reuse as new rows are encoded.The result also holds for inclusive causal execution; changes to prompts affecting only later rows preserve the finalized-history cache.Source: §A.2 · p. 14

A.3 MULTI-WINDOW PACKING

• For identical inputs and untruncated history, packed windows produce exactly the same predictions as isolated executions, without reading one another.The equivalence also preserves the token-count-weighted objective and requires shared reference perturbations and noise metadata when used during training.Source: §A.3 · pp. 14–15

B DIFFUSION-COMPATIBLE REGRESSION LOSSES

• The section develops a necessary-and-sufficient Bregman criterion, an FK-induced quadratic, two motion-state implementations, and matrix diagnostics with a controlled rotation experiment.It also explains why direct nonlinear geometric losses need not preserve the conditional-mean target.Source: B DIFFUSION-COMPATIBLE REGRESSION LOSSES · p. 15

B.1 CONDITIONAL TARGETS TRAIN THE POSTERIOR AVERAGE

• Strictly mean-consistent training uniquely recovers the conditional-mean active velocity and, across diffusion times, the streaming velocity field.Under the stated sampling formulation, its zero extension reproduces the conditional data law.Source: §B.1 · p. 15

B.2 A NECESSARY AND SUFFICIENT CHARACTERIZATION

• A differentiable loss is mean-consistent for every finitely supported law exactly when it is a Bregman divergence plus a target-only term; strict consistency requires a strictly convex potential.The target must be the first loss argument and the prediction the second.Source: §B.2 · pp. 16–17

B.3 DIRECT NONLINEAR LOSSES NEED NOT BE MEAN-CONSISTENT

• A direct squared loss after a nonlinear geometric map need not be mean-consistent, and adding it with any positive coefficient to a mean-consistent quadratic can break consistency.The expected direct-loss gradient at the target mean need not vanish.Source: §B.3 · p. 17

B.4 FK-INDUCED QUADRATIC OBJECTIVE

• The FK-induced quadratic is strictly mean-consistent, so its unique population optimum preserves the diffusion probability path and terminal distribution.Its deterministic positive-definite weight is built from a dataset-averaged response Gram matrix, including kinematic joint responses or decoder Jacobians.Source: §B.4 · p. 18
• Predictor-input-dependent weights preserve the conditional mean, whereas target-dependent weights generally change the minimizer; offline averaging avoids FK and Jacobian evaluations during training.The unique minimizer for target-dependent weights is (E[W(Y)])^-1E[W(Y)Y], which generally differs from the mean.Source: §B.4 · p. 18

B.5 RESPONSE OPERATORS FOR THE TWO MOTION STATES

• The 138-D surface response uses a neutral-SMPL-H Jacobian and fixed, area-weighted vertex displacements; FD2-Path restricts the response to the 135-D articulated block.Area weighting approximates mean-squared surface displacement, and the fixed rest-template areas and vertex correspondences set its scale.Source: §B.5 · pp. 18–19
• The 263-D joint response maps features to 22 joint centers, propagates induced bone swings through subtrees, and removes three conditioned root channels in the path-conditioned variant.Velocity, rotation, and contact channels have zero response; the identity term restores positive definiteness, and the path-conditioned matrix is renormalized with d = 260.Source: §B.5 · pp. 19–20

B.6 SPECTRAL STRUCTURE OF THE FK MATRICES

• The 138-D surface matrix has a 55.43% off-diagonal ratio, and the FD2 and FD2-Path matrices have condition numbers 23.8292 and 23.0458, respectively.The surface matrix’s ten largest eigenvalues carry 90% of its trace; fixing the root path gives the 135-D matrix full numerical rank.Source: §B.6 · p. 20, Table 6 · p. 20

B.7 EFFECT OF FK SUPERVISION ON THE SAMPLING DISTRIBUTION

• Figure 7 shows trace-normalized 263-D kinematic-response and 138-D surface-pullback matrices, descending eigenvalue spectra, and a 67-channel active block.Omitted channels in the 263-D heatmap have zero weight; each heatmap uses its own signed color scale.Source: Figure 7 · p. 21
• 30.15% (MSE) and 29.98% (FK-induced quadratic) match the 30.00% target, while pure nonlinear FK assigns 99.99% to the densest region.The controlled 6-D rotation experiment used 8,192 equally weighted rotations across eight regions and two training seeds.Source: §B.7 · pp. 20–22, Figure 8 · p. 22

C ROOT-PATH CONDITIONING AND MOTION REPRESENTATION

• FD2-Path conditions a frame-aligned planar root position and heading through three clean relative root channels while diffusing the remaining body state.The pelvis retains relative translation and rotation so articulation can vary around the commanded path.Source: C ROOT-PATH CONDITIONING AND MOTION… · p. 22
• HumanML3D-to-SMPL conversion requires approximate retargeting because joint positions do not uniquely determine bone twist and the root stream stores heading, not full pelvis orientation.For SEED, the 138-D state stores pelvis and body rotations directly, but uses neutral SMPL-H with fixed mean hand pose and β = 0, excluding actor-specific shape and variable hand articulation.Source: C ROOT-PATH CONDITIONING AND MOTION… · p. 22
• The representation diagnostics use HumanML3D source motions for root smoothness and rotation recovery, whereas the main-text HumanML3D benchmark uses native 263-D features.Source: C ROOT-PATH CONDITIONING AND MOTION… · p. 22

C.1 ROOT–PELVIS SEPARATION

• The root carries ground position and heading, while the pelvis stores vertical motion, local displacement, and residual 3D orientation relative to it.This separates the user-facing trajectory from gait-cycle pelvis sway; Figure 9 locates anatomical landmarks and the mass-weighted center of mass whose planar projection defines the ground root.Source: §C.1 · p. 22, Figure 9 · p. 23

C.2 MASS-WEIGHTED GROUND-PROJECTED ROOT PIVOT

• 20.3% lower planar jerk and 9.6% lower curvature: the mass-weighted center-of-mass projection is smoother than the pelvis projection.The pivot weights SMPL-H joints using segment masses and center locations, with the stated landmark mapping and unit-sum normalization.Source: §C.2 · p. 23, Table 8 · p. 24

C.3 HEADING FROM HIPS AND SHOULDERS

• Heading is estimated from hip and shoulder lines rather than pelvis orientation, and relative motion features are invariant to global yaw on the stated nondegenerate domain.The heading uses both landmark pairs without temporal smoothing; an epsilon guard stabilizes normalization.Source: §C.3 · p. 24

C.4 138-D LAYOUT AND RECONSTRUCTION

• The 138-D representation stores heading-relative root quantities and parent-local body rotations, which can reconstruct the original positions and rotations from the same initial pose.Decoding uses SMPL-H pose corrective blend shapes and linear blend skinning to produce surface vertices.Source: §C.4 · pp. 24–25

C.5 PATH ENCODING

• FD2-Path encodes each frame’s planar position and heading as three root channels, keeps those channels fixed during sampling, and generates the remaining body channels.The encoded root channels are standardized using the fixed training statistics before concatenation with noisy body channels.Source: §C.5 · p. 25

C.6 SOURCE-ROTATION FIDELITY

• The source-rotation round trip evaluates whether encoding and decoding preserve HumanML3D source rotations, with two motions omitted from the 263-D result after SVD reconstruction failures.The reported geodesic error is frame-weighted across 26,846 source motions.Source: §C.6 · p. 25, Table 9 · p. 25

D IMPLEMENTATION AND EVALUATION DETAILS

• The implementation and evaluation section collects training settings and protocols for denoiser-computation timing, motion quality, path tracking, and foot skate.The timing and FLOP measurements describe denoiser computation.Source: D IMPLEMENTATION AND EVALUATION DETAILS · p. 25

D.1 TRAINING CONFIGURATION

• FD2 uses a 114M-parameter transformer and frozen UMT5-XXL text features, with separate 138-D and native 263-D motion representations across SEED and HumanML3D.The full model uses γ = 1, while the ablation without FK uses γ = 0.Source: §D.1 · p. 25, Table 10 · p. 26

D.2 TRAINING-EFFICIENCY MEASUREMENTS

• Training-efficiency measurements compare K = 1 baselines with packing under specified GPU counts and window batches, and report peak memory summed across GPUs.The measurements use NVIDIA H200 GPUs; packing uses per-GPU batch sizes listed in the table rather than the default training batch.Source: §D.2 · p. 26

D.3 KV-CACHE INFERENCE MEASUREMENTS

• KV-cache latency is measured for uncached Full attention and cached Partial Attention on an RTX 4090, processing 30 active frames with up to 4,500 history frames.Reported latency is the median group-mean steady-state backbone execution time; startup, text encoding, decoding, and rendering are excluded. FLOP counts include projections, transformer blocks, output projection, and the newly finalized frame, but exclude elementwise operations and memory transfers.Source: §D.3 · p. 26

D.4 HUMANML3D EVALUATION

• HumanML3D generation models are evaluated on the standard split with the released text-to-motion evaluator and FD1 preprocessing and duration handling.Source: §D.4 · p. 26
• Table 4 reports means and 95% confidence intervals from repeated evaluations.Source: §D.4 · p. 26
• HumanML3D generation uses native 263-D features, while 138-D round-trip and root-smoothness diagnostics separately evaluate representation properties.Source: §D.4 · p. 26

D.5 PATH-CONTROL EVALUATION CONVENTIONS

• FD2-Path Position measures distance from the generated mass-weighted center-of-mass projection to the commanded root trajectory in decoded neutral SMPL-H coordinates.For 917 cases, mean and P95 use 183,130 conditioned frames.Source: §D.5 · pp. 26–27
• On HumanML3D, GMD and DART target planar pelvis positions, OmniControl targets 3D pelvis positions, and FD2-Path conditions on planar root position and heading.Source: §D.5 · p. 27
• Root and 3D report pooled-frame control-point and pelvis errors against ground truth, while Skate is the per-sequence fraction of frames with either foot skating.Foot contact and skate are determined using height, instantaneous-speed, and five-frame-average thresholds at 20 fps.Source: §D.5 · p. 27

• The evaluation uses the public Kimodo training partition and category-held-out Content test split of SEED; path-conditioned methods receive dense paths for the same 917 cases in Table 5.Source: §D.6 · p. 27
• For R@3 and FID, decoded motions are converted to SOMA77 joint positions and passed through the official TMR-SOMA-RP-v1 embedding and aggregation pipeline.Source: §D.6 · p. 27
• SEED Skate reports mean 3D foot speed over contact frames in cm/s, using shared 0.10 m height and 0.15 m/s speed thresholds for generated motions.Scores are averaged over sequences; sequences without contact frames receive zero, while parenthesized Kimodo and ARDY scores use their predicted contact labels.Source: §D.6 · p. 27