AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation

Yuchao Gu; Guian Fang; Yuxin Jiang; Weijia Mao; Song Han; Han Cai; Mike Zheng Shou

Source-linked AI summary

AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation

Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, Mike Zheng Shou

arXiv:2605.13724v1May 13, 2026cs.CVcs.AI

TL;DR

Existing consistency-distilled video models often lose performance as test-time sampling steps increase, limiting flexible any-step generation. AnyFlow distills arbitrary-time flow-map transitions with on-policy backward simulation, matching or surpassing consistency-based counterparts in few-step settings while improving with larger sampling budgets.

Problem
Existing consistency-based video distillation methods often degrade as sampling steps increase, limiting flexible any-step generation across inference budgets.
Method
AnyFlow learns arbitrary-time flow-map transitions and uses flow map backward simulation for efficient on-policy distillation over the full ODE trajectory.
Results
Across bidirectional and causal architectures from 1.3B to 14B parameters, AnyFlow matches or outperforms consistency-based counterparts in few-step regimes and improves as sampling budgets increase.
Takeaways & Limitations
AnyFlow offers a practical, scalable path to high-quality video generation with a single model supporting a wide range of sampling budgets.
Takeaways & Limitations
AnyFlow relies on external datasets for flow-map training, which can introduce mild distribution shift from the base model’s training distribution.

Abstract

from arXiv · show

Few-step video generation has been significantly advanced by consistency distillation. However, the performance of consistency-distilled models often degrades as more sampling steps are allocated at test time, limiting their effectiveness for any-step video diffusion. This limitation arises because consistency distillation replaces the original probability-flow ODE trajectory with a consistency-sampling trajectory, weakening the desirable test-time scaling behavior of ODE sampling. To address this limitation, we introduce AnyFlow, the first any-step video diffusion distillation framework based on flow maps. Instead of distilling a model for only a few fixed sampling steps, AnyFlow optimizes the full ODE sampling trajectory. To this end, we shift the distillation target from endpoint consistency mapping $(z_{t}\rightarrow z_{0})$ to flow-map transition learning $(z_{t}\rightarrow z_{r})$ over arbitrary time intervals. We further propose Flow Map Backward Simulation, which decomposes a full Euler rollout into shortcut flow-map transitions, enabling efficient on-policy distillation that reduces test-time errors (i.e., discretization error in few-step sampling and exposure bias in causal generation). Extensive experiments across both bidirectional and causal architectures, at scales ranging from 1.3B to 14B parameters, demonstrate that AnyFlow achieves performance matches or surpasses consistency-based counterparts in the few-step regime, while scaling with sampling step budgets.

1. Introduction

AnyFlow is introduced to enable video diffusion models whose quality can improve with flexible test-time sampling budgets, addressing the degradation of consistency-distilled methods as sampling steps increase. It learns arbitrary-time flow-map transitions and uses on-policy backward simulation to reduce discretization error and exposure bias.

Motivation: Any-step video models are motivated by the need to trade latency for quality at test time without retraining.They target rapid previews and higher-fidelity final outputs under different inference-step budgets.
Limitation: Consistency-based distillation methods can degrade as sampling steps increase, limiting robust any-step behavior.The introduction identifies rCM for bidirectional models and Self-Forcing for causal models as representative examples.
AnyFlow: AnyFlow learns two-time flow-map transitions z_t→z_r rather than only endpoint mappings z_t→z_0, supporting variable step sizes and inference budgets.This formulation generalizes endpoint consistency by learning transitions between arbitrary time pairs.
Method: Flow Map Backward Simulation decomposes Euler trajectories into shortcut transitions for efficient on-policy distillation that mitigates discretization error and causal exposure bias.AnyFlow first trains forward flow maps from pretrained video diffusion models, then optimizes reverse divergence on model rollouts.
Results: 84.05 at 4 NFEs and 84.41 at 32 NFEs show AnyFlow-FAR’s text-to-video scaling on the 14B causal model, surpassing Krea-Realtime-14B (83.25 at 4 NFEs).AnyFlow also reaches 87.87 VBench-I2V at 4 NFEs and 84.04 at 4 NFEs in 14B bidirectional text-to-video, outperforming rCM-14B (83.73 at 4 NFEs).

2. Related Work

Prior work accelerates diffusion through consistency mappings, while flow map models generalize transitions across arbitrary PF-ODE time pairs. Video distillation commonly combines forward initialization with on-policy refinement, and AnyFlow extends this paradigm to scalable any-step generation for bidirectional and causal models.

Consistency Models: Consistency Models learn z_t→z_0 mappings along teacher PF-ODE trajectories and use repeated noise injection and iterative denoising for multi-step generation.Annealed time schedules, segmented consistency objectives, and sCM improve training stability or simplify design choices.
Flow Map Models: Flow Map Models learn generalized transitions f_θ(z_t, t, r) between arbitrary PF-ODE points, recovering consistency modeling at r=0 and flow matching at t=r.Training remains challenging because transitions must be learned accurately across arbitrary time pairs; later work addresses JVP computation and FSDP scaling.
Video Diffusion Distillation: Video diffusion distillation typically uses forward training for initialization followed by on-policy distillation to reduce rollout drift during few-step inference.Examples include rCM, which combines sCM initialization with consistency backward simulation and distribution matching, and Self-Forcing for causal generation.
Video Diffusion Distillation: AnyFlow differs from TMD by improving simulation efficiency through flow-trajectory decomposition into shortcut transitions, rather than backbone sharing with an additional flow head.The comparison concerns two distinct ways of achieving efficient rollout in flow-map-based bidirectional video diffusion distillation.
Causal Video Diffusion: Causal video diffusion faces exposure bias and autoregressive error accumulation; AnyFlow retains on-policy distillation while introducing flow map backward simulation for any-step generation.Self-Forcing targets few-step rollout efficiency with a consistency formulation, whereas AnyFlow is designed specifically for the flow map formulation.

3. Preliminary

This section introduces flow maps as continuous-time ODE transition operators and explains their role in variable-step inference. It also presents MeanFlow and on-policy distillation as training foundations for reducing test-time errors.

Flow Map Formalism: The probability-flow ODE defines latent states z_t over continuous time, with velocity field v governing their evolution.The time variable satisfies t∈[0, 1].
Flow Map Formalism: The exact flow map Φ_r←t transports z_t to z_r and obeys identity and composition properties for ordered times.Specifically, Φ_t←t(z)=z and Φ_q←r∘Φ_r←t=Φ_q←t for t≥r≥q.
Flow Map Models: Neural flow maps learn transitions between arbitrary time pairs rather than only endpoints, supporting variable step sizes and any-step inference.The model uses the boundary condition f_θ(z_t,t,t)=z_t.
MeanFlow Objective: MeanFlow parameterizes averaged transport velocity on [r,t] and approximates transitions as f_θ(z_t,t,r)=z_t−(t−r)u_θ(z_t,r,t).Its training objective optimizes the parameterized averaged velocity u_θ.
On-policy Distillation: On-policy distillation trains on student self-rollout states guided by a strong teacher, using backward simulation and distribution matching to mitigate exposure bias and discretization error.DMD instantiates this paradigm through reverse-KL training and re-noising of self-generated samples.

4. Method

AnyFlow enables any-step video generation by learning flow-map transitions between arbitrary time pairs and refining them through on-policy distillation with backward simulation. This design addresses the test-time degradation and rollout errors of consistency-based sampling while supporting varied inference budgets and downstream continued training.

Motivation: AnyFlow replaces endpoint consistency mapping with flow-map transition learning between arbitrary time pairs, preserving the full ODE sampling trajectory.The model learns f_θ: (z_t, t, r) ↦ z_r rather than only predicting z_0.
Overview: AnyFlow trains in two stages: forward flow-map training on teacher-synthesized data, followed by joint forward training and DMD-based on-policy distillation.Stage 1 provides a stable transition operator and initialization; Stage 2 corrects rollout mismatch.
Forward Flow Map Training: Interpolated timestep conditioning keeps embedding norms aligned with the pretrained model and suppresses over-saturated results caused by zero-initialized conditioning.The interpolation uses fixed g = 0.25 and reduces to the pretrained timestep embedding when t = r.
On-Policy Distillation: Flow Map Backward Simulation shortcuts long trajectories by composing transitions between arbitrary time pairs instead of explicitly simulating every intermediate step.This design targets discretization error in few-step sampling and exposure bias in causal generation.
On-Policy Distillation: After distillation, AnyFlow uses a standard Euler scheduler and naturally supports different inference budgets because it trains transitions over varied time intervals.The method avoids the costly full-trajectory rollout required by consistency backward simulation.
Continued Training on Downstream Datasets: AnyFlow preserves the pretrained model’s instantaneous flow field, enabling continued downstream training that reduces identity and trajectory errors in specialized domains.This compatibility contrasts with Self-Forcing, whose distilled model is difficult to train further.

5. Experiments

Experiments show that AnyFlow performs strongly in few-step video generation while improving with larger sampling budgets across bidirectional and causal T2V settings. It also achieves efficient I2V generation, favorable qualitative results, and reduced training cost at larger step counts.

Quantitative Comparison: AnyFlow-Wan2.1-T2V-14B scores 84.04 at 4 NFEs, surpassing rCM-Wan2.1-T2V-14B at 83.73.On the causal backbone, AnyFlow-FAR-Wan2.1-14B reaches 84.05 at 4 NFEs and 84.41 at 32 NFEs.
Quantitative Comparison: AnyFlow-FAR-Wan2.1-14B achieves a VBench-I2V score of 87.87 with 4 NFEs, exceeding FastVideo-CausalWan2.2-A14B-Preview at 86.82.Its score is comparable to Wan2.1-I2V-14B using 50×2 NFEs, which scores 87.71.
Qualitative Comparison: At 14B scale, AnyFlow-FAR-Wan2.1-14B delivers the best causal T2V visual quality and motion with 4 NFEs, avoiding baselines’ blur, flicker, and unrealistic motion.For I2V, it maintains good first-frame faithfulness and smooth motion transitions, with temporal stability and motion quality comparable to Wan2.1-I2V-14B.
Comparison to ODE-Init: After flow-map training, AnyFlow produces clearer causal video results than Consistency ODE-Init at both 4 and 32 NFEs.The comparison fine-tunes the teacher on the same synthetic dataset for fairness.
Training Cost Breakdown: At 16 steps, flow-map backward simulation reduces training cost by 43.4% for causal models and 47.0% for bidirectional models versus consistency backward simulation.The savings come from shortcut transitions learned by the flow map.

6. Conclusion

AnyFlow introduces an any-step video diffusion distillation framework based on two-time flow maps and on-policy backward simulation. It matches or outperforms consistency-based methods in few-step sampling while improving with larger sampling budgets, though external-data dependence can cause distribution shift.

Conclusion: AnyFlow is the first any-step video diffusion distillation framework based on a two-time flow-map formulation.It learns transitions between arbitrary time pairs within a single model.
Conclusion: Flow map backward simulation enables on-policy distillation that reduces discretization error and exposure bias during sampling.The method combines this procedure with an improved forward flow-map training recipe.
Conclusion: Across bidirectional and causal architectures from 1.3B to 14B parameters, AnyFlow matches or outperforms consistency-based counterparts in few-step sampling and improves as sampling budgets increase.The results support a practical and scalable path toward high-quality any-step video generation.
Limitations: External datasets for flow-map training may create distribution shifts from the base model, including smoother textures.Using the same data as base-model pretraining could mitigate this issue.
Future Work: Future work should develop more effective and stable forward flow-map training strategies to improve robustness across different NFE regimes.Another direction is learning any-step causal video diffusion from both data and a teacher model.