PaperlayerView exact revision on arXiv

Source-linked AI summary

MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generators

Yushi Huang, Xiangxin Zhou, Jun Zhang, Liefeng Bo, Tianyu Pang

arXiv:2607.15273v2cs.CVcs.LG

TL;DR

MeanFlow uses average-velocity sampling, while existing forward-process RL targets instantaneous velocities, leaving efficient RL finetuning underexplored. MeanFlowNFT bridges them with an induced instantaneous-velocity predictor and improves image and video generation, including outperforming multi-step RL with few sampling steps.

  • Problem

    Applying forward-process RL to MeanFlow is challenging because DiffusionNFT targets instantaneous velocities while MeanFlow samples with average velocities.

  • Method

    MeanFlowNFT derives an induced instantaneous-velocity predictor from the MeanFlow identity, applies a DiffusionNFT-style objective, and preserves average-velocity few-step sampling.

  • Results

    Across image and video generation, MeanFlowNFT consistently improves MeanFlow baselines and prior few-step methods; on SD3.5-M, it achieves the best results on 6 of 8 metrics.

  • Takeaways & Limitations

    MeanFlowNFT can surpass multi-step RL-tuned generators while using only a few sampling steps, extending efficient reward optimization to MeanFlow models.

  • Takeaways & Limitations

    The study evaluates only DiffusionNFT-style forward-process RL on MeanFlow and does not examine other objectives or broader flow-map models.

Abstract

from arXiv · show

MeanFlow generators achieve fast few-step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task-specific objectives. In particular, DiffusionNFT offers an efficient forward-process RL framework that does not require reverse-process trajectories or likelihood estimation. However, applying such RL methods to MeanFlow remains underexplored. DiffusionNFT optimizes instantaneous velocities, whereas MeanFlow samples with average velocities. To bridge this gap, we introduce MeanFlowNFT. Inspired by the MeanFlow identity, which bridges average and instantaneous velocities, we construct an induced instantaneous-velocity predictor. We apply the DiffusionNFT objective to this predictor, making reward optimization well-defined for MeanFlow. Sampling remains based on the average velocity, preserving MeanFlow's fast few-step generation. We further prove that MeanFlowNFT inherits DiffusionNFT's strict policy-improvement guarantee. Experiments on image and video generation show that MeanFlowNFT consistently improves baselines. Moreover, it outperforms prior state-of-the-art RL-tuned few-step generators on most metrics ($6$ of $8$ on SD3.5-M), and can even surpass multi-step RL-tuned diffusion while using only a few sampling steps. For instance, on Wan 2.1, $4$-step MeanFlowNFT reaches a VBench score of $84.33$, surpassing $50$-step LongCat-Video RL ($82.57$).

1 Introduction

MeanFlowNFT introduces the first forward-process RL framework for MeanFlow generators by applying a DiffusionNFT-style objective to an induced instantaneous-velocity predictor. It preserves likelihood-free training and MeanFlow’s efficient few-step sampling while targeting the bottleneck caused by sequential instantaneous-velocity integration.

  • MeanFlow accelerates generation by predicting average velocity over a time interval instead of integrating instantaneous velocity through many sequential steps.
  • MeanFlowNFT is the first forward-process RL framework for MeanFlow generators.
  • The method derives an induced instantaneous-velocity predictor from the MeanFlow identity and applies the DiffusionNFT-style objective during RL training.
  • MeanFlowNFT keeps training likelihood-free and leaves the efficient few-step sampler unchanged.
  • MeanFlowNFT delivers strong generation quality while preserving MeanFlow’s few-step efficiency.The trainable and reference predictors use the same finite-difference estimate along the forward-process conditional velocity.

2 Preliminaries

The preliminaries define Flow Matching’s instantaneous-velocity formulation and MeanFlow’s average-velocity alternative for efficient few-step sampling. They also summarize DiffusionNFT’s forward-process reinforcement-learning framework and its policy-improvement guarantee.

  • Flow Matching: Flow Matching learns an instantaneous velocity predictor for a probability-flow ODE that transports Gaussian noise to the conditional data distribution.For rectified flow, the conditional velocity is v_t = ϵ − x_0, while the optimal squared-loss predictor is the deterministic marginal velocity.
  • MeanFlow: MeanFlow predicts the average velocity over finite intervals, enabling large sampling jumps instead of many instantaneous-velocity evaluations.Its average velocity depends on both interval endpoints and converges to the instantaneous velocity as s → t.
  • MeanFlow: The MeanFlow identity converts the path-integral average-velocity target into a trainable regression target using the conditional instantaneous velocity and a total-derivative correction.The correction vanishes when s = t, reducing the MeanFlow objective to the Flow Matching objective.
  • Few-step sampling: MeanFlow sampling updates states directly with predicted average velocities, requires no instantaneous-velocity evaluation at inference, and can use a single step in principle.The update follows the exact displacement identity x_s = x_t − (t − s)u(x_t, s, t).
  • DiffusionNFT: DiffusionNFT uses forward-process Flow Matching reinforcement learning to guide a reference model toward improved policies without reverse-sampler policy gradients.At its exact optimum, v_θ* = v_old + 2/β Δ, and β = 2α makes the trained model coincide with the positive policy.

3 MeanFlow Reinforcement via Forward-Process RL

MeanFlowNFT preserves MeanFlow’s average-velocity parameterization and few-step sampler while optimizing an induced instantaneous-velocity predictor with DiffusionNFT-style forward-process RL. Its theoretical guarantees transfer policy improvement to the deployed average velocity, while practical design choices support stable, efficient training.

  • 3.1 MeanFlowNFT: MeanFlowNFT keeps average-velocity predictions but constructs an induced instantaneous-velocity predictor for DiffusionNFT-style optimization.The induced predictor is built by substituting the MeanFlow network into the MeanFlow identity.
  • 3.1 MeanFlowNFT: Optimization of the induced predictor transfers improvement to the average velocity while retaining likelihood-free forward-process training and native few-step sampling.The method never unrolls the reverse denoising process or estimates per-step likelihoods.
  • 3.2 Theoretical guarantees: With guidance strength β = 2α(xt, c), the idealized induced optimum recovers the improved policy’s marginal instantaneous velocity for every interval start s ≤ t.This is the induced-velocity improvement target established by the corollary.
  • 3.2 Theoretical guarantees: If the induced optimum is attained on all intervals, MeanFlow consistency makes the optimal average velocity exact for the ODE induced by v+, so the resulting policy coincides with π+.The guarantee transfers the induced-velocity result to the deployed average-velocity network.
  • 3.3 Practical implementation: Practical training uses finite differences instead of an expensive JVP, shares the reference derivative to cancel destabilizing terms, and reuses conditional velocity vt to avoid extra evaluation.Using each predictor’s own derivative can cause collapse, whereas sharing the derivative saves computation; vt remains stable while tying the direction to bvθ collapses training.

4 Experiments

Experiments show that MeanFlowNFT improves few-step image and video generation, outperforming prior baselines while preserving stable training and test-time scaling. Ablations attribute this behavior to shared derivatives, model-free direction selection, and training on both zero-length and finite-interval pairs.

  • Image generation: 6 of 8 metrics: MeanFlowNFT achieves the best few-step results on SD3.5-M and outperforms distillation and recent few-step RL baselines.It records OCR 0.65 versus 0.30 for RTDMD and can surpass multi-step RL-tuned diffusion with only 4 sampling steps.
  • Image generation: Within 400 steps: CDM+DiffusionNFT diverges, while AnyFlow+DiffusionNFT and DMD/CDM+DiffusionNFT train unstably and collapse early.Applying DiffusionNFT directly to average-velocity or distribution-matching networks lacks the instantaneous-velocity structure needed for its policy-improvement guarantee.
  • Test-time scaling: N ∈ {2, 4, 8, 16, 32}: most metrics improve as sampling steps increase, while MeanFlowNFT remains more step-consistent than AnyFlow.Its samples vary less across step counts in layout and content, consistent with improved average-velocity estimation after RL.
  • Ablations: Sharing the derivative: reward rises smoothly across PickScore, HPSv2, and CLIPScore, whereas independent derivatives cause early reward collapse and uncontrolled predictor deviation.The shared derivative removes the derivative-difference term and stabilizes the reference relationship.
  • Ablations: Model-free direction and mixed pair training: using vt improves reward steadily, while training only on s=t pairs causes 4-step rollout reward to rise briefly before collapsing.The default combines zero-length instantaneous-velocity pairs with finite-interval average-velocity pairs, and vt avoids an additional network evaluation.

5 Conclusion

MeanFlowNFT is presented as the first forward-process RL method tailored to MeanFlow, using the MeanFlow identity to optimize instantaneous velocity while retaining average-velocity sampling. The method is theoretically connected to DiffusionNFT’s improved policy and empirically improves MeanFlow baselines, even surpassing multi-step RL with few sampling steps.

  • Method: MeanFlowNFT is the first forward-process RL method tailored for MeanFlow models.It addresses the mismatch between MeanFlow’s average-velocity prediction and DiffusionNFT’s instantaneous-velocity framework.
  • Method: The MeanFlow identity enables reward optimization on instantaneous velocity while preserving the average-velocity network for fast sampling.This bridges the two velocity formulations without replacing MeanFlow’s sampling mechanism.
  • Theory: In an idealized setting, MeanFlowNFT reaches DiffusionNFT’s improved policy and transfers that gain to the average-velocity model.The conclusion states this as a proved theoretical property.
  • Experiments: MeanFlowNFT consistently improves MeanFlow baselines and can surpass multi-step RL using only a few sampling steps.The experiments support both baseline improvement and few-step superiority over multi-step RL.

6 Limitations and Future Work … Appendix A. Related Work

The paper limits its study to DiffusionNFT-style forward-process RL for MeanFlow, while positioning the approach within established work on diffusion, flow, few-step generation, and RL alignment. It suggests the induced instantaneous-velocity construction may extend to other forward-process objectives and flow-map families.

  • 6 Limitations and Future Work: MeanFlowNFT studies only DiffusionNFT-style forward-process RL and does not evaluate other objectives such as RAM or AWM.The authors expect their induced instantaneous-velocity construction and implementation to carry over because RAM and AWM also use instantaneous velocity and frozen references.
  • 6 Limitations and Future Work: The study considers only MeanFlow, leaving broader flow-map families for future work.The passage identifies MeanFlow as one instance of a broader family of flow-map models.
  • Appendix A. Related Work: Diffusion and flow models are established paradigms for high-quality image and video generation, with latent diffusion improving efficiency through compressed latent spaces.Transformer backbones are also described as improving scalability.
  • Appendix A. Related Work: Few-step generation methods distill pretrained diffusion or flow models using progressive distillation, consistency models, distribution matching distillation, and adversarial distillation.A closely related direction learns flow maps, which transport samples directly between two points over long ranges.
  • Appendix A. Related Work: Reinforcement learning aligns diffusion and flow models with human preferences and task rewards through policy gradients, reward weighting, preference optimization, and reward backpropagation.The passage presents these as major existing approaches to reinforcement learning for diffusion models.
  • Appendix A. Related Work: RL for few-step generation commonly reinforces distribution-matching distillation, while other methods use reward-tilted targets or surrogate reward learning.The cited approaches either combine reward objectives with matching losses, alter target distributions, or post-train distilled generators.

Appendix B. Why Direct Plug-Ins Lack DiffusionNFT’s Guarantee

Directly applying DiffusionNFT’s loss to AnyFlow, DMD, or CDM preserves algebraic form but not its policy-improvement guarantee, because those models do not optimize the required marginal instantaneous velocity.

  • Why the guarantee fails: DiffusionNFT’s guarantee requires both a posterior-mean velocity identity and optimization of an instantaneous-velocity predictor.The pointwise optimum equals v+ when the optimized output is instantaneous velocity; reusing the loss for another output quantity breaks the policy interpretation.
  • AnyFlow: AnyFlow predicts interval-average velocity u rather than marginal instantaneous velocity v, so direct plug-in replaces vθ and vold with uθ and uold while retaining target vt.Conditioning on the interval endpoint s leaves the required posterior moments unchanged, but the optimized output remains an average velocity.
  • AnyFlow: For finite intervals, the direct AnyFlow objective does not guarantee u†(xt, s, t) = u+(xt, s, t), because the residual terms need not cancel.Both residual terms vanish as s approaches t, but DiffusionNFT’s identities do not force cancellation over a finite interval.
  • DMD and CDM: DMD and CDM’s distribution-matching objectives do not identify a unique generator map at fixed (xt, c, t), and their additional regression or alignment terms do not regress against the forward conditional velocity.Thus, their output semantics do not satisfy the condition underlying DiffusionNFT’s guarantee.

Appendix C. Proofs for MeanFlowNFT

Appendix C proves that the MeanFlow-induced optimum recovers the improved policy’s instantaneous velocity and, under attainment across all intervals, yields exact average-velocity sampling from that policy. Consequently, the deployed MeanFlow policy strictly improves over the old policy for any non-degenerate reward.

  • Proposition 3.1 and Corollary 3.2: With guidance strength β = 2α(x_t, c), the idealized pointwise optimum recovers the improved policy’s marginal instantaneous velocity for all s ≤ t.The proof derives the optimum from a strictly convex quadratic and identifies it with the DiffusionNFT reinforcement guidance.
  • Lemma 3.3: If u satisfies the MeanFlow consistency identity for all s ≤ t, then u is the exact average velocity of the ODE induced by v over [s, t].The proof establishes this by differentiating G(τ) = (τ − s)u(x_τ, s, τ) along ODE trajectories and integrating.
  • Theorem 3.4: Under attainment of the induced optimum for every interval, the optimal average velocity is exact for the ODE induced by v+, so the MeanFlow policy coincides with π+.The endpoint case gives the instantaneous velocity v+, and MeanFlow consistency extends this equality to all intervals.
  • Theorem 3.4: J(πθ*) = J(π+) > J(πold) for any non-degenerate reward.Thus the deployed MeanFlow policy inherits the strict policy-improvement guarantee.

Appendix D. More Implementation Details

MeanFlowNFT uses multi-reward objectives for image and video training, then applies short CFG-free RL finetuning runs with specified AnyFlow initialization and optimization settings. Evaluation additionally uses official VBench reporting across 16 dimensions on 8 NVIDIA H20 GPUs.

  • Reward: Image training uses equally weighted CLIPScore, PickScore, and HPSv2 rewards on the PickScore prompt set.The setup follows DiffusionNFT’s multi-reward configuration.
  • Reward: Video training combines HPSv3-general, HPSv3-percentile, VideoAlign motion-quality, and text-alignment rewards.HPSv3-general averages frame scores for the prompt “A high-quality image,” while HPSv3-percentile averages the top 30% of caption-conditioned frame scores.
  • Training: 6000 steps of flow-map pretraining construct the SD3.5-M MeanFlow policy using AnyFlow’s three-mode endpoint sampling and reverse-CFG scale 4.5.The modes allocate 50% to s = t, 25% to s = 0, and the remainder to s ∼U(0, t).
  • Training: 2000 steps on SD3.5-M and 1600 steps on Wan2.1 perform RL finetuning with CFG-free 4-step rollouts, β = 0.1, KL weight 10^-4, and learning rate 3×10^-6.Both runs use equally weighted reward dimensions, AdamW, and fresh (s, t) pairs from the same three-mode AnyFlow schedule.
  • Evaluation: 16 VBench dimensions are aggregated into Total, Quality, and Semantic scores, with evaluations run on 8 NVIDIA H20 GPUs.Full per-dimension results appear in Tables 3 and 4 alongside the aggregated results in Table 2.

Appendix E. Additional Test-time Scaling Results

Additional test-time scaling evaluations show that MeanFlowNFT preserves AnyFlow’s any-step scaling behavior while consistently achieving stronger generation quality on SD3.5-M and Wan2.1 1.3B.

  • Test-time scaling: MeanFlowNFT retains AnyFlow’s any-step scaling behavior and consistently delivers stronger generation quality on SD3.5-M and Wan2.1 1.3B.Figure 11 reports the remaining metrics for SD3.5-M, while Figure 12 reports corresponding results for Wan2.1 1.3B.

Appendix F. VBench Full Results

Appendix F provides the per-dimension VBench breakdown for the Wan2.1 1.3B video-generation experiment and additional test-time-scaling results for SD3.5-M and Wan2.1 1.3B.

  • Wan2.1 1.3B VBench results: The Wan2.1 1.3B VBench breakdown is split across Tables 3 and 4 for readability.The appendix reports per-dimension results from the experiment in Table 2.
  • Test-time scaling: Figure 11 reports additional quantitative results for MeanFlowNFT test-time scaling on SD3.5-M.The figure specifically concerns test-time scaling rather than the Wan2.1 1.3B breakdown.
  • Test-time scaling: Figure 12 reports MeanFlowNFT test-time-scaling results on Wan2.1 1.3B, combining VBench scores with evaluations on 256 held-out prompts.Total, Quality, and Semantic are VBench scores; HPSv3-G/HPSv3-P and MQ/TA are evaluated on the held-out prompts.
  • Wan2.1 1.3B VBench results: Table 3 presents the full VBench per-dimension results for Wan2.1 1.3B, covering part 1 of 2.Bold and underline identify the best and second-best results among few-step models.

Appendix G. More Qualitative Results

Appendix G presents qualitative comparisons of MeanFlowNFT for image and video generation against RL, distillation, and few-step baselines. Across these examples, MeanFlowNFT delivers more faithful and visually coherent results while maintaining strong quality across sampling steps.

  • Qualitative comparisons: The appendix compares MeanFlowNFT with multi-step and few-step RL methods, few-step distillation baselines, and direct DiffusionNFT applications for image generation.Video comparisons additionally include Wan2.1, LongCat-Video RL, and few-step video distillation baselines.
  • Qualitative comparisons: MeanFlowNFT produces more faithful and visually coherent image and video examples across different sampling steps.The qualitative results indicate that strong quality is preserved as the sampling-step count varies.
Loading 2607.15273v2…