Improved Mean Flows: On the Challenges of Fastforward Generative Models

Zhengyang Geng; Yiyang Lu; Zongze Wu; Eli Shechtman; J. Zico Kolter; Kaiming He

Source-linked AI summary

Improved Mean Flows: On the Challenges of Fastforward Generative Models

Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, Kaiming He

arXiv:2512.02012v2Dec 1, 2025cs.CVcs.LG

TL;DR

MeanFlow’s fastforward setting leaves a network-dependent training target and fixed classifier-free guidance scale, limiting standard regression and inference flexibility. The paper introduces iMF, which uses instantaneous-velocity regression, explicit guidance conditioning, and in-context conditioning, achieving 1.72 FID at 1-NFE on ImageNet 256×256 from scratch. The authors conclude that fastforward generation can be competitive without distillation, while noting tokenizer cost as a practical boundary.

Problem
Original MeanFlow has a network-dependent training target and fixes classifier-free guidance during training, sacrificing standard regression and inference-time flexibility.
Method
iMF reformulates training as instantaneous-velocity regression with legitimate noisy-sample inputs, treats guidance as conditioning, and uses in-context conditioning for diverse conditions.
Results
1.72 FID on ImageNet 256×256 is achieved with 1-NFE generation trained from scratch, a relative 50% improvement over original MF without distillation or pre-trained alignment models.
Takeaways & Limitations
Fastforward generative models can achieve highly competitive performance without pretraining and may serve as a stand-alone generation framework.
Takeaways & Limitations
At 1-NFE, tokenizer use incurs a non-negligible inference cost, motivating efficient tokenizers or pixel-space generation.

Abstract

from arXiv · show

MeanFlow (MF) has recently been established as a framework for one-step generative modeling. However, its ``fastforward'' nature introduces key challenges in both the training objective and the guidance mechanism. First, the original MF's training target depends not only on the underlying ground-truth fields but also on the network itself. To address this issue, we recast the objective as a loss on the instantaneous velocity $v$, re-parameterized by a network that predicts the average velocity $u$. Our reformulation yields a more standard regression problem and improves the training stability. Second, the original MF fixes the classifier-free guidance scale during training, which sacrifices flexibility. We tackle this issue by formulating guidance as explicit conditioning variables, thereby retaining flexibility at test time. The diverse conditions are processed through in-context conditioning, which reduces model size and benefits performance. Overall, our $\textbf{improved MeanFlow}$ ($\textbf{iMF}$) method, trained entirely from scratch, achieves $\textbf{1.72}$ FID with a single function evaluation (1-NFE) on ImageNet 256$\times$256. iMF substantially outperforms prior methods of this kind and closes the gap with multi-step methods while using no distillation. We hope our work will further advance fastforward generative modeling as a stand-alone paradigm.

1. Introduction

Fastforward generative models target very few or single-step samples, but MeanFlow has unresolved challenges in its network-dependent training target and fixed guidance scale. iMF reformulates training and guidance to improve stability and flexibility, achieving 1.72 FID at 1-NFE on ImageNet 256×256.

1. Introduction: Fastforward models reduce generation to very few or even one step by training across large time intervals.They approximate differential-equation trajectories using look-ahead mappings and other approximations.
1. Introduction: MeanFlow learns average velocity across time steps and uses the MeanFlow identity to create a trainable objective without infeasible training-time integration.The framework differs from directly learning the instantaneous velocity field underlying the ODE.
1. Introduction: Original MF has two unresolved issues: its training target depends on the network, while its classifier-free guidance scale is fixed during training.The fixed scale sacrifices inference-time flexibility.
1. Introduction: iMF reformulates the objective as a network-agnostic instantaneous-velocity regression with inputs restricted to noisy samples, substantially stabilizing training.The average velocity remains predicted by the network, but the regression target no longer depends on that network.
1. Introduction: iMF treats guidance values as conditioning variables and uses in-context conditioning for diverse conditions, retaining flexible CFG with 1-NFE sampling.The architecture accommodates guidance and other conditions without fixing the guidance scale before training.
1. Introduction: 1.72 FID on ImageNet 256×256 marks a relative 50% improvement over original MF in trained-from-scratch 1-NFE generation.The models use neither distillation nor pre-trained alignment models, and the result narrows the gap with multi-step methods.

2. Related Work

Fastforward generative modeling extends diffusion and flow-based generation by explicitly incorporating trajectory-solving acceleration into training. Related methods achieve this through large temporal jumps, trajectory learning, derivative regression, midpoint relationships, or MeanFlow refinements.

Diffusion and Flow-based Models: Diffusion and flow-based models learn trajectories represented by ODEs or SDEs and typically generate samples by numerical solving.These methods provide the foundation for modern generative approaches.
Fastforward Generative Models: Fastforward generative models explicitly incorporate ODE/SDE acceleration into training objectives rather than treating acceleration only as a sampling concern.They form an emerging category distinct from standard diffusion and flow-based designs.
Fastforward Generative Models: Related approaches make large time-step jumps, including endpoint leaping, arbitrary-time trajectory learning, derivative regression, and midpoint-based relationships.The listed examples include Consistency Models, Consistency Trajectory Models, Flow Map Matching, and Shortcut Models.
Fastforward Generative Models: MeanFlow improvements include scheduled Flow Matching-to-MeanFlow interpolation, pretrained-model finetuning, and mid-training with fixed regression targets.AlphaFlow, Decoupled MeanFlow, and CMT respectively use these strategies.

3. Background

Flow Matching learns instantaneous velocity fields for ODE-based generation, whereas MeanFlow learns average velocity between time steps and derives a differential identity for feasible training. Original MF approximates this identity with conditional velocity and network-predicted average velocity.

Flow Matching: Flow Matching uses z_t = (1 − t)x + tε and regresses the conditional velocity v_c = ε − x with a velocity-space loss.Data x comes from the data distribution and noise ε from the prior distribution.
Flow Matching: Because multiple data-noise pairs can produce the same z_t, the unique regression target is the marginal velocity v(z_t) = E[v_c | z_t].The marginal averages over all pairs satisfying z_t at time t.
Flow Matching: Flow Matching generates samples by numerically integrating dz_t/dt = v_θ(z_t) from t = 1 to 0.Euler or Heun solvers can perform the integration from prior samples.
MeanFlow: MeanFlow introduces average velocity u between times r and t, then differentiates it with respect to t to obtain the MeanFlow identity used for training.Direct integration of the average-velocity definition during training is intractable.
MeanFlow: The identity’s derivative term is computed with a Jacobian-vector product between derivatives of u and tangent vector [v, 0, 1].The shorthand JVP(u; v) denotes this computation at u(z_t) and v(z_t).
MeanFlow: Original MF replaces marginal velocity with conditional velocity and true average velocity inside the JVP with network prediction u_θ, creating its training target.Stop-gradient helps create an apparent target, after which MF performs one-step sampling with z_0 = z_1 − u_θ(z_1).

4. Improved Mean Flows

This section reformulates MeanFlow as a more standard v-loss, replaces fixed classifier-free guidance with conditioning, and introduces in-context conditioning for heterogeneous inputs. These changes address network-dependent targets, improve training behavior, preserve inference flexibility, and reduce model size.

MeanFlow as v-loss: MeanFlow can be viewed as a v-loss re-parameterized by the network that predicts average velocity u.The reformulation is equivalent to the original MeanFlow objective and is driven by the MeanFlow identity.
MeanFlow as v-loss: The improved formulation makes the prediction function depend only on zt by parameterizing the marginal velocity instead of using conditional velocity e −x.This removes the extra unknown-dependent input and yields a legitimate standard regression function.
MeanFlow as v-loss: Original MF’s loss has much higher variance and is non-decreasing under the reported comparison settings.The comparison uses MeanFlow-B/2 with basic ℓ2 loss, no adaptive weighting, and no classifier-free guidance; samples with t = r are excluded.
Flexible Guidance: Flexible guidance conditions the network on the CFG scale, allowing the scale to vary during training and inference while retaining 1-NFE sampling.The optimal CFG scale shifts with training and inference settings, so it cannot be fixed optimally beforehand.
Improved In-context Conditioning: In-context conditioning represents time, class, and guidance conditions as tokens and concatenates them with image latent tokens.Removing parameter-heavy adaLN-zero reduces model size by one third, from 133M to 89M for iMF-Base, while maintaining performance.

5. Experiments

Experiments evaluate iMF through 1-NFE ImageNet 256×256 generation, ablations, and system-level comparisons. The improvements progressively reduce FID, while iMF outperforms original MeanFlow and prior fastforward models trained from scratch.

MeanFlow as v-loss: The improved training formulation reduces w/o-CFG FID from 32.69 to 29.42 with the boundary-condition variant.This variant adds no extra parameters during training or inference.
MeanFlow as v-loss: The auxiliary-head variant achieves 5.68 FID with CFG, about a 10% relative improvement over original MeanFlow.The auxiliary head introduces no extra parameters or compute at inference time.
Flexible guidance: Richer CFG-conditioning lowers FID from 5.68 to 4.57 by enabling CFG intervals at inference without changing 1-NFE sampling.Using only ω-conditioning yields a smaller improvement from 5.68 to 5.52.
In-context conditioning: In-context conditioning reduces model size from 133M to 89M and improves FID from 4.57 to 4.09.Replacing parameter-heavy adaLN-zero provides the reported reduction and performance gain.
System-level comparison: iMF substantially outperforms fastforward models trained from scratch; at 2-NFE, it achieves 1.54 FID and further narrows the gap with multi-step models.The reported 1-NFE result also outperforms fastforward methods distilled from pretrained models.

6. Conclusion

The paper positions fastforward generation as a stand-alone direction and notes a practical inference cost from tokenizer use. Future work is directed toward more efficient or tokenizer-free generation.

Conclusion: Fastforward generative models trained without pretraining can achieve highly competitive performance, supporting their development as a stand-alone framework.The conclusion presents this result as a step toward stand-alone fastforward generation.
Conclusion: Tokenizer use incurs a non-negligible inference cost as 1-NFE generation improves.The paper identifies efficient tokenizers and pixel-space generation as future research directions.

A. Implementation Details

Implementation details cover the auxiliary velocity head, CFG training and guidance intervals, in-context conditioning, initialization, and evaluation. The training pipeline uses JVP-based computations and evaluates 50,000 generated samples with FID-50K.

Auxiliary head for vθ: The auxiliary v-head shares most computation with uθ, differs in the last eight layers, and predicts the marginal instantaneous velocity v.Its output supplies the JVP computation and is not used at inference time.
Auxiliary head for vθ: An auxiliary Flow Matching loss trains unshared auxiliary-head layers that otherwise receive no gradient.The auxiliary loss is adaptively weighted like the MeanFlow loss.
CFG conditioning: CFG training samples guidance scales from [1.0, 8.0], using a power distribution that biases toward smaller values.The scale ω=1 corresponds to the no-CFG case.
CFG conditioning: CFG interval training samples tmin from U[0, 0.5] and tmax from U[0.5, 1.0], turning guidance off outside the interval by setting ω=1.The values {ω, tmin, tmax} are provided as extra conditioning.
In-context conditioning: In-context conditioning processes time, class, and CFG conditions through positional embeddings, two-layer MLPs, and replicated tokens.The model conditions on r, t, c, and Ω={ω, tmin, tmax}.
Evaluation: Evaluation samples 50,000 images and reports FID against the ImageNet training set, using 50 images per class.For CFG-conditioned models, reported FID uses the optimal guidance scale and interval.

B. Additional Qualitative Results

The paper presents uncurated iMF-XL/2 samples for 1-NFE ImageNet 256×256 generation under class conditions. The evaluation uses CFG settings that favor individual quality over diversity, yielding an FID of 3.92 and an IS of 348.2.

The qualitative samples are uncurated and correspond to the classes provided as conditions.They come from the iMF-XL/2 model.
The results use iMF-XL/2 for 1-NFE ImageNet 256×256 generation.
CFG is set to ω = 6.0 with interval [tmin, tmax] = [0.2, 0.8], favoring IS over FID.This emphasizes individual quality at the expense of diversity and distributional coverage.
3.92 FID and 348.2 IS are reported for this evaluation setting.