Mean Flows for One-step Generative Modeling

1 Introduction

• FID 3.43 at 1-NFE on ImageNet 256×256 outperforms previous state-of-the-art methods in its class by a relative margin of 50% to 70%.The model is trained from scratch without pre-training, distillation, or curriculum learning; the study says it largely closes the gap with multi-step diffusion/flow models. Figure 1 compares 1-NFE MF, iCT, and Shortcut with IMM’s 1-step result, which uses 2-NFE guidance.Source: Figure 1 · p. 1, §1 · p. 2
• Consistency Models [43] [15] [31] impose a constraint on network behavior, but leave the underlying ground-truth field properties that should guide learning unknown.The paper says this can make training unstable and require a carefully designed discretization curriculum to progressively constrain the time domain [43] [15].Source: §1 · p. 1
• MeanFlow directly models average velocity and trains the network to satisfy its intrinsic relation to instantaneous velocity, without an extra consistency heuristic.Average velocity is displacement divided by a time interval, with displacement given by integrating instantaneous velocity. The ground-truth target makes the optimal solution independent, in principle, of the network, which can support more robust and stable training; classifier-free guidance can be incorporated into the target field at no additional sampling cost.Source: §1 · p. 2

Diffusion and Flow Matching

• Diffusion models reverse progressively added noise through SDEs or probability-flow ODEs, while Flow Matching models velocity fields between distributions and can be viewed as continuous-time Normalizing Flows.Diffusion models [42] [44] [19] [45] and Flow Matching methods [28] [2] [30] are cited; the probability-flow ODE formulation cites [45] [22], and the Normalizing Flows view cites [36].Source: §2 · p. 2

Few-step Diffusion/Flow Models

• Few-step generation includes distillation of pre-trained models [39] [14] [41], score distillation [50] [53], and standalone Consistency Models that constrain outputs across time steps to share trajectory endpoints.Early few-step training explorations built on distillation-based methods; further consistency models and training strategies are cited in [43] [15] [31] [49].Source: §2 · p. 2
• Two-time methods include Flow Maps [3], which represent displacement between time steps, Shortcut Models [13], which add flow self-consistency, and Inductive Moment Matching [52], which models stochastic-interpolant self-consistency.The paper contrasts Flow Map displacement with its own average-velocity basis.Source: §2 · p. 2

3 Background: Flow Matching

• Flow Matching [28] [30] [1] defines paths as z_t=a_t x+b_t epsilon and conditional instantaneous velocity; the schedule a_t=1−t, b_t=t gives v_t=epsilon−x.Here x is sampled from the data distribution and epsilon from the prior.Source: §3 · p. 2
• Because different data-prior pairs can yield the same z_t, Flow Matching [28] uses their expected conditional velocities as the marginal velocity field.The figure distinguishes conditional flows from the marginal field used as the ground-truth network target.Source: §3 · p. 3, Figure 2 · p. 3
• Although the marginal-velocity loss is infeasible to compute directly, minimizing the conditional Flow Matching loss is equivalent to minimizing it [28].The conditional loss trains the network against conditional velocity targets.Source: §3 · p. 3
• Samples are generated by solving an ODE from prior noise, approximating its velocity integral with discrete steps such as Euler; higher-order solvers are also applicable.Euler updates each state using the velocity at the current time and state.Source: §3 · p. 3
• Even straight conditional flows typically induce curved marginal trajectories, so coarse numerical ODE discretizations can be inaccurate.The curvature arises in the ground-truth marginal field, not only from neural-network approximation.Source: §3 · p. 3

4.1 Mean Flows

• MeanFlow models average velocity as displacement per time interval, distinct from instantaneous velocity, and trains it through an identity relating the two fields.The identity provides a regression target using instantaneous velocity and derivatives of the parameterized average-velocity field, avoiding integral computation during training.Source: §4.1 · pp. 3–5
• One-step sampling replaces the time integral with a single average-velocity evaluation, mapping z1 = ϵ to z0 = z1 − u(z1, 0, 1).Few-step sampling is also straightforward under this formulation.Source: §4.1 · p. 5

4.2 Mean Flows with Guidance

• MeanFlow supports classifier-free guidance by modeling the guided average-velocity field directly, preserving single-NFE sampling without a sampling-time guidance combination.The guided field is constructed from class-conditional and class-unconditional velocity fields.Source: §4.2 · p. 6

4.3 Design Decisions

• Design decisions investigate squared-L2 and adaptively weighted loss metrics, uniform or logit-normal sampling of (r, t), and conditioning on (r, t) or (t, t − r).Samples assign the larger time to t and smaller to r, with some samples set to r = t.Source: §4.3 · p. 7

5 Experiments

• Major experiments evaluate ImageNet 256×256 generation using FID on 50K generated images, with models trained from scratch on pre-trained VAE-tokenizer latents.The tokenizer produces a 32×32×4 latent input, and the study examines NFE with 1-NFE generation as the default.Source: §5 · p. 7

5.1 Ablation Study

• A non-zero r != t sampling ratio enables meaningful 1-NFE generation, with the best FID at 25%, whereas incorrect JVP computation fails to yield meaningful results.A 0% r != t ratio reduces to Flow Matching and fails for 1-NFE generation; a 100% ratio still yields a valid result. The JVP tangents for r and t are one-dimensional, but both variables are critical to determining the field u.Source: §5.1 · p. 7, Table 1 · p. 8
• All tested (r, t) embeddings yield meaningful 1-NFE results; embedding (t, t −r) performs best, direct (r, t) nearly as well, and interval-only t −r remains reasonable.MeanFlow can be viewed as Flow Matching with a modified target, and reduces to standard Flow Matching when r always equals t.Source: §5.1 · pp. 7–8, Table 1 · p. 8
• The logit-normal sampler performs best; loss power p = 1 is best, p = 0.5 is competitive, and squared L2 underperforms, while CFG substantially improves 1-NFE quality.The (r, t) pair is sampled independently, then ordered to enforce t > r and capped at a specified proportion of r != t; the CFG formulation supports 1-NFE sampling.Source: §5.1 · p. 8, Table 1 · p. 8
• MeanFlow shows promising 1-NFE scalability across larger model sizes and longer training durations.The models are trained from scratch, with CFG applied while maintaining 1-NFE sampling.Source: §5.1 · p. 8, Figure 4 · p. 9

5.2 Comparisons with Prior Work

• 3.43 FID for MeanFlow is over 50% better than IMM’s one-step 7.77 FID; at 1-NFE, it improves nearly 70% over Shortcut’s 10.60 FID.The authors also report that MeanFlow largely closes the gap between one-step and many-step diffusion/flow models.Source: §5.2 · p. 8, Table 2 · p. 9
• 2.20 FID at 2-NFE is on par with DiT’s 2.27 FID and SiT’s 2.15 FID, both using 250×2 NFE and the same XL/2 backbone.The results suggest few-step diffusion/flow models can rival many-step predecessors.Source: §5.2 · pp. 8–9, Table 2 · p. 9
• On CIFAR-10 32×32 unconditional generation, MeanFlow is competitive with prior approaches using 1-NFE sampling.Evaluation uses FID-50K and a roughly 55M-parameter U-net directly on pixel space; competitors use an EDM-style preconditioner, while MeanFlow does not.Source: §5.2 · p. 9

6 Conclusion

• Table 3 is labeled as covering unconditional CIFAR-10.The supplied caption does not provide table values or comparisons.Source: Table 3 · p. 9
• MeanFlow presents a principled and effective framework for one-step generation.Source: §6 · p. 9
• The formulation describes underlying quantities at coarsened levels of granularity.Source: §6 · p. 9
• The work relates its generative modeling scenario to multi-scale simulation problems in physics involving ranges of scales, lengths, and resolutions.Source: §6 · p. 9
• Numerical simulation is inherently limited by computers’ ability to resolve the range of scales.Source: §6 · p. 9
• Describing quantities at coarser granularities is presented as a common theme underlying important applications in physics.Source: §6 · p. 9
• The authors hope the work will bridge research in generative modeling, simulation, and dynamical systems in related fields.Source: §6 · p. 9

A Implementation

• For ImageNet 256×256, a standard VAE produces 32×32×4 latents for a DiT backbone conditioned on two time variables through summed positional-embedding MLP outputs.The DiT blocks are left unchanged; architectural improvements are described as orthogonal and possible.Source: A Implementation · p. 14
• For class-unconditional CIFAR-10 generation, the implementation uses a 32×32×3 pixel-space U-net and concatenated positional embeddings for (t, t−r), without an EDM preconditioner.The setup follows standard Flow Matching practice [29]; the U-net is developed from [44] and is approximately 55M parameters.Source: A Implementation · p. 14
• Training uses Adam at learning rate 0.0006 with batch size 1024 for 800K iterations, including 10K warm-up; 75% of samples use r≠t.Other settings include dropout 0.2, EMA decay 0.99995, lognorm(−2.0, 2.0) time sampling, and adaptive-weighting power p=0.75.Source: A Implementation · p. 14

B.1 Improved CFG for MeanFlow

• With ω′=2.0, varying κ to mix class-conditional and class-unconditional predictions further improves generation quality.The main-paper ablation set κ=0 and did not include this improvement.Source: Table 5 · p. 15, §B.1 · p. 15
• The revised target combines ωv(zt, t | c), κucfg(zt, t, t | c), and (1−ω−κ)ucfg(zt, t, t), mixing conditional and unconditional predictions.κ controls the class-conditional term; setting κ=0 recovers the CFG case.Source: §B.1 · p. 15, Table 5 · p. 15
• The construction preserves the original CFG formulation with effective guidance ω′=ω/(1−κ), while retaining the same loss function.For fixed ω′=2.0, ω is set to (1−κ)·ω′.Source: §B.1 · p. 15, Table 5 · p. 15

B.2 Loss Metrics

• The powered L2 loss uses regression error ∆ = uθ − utgt and a user-specified exponent γ.The loss is written as Lγ = ∥∆∥_2^γ.Source: §B.2 · p. 15
• Minimizing the powered loss is equivalent to adaptively weighting squared L2 error, with weight proportional to the error norm raised to γ−1 [15].The weighted loss uses the stop-gradient operator on the weight.Source: §B.2 · p. 15
• The practical weighting sets p = 1−γ and uses a small c>0 to avoid division by zero; p=0.5 is similar to the Pseudo-Huber loss [43].The adaptively weighted loss is sg(w) · L, where sg denotes stop-gradient.Source: §B.2 · p. 15

B.3 On the Sufficiency of the MeanFlow Identity

• Starting from definitional Eq. (4), the paper derives MeanFlow Identity Eq. (6), establishing necessity; it then states Eq. (6) is sufficient for Eq. (4), completing the equivalence.At t = r, the cited text gives C1 = C2 and says this indicates Eq. (6) ⇒Eq. (4).Source: §B.3 · pp. 15–16
• The integration constants cancel because S|t=r = 0 and the integral from r to t vanishes when t = r.Although equality of derivatives generally gives equality of integrals only up to constants, these boundary conditions establish C1 = C2.Source: §B.3 · pp. 15–16
• Modeling average velocity u rather than displacement S makes the boundary condition automatic; directly parameterizing S would require an extra condition S|t=r = 0.The sufficiency follows from this formulation choice.Source: §B.3 · p. 16

B.4 Analysis on Jacobian-Vector Product (JVP) Computation

• Because the JVP product is stop-gradient, it is constant during SGD backpropagation with respect to θ and adds no parameter-backpropagation overhead.The product is stop-gradient under Eq. (10).Source: §B.4 · p. 16
• JVP’s extra backward pass propagates only to input variables, not parameters θ, making it less costly than standard parameter backpropagation.The passage describes analogous interfaces in deep-learning libraries such as JAX.Source: §B.4 · p. 16
• 0.052 sec/iter for MeanFlow versus 0.045 sec/iter for Flow Matching, a 16% wall-clock overhead in B/4 on v4-8 TPUs.The benchmark used a JAX implementation.Source: §B.4 · p. 16

• Fig. 5 shows curated generation examples from the 1-NFE model on ImageNet 256×256.Source: C Qualitative Results · p. 16