PaperlayerView exact revision on arXiv

Source-linked AI summary

AlphaFlow: Understanding and Improving MeanFlow Models

Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Qu, Ivan Skorokhodov

arXiv:2510.20771v1cs.CVcs.LG

TL;DR

Few-step generative models seek to reduce diffusion sampling cost, but MeanFlow’s success and optimization behavior are not fully understood. The paper decomposes MeanFlow, introduces α-Flow with an annealed curriculum, and finds consistently stronger ImageNet performance than MeanFlow, including FID scores of 2.58 and 2.15 at 1-NFE and 2-NFE.

  • Problem

    Few-step diffusion generation is motivated by slow sampling, while MeanFlow’s objective and the role of its flow-matching supervision require deeper understanding.

  • Method

    α-Flow unifies trajectory flow matching, Shortcut Models, and MeanFlow, using a curriculum that anneals from trajectory flow matching to MeanFlow.

  • Results

    α-Flow consistently outperforms MeanFlow across scales and one- and few-step settings; its largest DiT-XL/2+ model achieves FID 2.58 at 1-NFE and 2.15 at 2-NFE.

  • Takeaways & Limitations

    The analysis establishes flow-matching supervision as necessary for MeanFlow and supports α-Flow as a generalization for stronger few-step image generation from scratch.

  • Takeaways & Limitations

    The authors report occasional unstable training in large-scale models with guidance integration, so α-Flow does not eliminate known consistency-model instability issues.

Abstract

from arXiv · show

MeanFlow has recently emerged as a powerful framework for few-step generative modeling trained from scratch, but its success is not yet fully understood. In this work, we show that the MeanFlow objective naturally decomposes into two parts: trajectory flow matching and trajectory consistency. Through gradient analysis, we find that these terms are strongly negatively correlated, causing optimization conflict and slow convergence. Motivated by these insights, we introduce $α$-Flow, a broad family of objectives that unifies trajectory flow matching, Shortcut Model, and MeanFlow under one formulation. By adopting a curriculum strategy that smoothly anneals from trajectory flow matching to MeanFlow, $α$-Flow disentangles the conflicting objectives, and achieves better convergence. When trained from scratch on class-conditional ImageNet-1K 256x256 with vanilla DiT backbones, $α$-Flow consistently outperforms MeanFlow across scales and settings. Our largest $α$-Flow-XL/2+ model achieves new state-of-the-art results using vanilla DiT backbones, with FID scores of 2.58 (1-NFE) and 2.15 (2-NFE).

1 INTRODUCTION

Few-step generative modeling addresses diffusion’s slow inference, but existing methods still trail multi-step models. This work analyzes MeanFlow, introduces α-Flow, and reports stronger ImageNet results with vanilla DiT models.

  • Motivation: Diffusion models often require many denoising steps for high-fidelity samples, motivating efficient few-step generators.Consistency models enable training few-step models from scratch, but a performance gap with multi-step diffusion models remains.
  • MeanFlow analysis: MeanFlow’s objective contains trajectory flow matching and trajectory consistency, whose negative correlation creates optimization conflict and slow convergence.The analysis also identifies border-case flow matching as surrogate supervision, while over 75% of computation is spent on it.
  • α-Flow: α-Flow unifies trajectory flow matching, Shortcut Models, and MeanFlow through a curriculum that transitions from flow matching to MeanFlow.The curriculum reduces reliance on border-case supervision and improves convergence.
  • Results: α-Flow consistently outperforms MeanFlow across model scales and one- and few-step settings on class-conditional ImageNet-1K 256x256.The largest DiT-XL/2+ model achieves FID 2.58 at 1-NFE and 2.15 at 2-NFE.

2 PRELIMINARIES

Diffusion models generate samples through noisy or deterministic trajectories, while consistency-based methods aim to support one- or few-step generation. MeanFlow and related trajectory methods address this setting, but continuous consistency training can require costly Jacobian-vector products.

  • Diffusion models and flow matching: Flow matching replaces diffusion’s noisy forward process with a deterministic straight-line path between noise and data distributions.A neural vector field is trained to model the trajectory velocity.
  • Consistency models: Consistency models directly map noisy inputs to clean samples, enabling one-step generation by enforcing agreement across nearby timesteps.Discrete consistency training involves a step-size trade-off between discretization error and training stability.
  • Consistency training: Continuous-time consistency training removes discretization error but relies on Jacobian-vector products that can hinder scalability and efficiency.The continuous gradient is the limiting form of the discrete-time gradient divided by the timestep difference.
  • Trajectory models: Consistency trajectory models enforce consistency from t to r, allowing arbitrary jumps during inference and thereby supporting multi-step generation.Shortcut Models impose consistency between one shortcut step and two half-sized shortcut steps.
  • MeanFlow: MeanFlow is presented as a strong one- or few-step diffusion and flow model whose effectiveness motivates further analysis.Its training and inference framework is examined in the subsequent analysis.

3 ANALYZING MEANFLOW TRAINING

MeanFlow decomposes into trajectory flow matching and trajectory consistency, whose conflicting gradients make joint optimization difficult. The analysis motivates α-Flow, which unifies related objectives and uses an annealed intermediate timestep to improve optimization.

  • 3.1 UNDERSTANDING THE OBJECTIVE: MeanFlow decomposes into trajectory flow matching and trajectory consistency, with trajectory flow matching implicitly supplying trajectory consistency’s missing boundary condition.The consistency term is a reweighted continuous consistency loss without an explicit boundary condition.
  • 3.2 EMPIRICAL ANALYSIS: Gradient cosine similarity is typically below −0.4 between trajectory flow matching and trajectory consistency, revealing strong optimization conflict.The authors attribute this to the broad solution manifold of trajectory consistency versus the narrow manifold of trajectory flow matching.
  • 3.2 EMPIRICAL ANALYSIS: Border-case flow matching LFM′ lowers LTFM while interfering less with LTCc because it applies only at r = t, where LTCc = 0.Figure 2 compares gradient similarities and evaluated losses under 0% versus 75% LFM′ training allocation.
  • 4.1 α-FLOW: UNIFYING ONE, FEW, AND MANY-STEP FLOW-BASED MODELS: The α-Flow loss unifies trajectory flow matching, Shortcut Model training, and MeanFlow through an intermediate timestep s determined by the consistency ratio α.The intermediate timestep is s = α · r + (1 −α) · t, and α controls its position within the interval.
  • 4.1 α-FLOW: UNIFYING ONE, FEW, AND MANY-STEP FLOW-BASED MODELS: Annealing α from 1 to 0 interpolates between trajectory flow matching and MeanFlow without complex timestep partitioning.Once t and r are sampled, s is immediately determined by the fixed or scheduled α.

5 EXPERIMENTS

Experiments evaluate α-Flow on ImageNet-1K 256×256 against prior few-step models and study schedules, flow-matching ratios, and sampling choices. α-Flow improves over MeanFlow across major settings, while curriculum duration and sampling strategy materially affect results.

  • 5.1 COMPARISON WITH BASELINE: Across 240-epoch training, α-Flow-XL/2 reaches 2.95 FID, improving over MeanFlow-XL/2 by 15%.The same comparison reports 164.6 FDD and a 12% relative improvement over MeanFlow-XL/2.
  • 5.2 ABLATION STUDY: Longer trajectory-flow-matching pretraining and smoother transitions consistently improve α-Flow across evaluated metrics.The best schedule, Sigmoid150K→250K, outperforms Constant0.0, and longer transition durations improve generation quality.
  • 5.2 ABLATION STUDY: α-Flow outperforms MeanFlow across evaluated flow-matching ratios, with α-Flow’s best 1-NFE FID at 25% r = t versus MeanFlow’s best at 75%.The reported best FID values are 40.0 for α-Flow and 43.1 for MeanFlow.
  • 5.2 ABLATION STUDY: The flow-matching ratio trades off one-step and two-step performance: higher ratios worsen NFE=1 but improve NFE=2 generation.For α-Flow, the 75% ratio performs worse at NFE=1 but better at NFE=2 than the 50% version.
  • 5.2 ABLATION STUDY: Consistency sampling is strongest for α-Flow-XL/2 and α-Flow-XL/2+, while ODE sampling is strongest for MeanFlow-XL/2.The best reported FIDs are 2.09 and 2.28 for the α-Flow models under consistency sampling, versus 2.39 for MeanFlow under ODE sampling.

6 CONCLUSION

The paper analyzes MeanFlow’s objective and uses that analysis to propose α-Flow for stronger few-step image generation models trained from scratch.

  • 6 CONCLUSION: α-Flow generalizes MeanFlow while enabling consistently stronger few-step image generation models trained from scratch.The conclusion attributes the proposal to a principled analysis of MeanFlow and identifies flow-matching supervision as necessary during training.

7 REPRODUCIBILITY STATEMENT

The paper describes implementation and release steps intended to support reproducibility, including detailed methods, code, and checkpoints. The surrounding context situates this work within few-step diffusion and trajectory-based generative modeling.

  • 7 REPRODUCIBILITY STATEMENT: Implementation details are provided in Section F, with source code and ImageNet-1K 256×256 checkpoints slated for public release.The planned release covers training, inference, and evaluation code.
  • 7 REPRODUCIBILITY STATEMENT: Few-step diffusion research targets the high inference cost caused by iterative sampling in conventional diffusion models.The related-work passages describe distillation, consistency models, and trajectory-based methods as approaches to reducing sampling steps.

B LIMITATIONS

The paper identifies several limitations affecting α-Flow’s robustness, monitoring, theoretical interpretation, and scaling behavior.

  • Large-scale models with guidance integration occasionally show unstable training for both MeanFlow and α-Flow.The authors caution that α-Flow does not eliminate the known instability issues of consistency models.
  • α-Flow’s curriculum delays consistency training, leaving few-step performance weak before the transition point and making progress harder to monitor.Pure flow matching supervision is used up to k_s iterations.
  • The gradient analysis is empirical and does not fully explain theoretically why flow matching is critical for consistency.
  • Batch-size scaling may exhibit diminishing returns beyond a certain size, limiting the benefits of larger batches.The passage also notes that the observed fine-tuning improvements may be sensitive to hyperparameters.

C FAILED EXPERIMENTS

The authors report unsuccessful experiments involving decomposed loss weighting and consistency sampling, while noting that some directions may have been underexplored.

  • The authors present these failures as potentially underexplored directions or genuine dead-ends rather than definitive negative conclusions.
  • Every tested decomposed-training configuration performed worse than the default adaptive loss heuristic.The experiments used individually tuned weighting functions for the MeanFlow objective’s terms.
  • Consistency sampling did not deliver the anticipated improvements, although the optimal setting was not fully reported in the supplied passage.

D PROOFS OF THINGS

The proofs establish reparameterizations connecting MeanFlow-related losses to consistency-style objectives and characterize α-Flow’s unified formulation.

  • The MeanFlow loss is reparameterized into a consistency-related loss with an associated loss-weighting function.
  • Theorem 1’s proof treats flow matching, Shortcut Models, and MeanFlow within the same formulation, with α defined from the timestep interval.The proof sets Δt = t − s and α = Δt/(t − r).
  • The equivalence proof recovers a consistency-model setting by choosing ṽ_s,t = v_t, r = 0, and a corresponding model parameterization.
  • The discrete consistency objective corresponds to a special timestep partition where Δt = α · t for fixed α.

E ANALYSIS DETAILS

The analysis uses controlled DiT experiments, batch-size fine-tuning, sampling comparisons, and ablations to evaluate α-Flow and its training choices.

  • The loss and gradient-cosine analyses use repeated 1000-iteration evaluations with reported means and 5%–95% percentiles.Gradient similarities are computed pairwise on the same batch for the relevant losses.
  • The DiT-B/2-non-cfg model serves as the ablation and analysis configuration for ImageNet 256×256 experiments.
  • G.1 ABLATION STUDY OVER BATCH SIZE: α-Flow-XL/2+ achieves 1-NFE FID 2.58 versus 3.06 for MeanFlow-XL/2+, and 2-NFE FID 2.15 versus 2.16.It is worse on 2-NFE FDD, with 96.8 versus 93.4.
  • G.1 ABLATION STUDY OVER BATCH SIZE: Batch size 1024 performs well across metrics and is selected for the MeanFlow-XL/2+ configuration.Batch size 512 gives the best 1-NFE FID of 3.05 and FDD of 164.3, while 1024 gives the best FDD of 93.4.
  • The sampling implementation supports consistency and ODE branches over a sequence of timesteps, with NFE-specific evaluation.
  • The study includes an overall α-Flow ablation and a fine-tuning batch-size ablation alongside the stated implementation configurations.

G.2 ABLATION STUDY OVER α-FLOW DESIGN SPACE

The ablation examines α-Flow design choices, including the training objective, adaptive weighting, consistency-step ratio, and their combined effect on the gap between Shortcut Models and MeanFlow.

  • The ablation uses a DiT-B/2-non-cfg model pretrained with flow matching for 200k iterations and then fine-tuned on α-Flow for 200k iterations, with α fixed and r = t for 25% of samples.
  • The model converges only when the target velocity is set to v_t and EMA is removed for u_θ−, differing from the Shortcut Model setup.
  • The new adaptive weight performs better empirically than the original MeanFlow adaptive weight, especially in the ablation comparison.
  • α = 5 × 10^-3 is the optimal consistency step ratio and is used as the schedule’s clamping value.
  • Combining the α-Flow design improvements significantly reduces the performance gap between Shortcut Models and MeanFlow.

I RANDOM VS BALANCED CLASSES FOR FID COMPUTATION

The section compares random and balanced class sampling for FID computation and reports that balanced sampling can improve FID while leaving FDD and FCD largely unchanged.

  • The balanced protocol generates 50 samples for each of 1000 classes instead of sampling 50,000 classes independently at random.
  • Because balanced sampling is not standard practice, the authors report it separately from random class sampling with an appropriate notice.
  • Table 6 compares balanced and random class sampling using FID, FDD, and FCD.
  • Balanced class sampling greatly improves FID results, whereas FDD and FCD are barely affected.
  • The accompanying sample figures show uncurated ImageNet examples across multiple classes and one- or two-step generation settings.
Loading 2510.20771v1…