PaperlayerView exact revision on arXiv

Source-linked AI summary

Few-Step Distillation for Text-to-Image Generation: A Practical Guide

Yifan Pu, Yizeng Han, Zhiwei Tang, Jiasheng Tang, Fan Wang, Bohan Zhuang, Gao Huang

arXiv:2512.13006v1cs.CV

TL;DR

Text-to-image diffusion is computationally costly, and the applicability of newer distillation methods to open-ended prompts remains unclear. The paper proposes a unified T2I comparison and adaptation study; its results show a trade-off between few-step efficiency and peak performance.

  • Problem

    Text-to-image diffusion requires many network evaluations, while the performance and adaptability of newer distillation methods for open-domain prompts remain unclear.

    Source: §1 · pp. 1–2

  • Method

    The paper compares distillation methods and proposes a unified experimental plan for adapting them to text-to-image generation.

    Source: §1 · p. 2

  • Results

    sCM reaches 52.81% overall on GenEval at NFE=2 versus the teacher's 53.58%, while MeanFlow requires more steps for high fidelity and collapses at NFE=1–2.

    Source: §4.3 · pp. 7–8

  • Takeaways & Limitations

    The report offers a basis for rigorous and fair empirical evaluation of methods adapted to text-to-image synthesis.

    Source: §1 · p. 2

    The IMM-to-consistency-model reduction depends on the stated boundary condition at s=0.

    Source: §3.4 · p. 5

Abstract

from arXiv · show

Diffusion distillation has dramatically accelerated class-conditional image synthesis, but its applicability to open-ended text-to-image (T2I) generation is still unclear. We present the first systematic study that adapts and compares state-of-the-art distillation techniques on a strong T2I teacher model, FLUX.1-lite. By casting existing methods into a unified framework, we identify the key obstacles that arise when moving from discrete class labels to free-form language prompts. Beyond a thorough methodological analysis, we offer practical guidelines on input scaling, network architecture, and hyperparameters, accompanied by an open-source implementation and pretrained student models. Our findings establish a solid foundation for deploying fast, high-fidelity, and resource-efficient diffusion generators in real-world T2I applications. Code is available on github.com/alibaba-damo-academy/T2I-Distill.

1 Introduction

Text-to-image diffusion models produce high-quality images but require many sampling evaluations, while the suitability of newer distillation methods for open-ended prompts remains unclear. The report compares methods, proposes a unified T2I evaluation plan, and provides implementation resources.

1 Introduction

  • The report identifies high latency from hundreds of network evaluations and unclear T2I performance for sCM, MeanFlow, and IMM as central challenges.The uncertainty concerns their performance, adaptability, and potential advantages in open-domain text-to-image synthesis.Source: §1 · pp. 1–2
  • Distribution distillation matches a fast student to a slow teacher, while trajectory distillation trains students to predict teacher sampling segments.DMD minimizes distribution difference with regression loss, and LADD uses an adversarial latent-space game; trajectory distillation can also capture classifier-free guidance.Source: §1 · pp. 1–2
  • The report proposes a theoretical comparison of methods' mechanisms, stability, and scalability, plus a unified experimental plan for adapting them to T2I.The plan particularly emphasizes sCM and MeanFlow and aims at rigorous, fair evaluation.Source: §1 · p. 2
  • The proposed modular codebase includes adaptations, training recipes, evaluation pipelines, and baseline models to support reproducibility and extension.The report also reviews few-step generation approaches, typically targeting one to eight network evaluations.Source: §1 · pp. 1–2

2 A brief overview of each method

The overview introduces sCM, MeanFlow, and IMM as distinct approaches to few-step generation, with IMM matching distributions across time and MeanFlow also adapted for teacher-based distillation.

2 A brief overview of each method

  • The overview frames the chapter as an analysis of each method's mechanism and innovations, alongside an actionable plan for adapting it to text-to-image synthesis.The analysis is intended to inform subsequent experimental design.Source: §2 · p. 2

2.1 stabilized Continuous-time Consistency Models (sCM)

  • sCM combines TrigFlow, architectural improvements, and adaptive loss weighting to address instability in continuous-time consistency-model training.Consistency models map a sample at a noise level directly to its clean sample.Source: §2.1 · pp. 2–3

2.2 MeanFlow

  • MeanFlow models average velocity between two time steps, and the proposed distillation algorithm uses a flow-matching teacher to provide instantaneous-velocity targets.This extends the described standard MeanFlow training algorithm with teacher-provided velocity.Source: §2.2 · p. 3

2.3 Inductive Moment Matching (IMM)

  • IMM is trained from scratch to make mappings to a target time consistent across starting times, using MMD to align distributions and implicitly their moments.The target time s is earlier than both starting times r and t.Source: §2.3 · p. 3

3 Methodological Relationships and Comparative Insights

The section relates the methods through special cases and conversions: Flow Matching is a MeanFlow setting, FM and TrigFlow are interoperable, and MeanFlow and IMM connect to consistency-model objectives under stated conditions.

3.1 FM v.s. MeanFlow: Flow Matching as a Special Case of MeanFlow (when r ≡t)

  • Setting MeanFlow's reference time r equal to current time t removes its second target term, recovering Flow Matching's instantaneous-velocity objective.The general formulation conditions on both t and r; this specialization reduces the target to v.Source: §3.1 · p. 3

3.2 FM v.s. sCM: Flow Matching and TrigFlow are interconvertible without re-training

  • Flow Matching and TrigFlow models and samplers can be converted in both directions without retraining, enabling cross-framework use.The conversion maps time and state variables and transforms model outputs between the frameworks.Source: §3.2 · p. 4

3.3 sCM v.s. MeanFlow: MeanFlow as a Synchronization Limit of sCM

  • MeanFlow's gradient is equivalent to sCM's when the target network is synchronized with the online network at every step; MeanFlow therefore omits sCM's EMA target stabilization.The comparison describes MeanFlow as a simplified variant with a more current, self-referential training dynamic.Source: §3.3 · p. 5

3.4 CM v.s. IMM: Consistency Models as a Special Case of Implicit Moment Matching

  • With a single-particle estimate and the stated boundary-condition assumption, the IMM loss reduces to a discrete-time consistency-model loss using ℓ2 distance.The passage also specifies a negative squared Euclidean kernel and target time s=0 for the derivation.Source: §3.4 · p. 5

4 Text-to-Image Adaptation with FLUX.1-lite

The section adapts sCM and MeanFlow to the 8B FLUX.1-lite teacher and compares their few-step text-to-image behavior. It finds a budget-dependent trade-off: sCM is stable at one or two evaluations, while MeanFlow needs four for high fidelity.

4.1 Experimental Setup

  • Experiments use an 8-billion-parameter FLUX.1-lite teacher and a proprietary text-to-image dataset, with sCM trained for 3,000 iterations and MeanFlow for 25,000.The dataset is not publicly released; the stated iteration counts reflect the observed GenEval trends for each method.Source: §4.1 · p. 6

4.2 Methodological Adaptation for Text-to-Image Generation

  • Rescaling the timestep input to [0, 1] stabilizes sCM training, and the resulting student performs comparably to the teacher on GenEval.The original [0, 1000] range led to continually increasing gradient norms and eventual collapse; the rescaled student was distilled at 1024 × 1024 for 120K iterations.Source: §4.2.1 · p. 6

4.2.2 MeanFlow Adaptation

  • MeanFlow adds a second timestep embedding for (t − r), sums it with the embedding for t, and uses a pretrained flow-matching teacher’s velocity as its distillation target.The teacher signal replaces the noisy raw target, while the resulting objective focuses on trajectory-curvature correction.Source: §4.2.2 · p. 6
  • For the vanilla MeanFlow target, setting γ=2 yields superior performance, increasing GenEval from 44.04% to 48.65%; this configuration effectively minimizes the fourth power of the regression-error norm.Source: §4.2.2 · p. 7
  • Improved CFG raises MeanFlow’s GenEval score from 48.65% to 51.41% by mixing class-conditional and unconditional predictions in the regression target.The modification is described as aligning more closely with standard classifier-free guidance practice.Source: §4.2.2 · p. 7

4.3 Quantitative and Qualitative Results

  • At NFE=2, sCM scores 52.81% on GenEval versus the teacher’s 53.58%; at NFE=4, MeanFlow scores 80.03 versus sCM’s 77.85 on DPG-Bench.MeanFlow’s GenEval score falls to 0.78% at NFE=1, while Table 2 reports teacher-level DPG-Bench performance for MeanFlow at NFE=4.Source: Table 1 · p. 7, Table 2 · p. 7, §4.3 · pp. 7–8
  • The rescaled teacher performs nearly identically to original FLUX.1-lite, indicating that timestep normalization does not compromise generation quality.The evaluation uses GenEval and DPG-Bench.Source: §4.3 · p. 7
  • Figure 1 shows sCM producing coherent images at NFE=1, while MeanFlow produces noise at NFE=1 and sharper, semantically correct images at NFE=4.For sCM, additional steps mainly refine high-frequency details and textures.Source: Figure 1 · p. 8, §4.3 · p. 8

5 Codebase

The codebase implements multiple distillation algorithms for FLUX.1-lite through two training pipelines, MeanFlow and simplified Consistency Matching (sCM). It uses a modified teacher–student architecture and on-the-fly latent encoding.

5 Codebase

  • The implementation provides MeanFlow trajectory distillation with Jacobian-vector products and sCM with tangent-vector matching and learned per-sample variance.MeanFlow also supports optional classifier-free guidance; sCM uses a TrigFlow reparameterization.Source: §5 · p. 8
  • The described framework employs a teacher–student paradigm based on a modified FLUX.1-lite MMDiT architecture; the cited passage does not specify whether this applies to both pipelines.The passage also reports DeepSpeed ZeRO, bfloat16 mixed-precision, and gradient checkpointing.Source: §5 · p. 8
  • The pipeline encodes inputs on the fly with a pretrained AutoencoderKL in a 16-channel latent space with 8× downsampling.This latent encoding is part of the training pipeline.Source: §5 · p. 8

A.1 Interconversion between TrigFlow and Flow Matching

This section aligns TrigFlow and Flow Matching by matching their signal-to-noise ratios and deriving conversions between their time variables and estimators. The conversions map samples between the frameworks and use the Flow Matching estimator to construct an optimal TrigFlow estimator.

A.1 Interconversion between TrigFlow and Flow Matching

  • Matching the models’ signal-to-noise ratios yields an interconversion relationship between the Flow Matching and TrigFlow time variables.The passage gives tFM = sin(tTrig) / (sin(tTrig) + cos(tTrig)) and tTrig = arctan(tFM / (1 − tFM)).Source: §A.1 · p. 10
  • The derivation maps noisy samples from the two frameworks to the same distribution and uses the Flow Matching estimator to construct the optimal TrigFlow estimator.The stated optimality relies on conditional expectation minimizing mean squared error.Source: §A.1 · pp. 10–11
  • The conversion formulas express the TrigFlow estimator in terms of the Flow Matching estimator.The section introduces these formulas after deriving the relationship between the frameworks.Source: §A.1 · p. 11

A.2 sCM v.s. MeanFlow: Equivalence of Gradients

Under the Flow Matching parameterization, the sCM and simplified MeanFlow training-objective gradients are structurally equivalent. Their error-signal terms are numerically identical, with the gradients differing only by sCM’s weighting factor t.

A.2 sCM v.s. MeanFlow: Equivalence of Gradients

  • Under the Flow Matching parameterization, sCM and simplified MeanFlow have structurally equivalent training-objective gradients.The parameterization uses fθ(xt, t) = xt − tFθ(xt, t), satisfying fθ(x0, 0) = x0.Source: §A.2 · p. 11
  • The sCM gradient follows from differentiating its loss with respect to θ, using ∇θfθ(xt, t) = −t∇θFθ(xt, t).Here, dt is identified as the velocity of the path xt.Source: §A.2 · p. 11
  • The methods’ error-signal terms are numerically identical, and their gradients differ only by the weighting factor t in sCM.This comparison assumes θ− represents stopped-gradient parameters, so Fθ is numerically equal to Fθ−.Source: §A.2 · p. 12

Related papers

Loading 2512.13006v1…