PaperlayerView exact revision on arXiv

Source-linked AI summary

SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation

Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Song Han, Enze Xie

arXiv:2503.09641v4cs.GR

TL;DR

Ultra-few-step consistency models can lose quality, while existing distillation approaches incur instability or computational overhead. SANA-Sprint transforms a pre-trained flow-matching model for continuous-time consistency distillation and combines it with latent adversarial distillation. It achieves state-of-the-art speed-quality results, including 7.59 FID and 0.74 GenEval in one step while generating 1024×1024 images in 0.1 seconds on H100.

  • Problem

    Consistency models can suffer quality erosion below four steps, while GAN- and VSD-based methods introduce instability, tuning demands, or computational overhead.

  • Method

    SANA-Sprint transforms a pre-trained flow-matching model into a TrigFlow model for sCM distillation and combines sCM with LADD in a unified step-adaptive framework.

  • Results

    7.59 FID and 0.74 GenEval are achieved in one step, surpassing FLUX-schnell's 7.94 FID and 0.71 GenEval while generating 1024×1024 images in 0.1 seconds on H100.

  • Takeaways & Limitations

    SANA-Sprint supports high-quality ultra-fast generation and ControlNet-guided interaction at 0.25 seconds on H100 for human-in-the-loop workflows.

  • Takeaways & Limitations

    The transformation prioritizes existing noise schedules but deviates from the unit variance principle used in prior TrigFlow work.

Abstract

from arXiv · show

This paper presents SANA-Sprint, an efficient diffusion model for ultra-fast text-to-image (T2I) generation. SANA-Sprint is built on a pre-trained foundation model and augmented with hybrid distillation, dramatically reducing inference steps from 20 to 1-4. We introduce three key innovations: (1) We propose a training-free approach that transforms a pre-trained flow-matching model for continuous-time consistency distillation (sCM), eliminating costly training from scratch and achieving high training efficiency. Our hybrid distillation strategy combines sCM with latent adversarial distillation (LADD): sCM ensures alignment with the teacher model, while LADD enhances single-step generation fidelity. (2) SANA-Sprint is a unified step-adaptive model that achieves high-quality generation in 1-4 steps, eliminating step-specific training and improving efficiency. (3) We integrate ControlNet with SANA-Sprint for real-time interactive image generation, enabling instant visual feedback for user interaction. SANA-Sprint establishes a new Pareto frontier in speed-quality tradeoffs, achieving state-of-the-art performance with 7.59 FID and 0.74 GenEval in only 1 step - outperforming FLUX-schnell (7.94 FID / 0.71 GenEval) while being 10x faster (0.1s vs 1.1s on H100). It also achieves 0.1s (T2I) and 0.25s (ControlNet) latency for 1024 x 1024 images on H100, and 0.31s (T2I) on an RTX 4090, showcasing its exceptional efficiency and potential for AI-powered consumer applications (AIPC). Code and pre-trained models will be open-sourced.

1. Introduction

Diffusion models require many denoising steps, while existing distillation methods trade off stability, flexibility, memory, or ultra-few-step quality. SANA-Sprint addresses these constraints with hybrid consistency and adversarial distillation, delivering fast, step-adaptive generation and real-time ControlNet interaction.

  • Diffusion models typically require 50-100 iterative denoising steps, motivating time-step distillation for efficient inference.
  • GAN-based distillation can suffer training instability, mode collapse, and architectural rigidity requiring careful hyperparameter tuning.
  • Consistency models avoid some instability but can lose quality below four steps because trajectory truncation errors degrade text-to-image semantic alignment.
  • SANA-Sprint transforms pre-trained Flow Matching models into TrigFlow models without retraining the teacher, then combines sCM with LADD for alignment and fidelity.
  • 7.59 FID and 0.74 GenEval surpass FLUX-schnell’s 7.94 FID and 0.71 GenEval, while SANA-Sprint is reported as 10× faster.
  • 0.10s-0.18s H100 latency supports 1024×1024 generation, while ControlNet integration enables real-time interactive generation in 0.25s.

3. Method

SANA-Sprint transforms a pre-trained flow matching model into a TrigFlow model for continuous-time consistency distillation, then combines consistency and adversarial supervision. The method also addresses schedule mismatches, training instability, and few-step quality through differentiable transformations, stabilization changes, and timestep sampling.

  • Training-Free Transformation to TrigFlow: A training-free mathematical transformation converts a pre-trained flow matching model into a theoretically lossless TrigFlow counterpart.The transformation computes an equivalent flow-matching timestep, rescales the input, and deterministically transforms the output.
  • Training-Free Transformation to TrigFlow: TrigFlow and flow matching mismatch in time domains, noise schedules, data scales, and prediction targets, requiring explicit input/output transformations.TrigFlow uses [0, π/2], while flow matching uses [0, 1]; their schedules and velocity targets also differ.
  • Training-Free Transformation to TrigFlow: The lossless transformation lets SANA-Sprint directly adopt the sCM training framework without separate algorithm designs or a dedicated TrigFlow pre-training stage.The transformed inputs and outputs remain differentiable and compatible with automatic differentiation.
  • Stabilizing Continuous-Time Distillation: SANA-Sprint stabilizes continuous-time distillation with dense timestep embeddings and QK normalization, addressing excessive gradient norms and model collapse during scaling.The timestep noise coefficient changes from 1000t to t, while RMS normalization is added to Query and Key in self- and cross-attention during teacher fine-tuning.
  • Improving Continuous-Time CMs with GAN: The hybrid objective combines sCM loss with GAN loss to preserve teacher alignment and diversity while providing direct adversarial supervision for fidelity and cross-timestep learning.The adversarial term uses a default λ = 0.5, and its supervision addresses the local nature of consistency-model distillation.
  • Improving Continuous-Time CMs with GAN: A modified timestep sampling distribution significantly improves one- and few-step generation capability.This modification is evaluated in Tab. 6.

4. Experiments

Experiments evaluate SANA-Sprint’s efficiency, quality, step flexibility, and training components. Results show strong performance from 1–4 steps, with hybrid losses and schedule transfer improving stability or quality.

  • Experimental Setup: SANA-Sprint is evaluated on MJHQ-30K using FID and CLIP Score, and on GenEval, with efficiency comparisons against state-of-the-art methods.The comparison reports throughput on batch=10 and latency on batch=1; speed measurements use one A100 GPU with BF16 precision.
  • Efficiency and Performance Comparison: 5.34 samples/s and 0.32s latency are achieved by SANA-Sprint 0.6B at 4 steps, with 6.48 FID and 0.76 GenEval.At 2 steps, the 0.6B model reaches 6.46 samples/s and 0.25s latency with 6.54 FID.
  • Efficiency and Performance Comparison: 7.22 samples/s and 0.21s latency are achieved in single-step mode by SANA-Sprint 0.6B, maintaining 7.04 FID and 0.72 GenEval.These results are reported as comparable to FLUX-schnell with significantly higher efficiency.
  • Analysis: Removing schedule transfer causes training divergence, whereas incorporating it produces decent results within 5,000 iterations.The ablation compares a flow-matching SANA model with and without transformation to TrigFlow.
  • Analysis: Combining sCM and LADD yields 8.11 FID and 28.02 CLIP score, improving over LADD alone at 12.20 FID and 27.00 CLIP score.The results support complementary benefits from the two losses, while 50% max-time weighting is selected as the default balance.
  • Visual Comparison: Figure 5 shows high-quality images from one or two steps, with image quality improving when the number of steps increases.The visual comparison is against the teacher model SANA.

5. Related Work

Related work organizes diffusion step distillation into trajectory-based and distribution-based paradigms, including consistency-model and GAN-based variants.

  • Distillation Paradigms: Trajectory-based methods include direct distillation, progressive distillation, and consistency-model variants such as LCM, CTM, MCM, PCM, and sCM.Distribution-based methods include GAN-based distillation and VSD variants.

6. Conclusion

The conclusion presents SANA-Sprint as a one-step model that retains multi-step flexibility through hybrid distillation. It reports strong speed-quality performance and identifies real-time interactive generation as a practical direction.

  • Conclusion: SANA-Sprint combines continuous-time consistency distillation and latent adversarial distillation for ultra-fast one-step generation with multi-step sampling flexibility.The model is introduced as an efficient diffusion model for text-to-image generation.
  • Conclusion: 0.1s H100 latency for 1024×1024 images establishes the reported speed-quality tradeoff, while instant feedback supports real-time interactive applications.The conclusion connects this capability to responsive creative tools and AIPC.

A. Pseudo Code for Training-Free Transformation from Flow to Trigflow

The training-free transformation wraps a trained flow-matching model as a TrigFlow model using mathematical input, timestep, and output transformations.

  • Transformation Procedure: The procedure converts a TrigFlow timestep to a flow-matching timestep before processing the transformed input through the pretrained flow-matching model.The input feature is scaled according to the converted timestep.
  • Transformation Procedure: The flow-matching model output is transformed back into the TrigFlow representation through an output formula using the converted timestep and scaled input.The implementation defines this wrapper through the TrigFlowModel forward method.
  • Training-Free Conversion: The transformation requires no additional training and enables a pretrained flow-matching model to follow the TrigFlow consistency-distillation algorithm.The approach is based on theoretical equivalence between the two formulations.

B. Transformation Algorithm

The transformation algorithm converts a pre-trained flow matching model into its TrigFlow counterpart without additional training. It maps timesteps, rescales noisy inputs, and applies a deterministic transformation to produce the output.

  • The algorithm performs a training-free transformation from a flow matching model to its TrigFlow counterpart.It takes a noisy sample, its TrigFlow timestep, and a pre-trained flow matching model as inputs.
  • The transformed inputs are passed through the model to produce a deterministic transformed result.
  • The procedure computes the equivalent flow matching timestep from the TrigFlow timestep.
  • The procedure rescales the TrigFlow noisy input to obtain the corresponding flow matching input.

C. Training Algorithm of SANA-Sprint

This section presents SANA-Sprint’s training algorithm, emphasizing its modifications relative to standard sCM training. The procedure includes transformations and parameter updates specific to SANA-Sprint.

  • SANA-Sprint’s training algorithm highlights modified steps that differ from the standard sCM training algorithm.
  • The complete procedure includes SANA-Sprint-specific transformations and parameter updates.
  • Algorithm 2 outlines the training procedure for SANA-Sprint.

D. Proof of Proposition 3.1

The proof motivates and establishes a transformation between TrigFlow and flow matching that preserves SNR while correcting scale differences. It further shows that, under the parameterization, TrigFlow training is equivalent to flow matching training apart from weighting and timestep sampling.

  • Motivation: Directly applying flow matching to TrigFlow-generated data is infeasible because their time parameterization, SNR, and output requirements differ.
  • Training algorithm: Algorithm 2 initializes transformed pretrained and student models, then alternates discriminator and generator updates until convergence.
  • Time and scale transformation: The timestep mapping relates t_FM and t_Trig through the stated sine, cosine, and arctangent transformations.
  • Time and scale transformation: The transformation preserves equal SNRs for the flow matching and TrigFlow noisy samples, while their scales remain different.
  • Time and scale transformation: A scale factor λ(t_FM) aligns the TrigFlow coefficients with the flow matching coefficients.
  • Distributional equivalence: The transformed TrigFlow sample follows the flow matching model’s training distribution.
  • Estimator and training equivalence: Conditional expectation gives the optimal flow matching estimator because it minimizes the MSE loss.
  • Estimator and training equivalence: Training TrigFlow with the proposed parameterization is equivalent to training flow matching, except for the loss weighting function w(t) and timestep sampling distribution p(t).

E. Full Related Work

The related work situates SANA-Sprint within advances in text-to-image models, diffusion step distillation, and real-time image generation. These lines of work emphasize scalable architectures, accelerated inference, and model compression.

  • Text-to-image generation: Text-to-image research progressed from Stable Diffusion to transformer-based architectures such as DiT and PixArt-α.
  • Diffusion model step distillations: Diffusion step distillation includes trajectory-based methods such as Direct Distillation and Progressive Distillation.
  • Diffusion model step distillations: Consistency models, including LCM, CTM, MCM, PCM, and sCM, predict the PF-ODE solution from a noisy input.
  • Real-time image generation: Real-time image generation research combines one-step distillation, backward distillation, quantization, and weight dilation to improve efficiency.

F.1. Experimental Setup

The experiments evaluate SANA-Sprint across model scales, inference settings, noise distributions, and qualitative T2I and ControlNet results. They use FID, CLIP Score, and GenEval alongside visual demonstrations of speed, detail, semantic understanding, and control.

  • Model and training setup: Teacher models are fine-tuned from SANA 0.6B and 1.6B, with architecture, training data, and other hyperparameters aligned with SANA-1.5.
  • Evaluation protocol: Evaluation uses FID and CLIP Score on MJHQ-30K plus GenEval on 553 prompts to assess image quality and text-image alignment.GenEval emphasizes text-image alignment and improvement potential.
  • Inference settings: Inference timestep optimization searches settings sequentially for 1-, 2-, and 4-step generation across the 0.6B and 1.6B models using FID and CLIP Score.The procedure first optimizes t_max for one-step inference, then searches intermediate timesteps for multi-step settings.
  • Noise-distribution ablations: The sCM noise configuration (P_mean, P_std) = (0.0, 1.6) is selected after tracking FID and CLIP-Score over 40k iterations because it reduces FID, improves CLIP-Score, and avoids instability.Extreme mean values, including 0.6 and −0.8, can cause training instability or failure.
  • Noise-distribution ablations: The LADD discriminator noise study identifies (−0.6, 1.0) as its optimal setting, while high-noise biasing can destabilize training.The study compares sampled-time distributions and their resulting FID and CLIP-Score performance.
Loading 2503.09641v4…