PaperlayerView exact revision on arXiv

Source-linked AI summary

SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories

Zhangkai Wu, Xuhui Fan, Hongyu Wu, Longbing Cao

arXiv:2502.16972v4cs.CVcs.LG

TL;DR

Pre-trained diffusion models can require extensive resources and many evaluations, while consistency and rectified-flow distillation have different trajectory-related limitations. SCoT combines straightness with consistency, and the paper reports better efficiency and generation quality than existing distillation methods on CIFAR-10 and ImageNet.

  • Problem

    Consistency distillation can require multiple sampling steps, while rectified-flow distillation relies on numerical ODE solvers whose approximation errors may degrade output quality.

    Source: §1 · p. 2

  • Method

    SCoT distills pre-trained diffusion models by regulating trajectory velocities toward teacher point changes and making different-time points converge to the same future value.

    Source: §1 · p. 2

  • Results

    Across CIFAR-10 and ImageNet, SCoT outperforms existing distillation methods in efficiency and generation quality; on CIFAR-10 it achieves FID 2.30 with 2 NFEs.

    Source: §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    SCoT produces straight and consistent trajectories, with straightness facilitating projection-function approximation and faster sample generation.

    Source: §1 · p. 2

    The authors leave a detailed study of using one shared random noise vector for both losses to future work.

    Source: §3.1 · p. 5

Abstract

from arXiv · show

Pre-trained diffusion models are commonly used to generate clean data (e.g., images) from random noises, effectively forming pairs of noises and corresponding clean images. Distillation on these pre-trained models can be viewed as the process of constructing advanced trajectories within the pair to accelerate sampling. For instance, consistency model distillation develops consistent projection functions to regulate trajectories, although sampling efficiency remains a concern. Rectified flow method enforces straight trajectories to enable faster sampling, yet relies on numerical ODE solvers, which may introduce approximation errors. In this work, we bridge the gap between the consistency model and the rectified flow method by proposing a Straight Consistent Trajectory~(SCoT) model. SCoT enjoys the benefits of both approaches for fast sampling, producing trajectories with consistent and straight properties simultaneously. These dual properties are strategically balanced by targeting two critical objectives: (1) regulating the gradient of SCoT's mapping to a constant, (2) ensuring trajectory consistency. Extensive experimental results demonstrate the effectiveness and efficiency of SCoT.

1 Introduction

Trajectory distillation seeks to accelerate sampling from pre-trained diffusion models, but consistency and rectified-flow approaches each have limitations. SCoT combines straightness and consistency in its trajectories and reports state-of-the-art results for generating high-quality data in few or even one step.

1 Introduction

  • Trajectory distillation trains a student using trajectories between noise and clean-image pairs generated by pre-trained teachers, aiming to accelerate sampling.Pre-trained diffusion models can require extensive resources to train and many evaluations to generate high-quality samples.Source: §1 · p. 1
  • Consistency distillation typically needs multiple steps, while rectified-flow distillation uses numerical ODE solvers whose approximation errors can degrade output quality.The stated consistency-model challenge is tracking trajectories with high curvature.Source: §1 · p. 2
  • SCoT targets straightness by matching student velocities to teacher point changes and consistency by projecting points from different times to a shared future value.The paper presents these as its two trajectory-regulation objectives.Source: §1 · p. 2
  • SCoT is the first method described here to produce straight and consistent trajectories, and it reports state-of-the-art results for high-quality generation in N steps or even one step.Its design unifies consistency models and rectified flows.Source: §1 · p. 2

2 A trajectory perspective on Pre-trained diffusion model distillations

The section frames diffusion-model distillation as constructing trajectories between noise and clean images, with different approaches imposing different trajectory properties. It describes rectified-flow and consistency-based methods, including their respective mechanisms and trade-offs.

2 A trajectory perspective on Pre-trained diffusion model distillations

  • The trajectory uses a time interval from 0 to 1, with ordered time steps running from tN = 1 to t1 = 0.The notation assigns θ to the teacher and ϕ to the student distillation model.Source: §2 · p. 2
  • A diffusion model forms a noisy image from a clean image and Gaussian noise, while sampling can use iterative denoising or an ODE solver to produce a clean image.The section denotes the clean image as x0, the noisy image at time t as xt, and random noise as x1.Source: §2 · p. 3
  • Rectified flows linearly interpolate between noise and clean data and learn a velocity field to transport between their distributions.The section describes trajectory distillation as constructing advanced trajectories within noise–image pairs to accelerate sampling without compromising image quality.Source: §2 · p. 3
  • Reflow trains a new velocity to approximate the straight direction from noise to its paired clean image, which may allow fewer steps for high-quality generation.The teacher trajectory may not be straight because its training noise and clean-image examples are unpaired.Source: §2 · p. 3
  • Consistency models train a projection that maps noisy images at any time to the clean image, while CTM introduces a multi-step consistent mapping function.CTM leaves its neural-network component unconstrained and specifies a boundary condition for the mapping.Source: §2 · p. 3
  • MeanFlow and FlowMap are described as concurrent methods with the same unifying target, but the text says it is unclear whether each achieves straight and consistent trajectories in practice.MeanFlow uses a correction from the starting-step gradient, whereas SCoT uses the terminating-step gradient.Source: §2 · p. 3

3 Straight-Consistent Trajectories (SCoT) model

SCoT learns a trajectory-based projection from noisy inputs to future points, combining straightness and consistency objectives. Its direct trajectory mapping is designed to retain these properties while avoiding the ODE approximation errors associated with velocity-based sampling.

3 Straight-Consistent Trajectories (SCoT) model

  • The projection Gϕ(xt, t, s) takes the current point and time plus a future time, and outputs the point at that future time.The output is written as bxs := Gϕ(xt, t, s).Source: §3 · p. 3

3.3 Connections to Other Distillation Methods

  • SCoT’s velocity loss shares Reflow’s target, but SCoT directly learns the trajectory through Gϕ and avoids ODE approximation errors during sampling.CTM uses a similar projection format but may require multiple steps when the student trajectory has high curvature.Source: §3.1 · p. 4, §3.3 · p. 5

3.1 Main Components of SCoT

  • SCoT combines a velocity regulator for trajectory straightness with a consistency regulator that maps points along a trajectory to the same future value.The velocity target follows changes in a pre-trained noise-image pair; consistency requires agreement across points at different times.Source: §3.1 · pp. 4–5

3.2 Training from scratch and Sampling

  • Dynamic generation of bx0 with the pre-trained model avoids pre-computed pairs, while training from scratch was not explored due to resource constraints.A shared noise vector for both losses is proposed as a possible improvement, but its effect is left for future study; sampling may be multi-step or single-step.Source: §3.1 · p. 5, §3.2 · p. 5

3.3 Connections to Other Distillation Methods

  • Table 1 contrasts straight-trajectory objectives: InstaFlow learns change magnitude from random noise x1, whereas FlowDreamer approximates the teacher velocity.The caption identifies the table’s scope as objectives for enforcing straight student trajectories.Source: Table 1 · p. 5
  • Unlike SCoT, Shortcut still uses an ODE solver for sampling; Boot aligns velocity, while InstaFlow and SlimFlow focus on reconstruction without velocity and consistency regularization.The paper notes that Shortcut’s one-step Euler solver for consistency might not be optimal; it also states that the unregularized trajectories of Boot, InstaFlow, and SlimFlow are not well defined.Source: §3.3 · pp. 5–6

4 Experiments

The experiments evaluate SCoT on CIFAR-10 and ImageNet, compare its generation quality with diffusion baselines, and examine consistency, velocity, and loss-weighting choices. Results report competitive low-NFE generation and gains from the consistency and velocity losses, while normalized velocity weighting performs best across training iterations.

4.1 Experimental Setup

  • Evaluation uses CIFAR-10 and ImageNet, with EDM2 and CM3 teachers and DDPM++ and ADM students, respectively.CIFAR-10 uses 32 × 32 images and ImageNet 64 × 64 images; the models train for 130k and 40k iterations, respectively.Source: §4.1 · pp. 6–7

4.2 Distillation Results

  • SCoT reaches FID 2.30 on CIFAR-10 and 2.60 on ImageNet with 2 NFEs, outperforming CD on both; SlimFlow reports FID 4.17.On ImageNet, SCoT closely approaches EDM at FID 2.44; the passage notes that CTM's stronger FID results incorporate GAN loss, which SCoT does not use.Source: Table 2 · p. 7, §4.2 · pp. 7–8, Table 3 · p. 8

4.3 Trajectory Analysis

  • Soft consistency distills a randomly determined portion of the teacher interval, with µ = 0 representing the full interval, and is reported as efficient and high-performing.The authors report that it outperforms local consistency and performs comparably to global consistency.Source: §4.3 · p. 8

4.4 Consistency Guarantee

  • FID falls from 15.7 to 5.6 at NFE=1 and from 16.4 to 3.9 at NFE=2 after adding the consistency loss.At NFE=1, IS rises from 30.4 to 63.1; at NFE=2, it rises from 29.8 to 61.4. Precision and Recall also increase at NFE=1, from 0.49 to 0.72 and 0.45 to 0.57, respectively.Source: §4.4 · p. 8

4.5 Velocity Guarantee

  • FID falls from 14.7 to 4.8 for Step=1 (NFE=2), while IS reaches 67.6 and 68.2 for Step=1 (NFE=1) and Step=2 (NFE=2).For Step=1 (NFE=1), Precision improves from 0.56 to 0.71 and Recall from 0.54 to 0.58.Source: §4.5 · pp. 8–9

4.6 Loss Weighting

  • Normalized Velocity Loss, which clips adaptive weights to [0.01, 10], achieves the best FID across all reported iterations.Adaptive Weighting performs relatively well early but worsens later; Without Scaling is slightly worse early but achieves improved stability and FID later.Source: Table 6 · p. 10, §4.6 · p. 10

5 Conclusion

SCoT produces straight and consistent trajectories for fewer-step generation, and its experiments on CIFAR-10 and ImageNet report improved efficiency and generation quality. The theoretical motivation assigns trajectory straightness and cross-time consistency distinct, complementary roles.

5 Conclusion

  • SCoT outperforms existing distillation methods in efficiency and generation quality on CIFAR-10 and ImageNet, with the stated goal of high-quality generation in fewer steps.The conclusion describes compact models generating data in N steps or even a single step.Source: §5 · p. 10
  • The conclusion identifies high-resolution image synthesis and conditional generation as future directions.Source: §5 · p. 10

A Theoretical Motivation for Loss Design

  • The projection function combines a straight-path term with a neural residual correction that provides flexibility.The residual correction is learned by gϕ.Source: A Theoretical Motivation for Loss Design · p. 15
  • The velocity loss reduces trajectory curvature for fewer-step approximation, while the consistency loss keeps the mapping valid across time.Together, the objectives are intended to improve generation quality and efficiency, especially for 1-step or 2-step sampling.Source: A Theoretical Motivation for Loss Design · p. 15

B Related Work

The related work groups diffusion distillation by its objective: matching outputs, distilling trajectories, or refining output distributions adversarially. Trajectory methods include consistency-based and rectified-flow approaches.

B Related Work

  • Output-reconstruction methods align teacher and student outputs or distributions, including approaches for one-step denoising and Fisher-divergence alignment.These methods aim to reduce sampling steps while maintaining output fidelity.Source: B Related Work · p. 15
  • Trajectory distillation matches the denoising path: consistency methods emphasize stability across timesteps, while rectified flows seek straighter paths.The passage also notes work integrating consistency modeling into rectified flows.Source: B Related Work · pp. 15–16
  • Adversarial distillation uses a classifier or discriminator loss to bring the student output distribution closer to the teacher's, and can complement other strategies.The passage reports competitive performance with fewer sampling steps for cited studies.Source: B Related Work · p. 16

C.1 Datasets

The experiments use CIFAR-10 and ImageNet datasets. The supplied dataset descriptions give their image counts, resolutions, and class counts.

C.1 Datasets

  • CIFAR-10 contains 60,000 32×32 color images across 10 classes, split into 50,000 training and 10,000 test images.Source: §C.1 · p. 16
  • ImageNet contains over 1.2 million training images and around 50,000 validation images across 1,000 classes.Source: §C.1 · p. 16
  • The architectural comparison covers DDPM++ for CIFAR-10 and ADM for ImageNet, detailing U-Net differences in normalization, attention, conditioning, and up/downsampling.Source: Table 9 · p. 17

C.2 Metrics

The metrics assess generation quality and computational cost. They include distributional and manifold-based image metrics, plus model-size and operation-count measures.

C.2 Metrics

  • FID compares real and generated feature distributions as Gaussians in InceptionV3 space, capturing fidelity and diversity; clean-fid standardizes preprocessing and activations.Source: §C.2 · p. 17
  • sFID approximates FID using one-dimensional projections of InceptionV3 feature embeddings for a more computationally efficient quality measure.Source: §C.2 · p. 17
  • Precision measures generated samples within the real-data manifold, while recall measures coverage of the real-data distribution by generated samples.The manifolds are approximated using k-nearest neighbors in InceptionV3 feature space.Source: §C.2 · pp. 17–18
  • Inception Score reflects sample quality and class diversity using the KL divergence between conditional and marginal class distributions.For datasets with limited categorical variation, the passage says IS primarily reflects fidelity.Source: §C.2 · p. 18
  • Parameter counts measure learnable model size, while MACs and FLOPs indicate computational cost; FLOPs count additions and multiplications in a forward pass.The supplied FLOPs expression is 2 × H × W × k^2 × C_in × C_out; MACs are described as a hardware-aware inference-cost indicator.Source: §C.2 · p. 18

D.1 Time Efficiency

Table 11 compares SCoT training throughput under different loss functions across datasets, reporting images per second on an H100 GPU.

D.1 Time Efficiency

  • Table 11 compares SCoT throughput across different loss functions and datasets.The comparison is reported in images per second on an H100 GPU.Source: Table 11 · p. 18, §D.1 · p. 18

D.2 Sample Comparisons

The ImageNet samples show progressive refinement in one-step SCoT generation as training advances, from blurry images to high-quality, semantically consistent images.

D.2 Sample Comparisons

  • At the initial stage, generated images are blurry and have limited semantic structure, while 10k-step samples show clearer boundaries and texture.Figure 3 depicts the initial stage; the text describes the 10k-step progression.Source: §D.2 · p. 18, Figure 3 · p. 19
  • At 30k training steps, SCoT produces high-quality, semantically consistent one-step ImageNet samples.The samples are described as demonstrating refinement during training.Source: §D.2 · pp. 18–19
  • The comparison uses 6,000 samples at each training stage and trains SCoT with the DSM loss adopted from CTM (Kim et al., 2024).The stated purpose of this loss is to enhance sample quality and training stability.Source: §D.2 · p. 19

E Parameterization of SCoT

The parameterization of Gϕ is formulated to satisfy its boundary condition, while leaving gϕ unconstrained for representation by a neural network.

E Parameterization of SCoT

Related papers

Loading 2502.16972v4…