PaperlayerView exact revision on arXiv

Source-linked AI summary

Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models

Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, Zhihua Zhang

arXiv:2305.18455v2cs.LGcs.CV

TL;DR

The paper asks how knowledge from pre-trained diffusion models can be transferred to other generative models without original training data, despite supervision challenges for implicit generators. It proposes Diff-Instruct, which uses Integral Kullback-Leibler divergence to instruct differentiable generators, and reports state-of-the-art single-step diffusion models plus consistent GAN-generator improvements. These results position Diff-Instruct as a data-free paradigm for transferring diffusion knowledge across generative-model classes.

  • Problem

    Transferring knowledge from pre-trained diffusion models to implicit generators without original data is challenging because implicit models lack explicit diffusion score information.

  • Method

    Diff-Instruct instructs differentiable generative models by minimizing Integral Kullback-Leibler divergence along a diffusion process.

  • Results

    Diff-Instruct achieves state-of-the-art single-step diffusion-based models and consistently improves pre-trained GAN generators across various settings.

  • Takeaways & Limitations

    Diff-Instruct enables data-free knowledge transfer from pre-trained diffusion models into generic generators.

  • Takeaways & Limitations

    The work does not investigate using multiple instructor models or combining Diff-Instruct with real training data.

Abstract

from arXiv · show

Due to the ease of training, ability to scale, and high sample quality, diffusion models (DMs) have become the preferred option for generative modeling, with numerous pre-trained models available for a wide variety of datasets. Containing intricate information about data distributions, pre-trained DMs are valuable assets for downstream applications. In this work, we consider learning from pre-trained DMs and transferring their knowledge to other generative models in a data-free fashion. Specifically, we propose a general framework called Diff-Instruct to instruct the training of arbitrary generative models as long as the generated samples are differentiable with respect to the model parameters. Our proposed Diff-Instruct is built on a rigorous mathematical foundation where the instruction process directly corresponds to minimizing a novel divergence we call Integral Kullback-Leibler (IKL) divergence. IKL is tailored for DMs by calculating the integral of the KL divergence along a diffusion process, which we show to be more robust in comparing distributions with misaligned supports. We also reveal non-trivial connections of our method to existing works such as DreamFusion, and generative adversarial training. To demonstrate the effectiveness and universality of Diff-Instruct, we consider two scenarios: distilling pre-trained diffusion models and refining existing GAN models. The experiments on distilling pre-trained diffusion models show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. The experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models across various settings.

1 Introduction

Diff-Instruct addresses the challenge of transferring knowledge from pre-trained diffusion models to other generative models without original training data. It provides a universal, mathematically grounded framework and demonstrates strong results for diffusion distillation and GAN improvement.

  • Motivation: Pre-trained models can provide useful knowledge when large amounts of high-quality training data are difficult to access.The paper frames model-driven learning as an efficient way to support new tasks with a base model that can be further improved.
  • Motivation: Knowledge transfer from diffusion models to implicit generators is difficult because implicit models lack the explicit multi-level score information used by diffusion models.Implicit generators are valued for flexibility and efficient generation, but receiving supervision from diffusion score networks is technically challenging.
  • Method: Diff-Instruct instructs arbitrary implicit generative models whose generated samples are differentiable with respect to their parameters.The framework can operate across model types, including GAN generators and fixed-time U-Nets for single-step diffusion generation.
  • Method: Diff-Instruct directly minimizes Integral Kullback-Leibler divergence and connects to DreamFusion and generative adversarial training.The paper also identifies the SDS objective as a special case when the generator outputs a Dirac’s Delta distribution.
  • Results: Diff-Instruct achieves state-of-the-art single-step diffusion-based models on ImageNet 64×64 and consistently improves pre-trained GAN generators across settings.The diffusion results span diffusion distillation and direct training baselines.

2 Preliminary

The preliminary section contrasts explicit diffusion models with efficient but harder-to-train implicit models. It introduces their operating principles and motivates transferring diffusion-model knowledge to generators and differentiable renderers.

  • Diffusion models: Diffusion models transform data distributions toward simple noise through a forward diffusion process and learn marginal score functions.Samples are generated by simulating a stochastic differential equation using the learned score network.
  • Diffusion models: SDE simulation makes diffusion-model sampling significantly slower than sampling from implicit models.This speed difference motivates interest in transferring diffusion knowledge to more efficient generators.
  • Generative adversarial networks: GANs map easy-to-sample latent vectors to data-space samples, enabling efficient generation but making training challenging through adversarial optimization.A discriminator distinguishes real and generated samples, while generator and discriminator objectives are optimized alternately.
  • Generative adversarial networks: Certain GAN objectives correspond to minimizing divergences between generated and data distributions, including KL divergence.The paper later establishes an equivalence between KL-minimizing adversarial training and Diff-Instruct.
  • Neural radiance fields: Neural radiance fields map mesh coordinates to volume properties, and differentiable rendering permits parameter updates from rendered two-dimensional images.This illustrates how differentiable generated outputs can receive external instruction.

3 Diff-Instruct

Diff-Instruct transfers knowledge from pre-trained diffusion models to differentiable implicit generators without training data. It minimizes an Integral KL divergence across diffusion time levels to update the generator, supporting distillation and GAN refinement.

  • Optimization: The Diff-Instruct gradient updates generator parameters by integrating score-based instructions over diffusion time levels.The pipeline applies the resulting gradient to the generator, while the diffusion model acts as the teacher.
  • Instruction criterion: IKL integrates KL divergence across diffusion time levels, combining multi-level knowledge from the pre-trained diffusion model.The divergence is defined using marginal densities from the same diffusion process and is nonnegative, with equality only when the distributions match almost everywhere.
  • Instruction criterion: IKL remains finite for some distributions with disjoint supports where vanilla KL divergence is infinite.This robustness motivates using diffusion processes to compare distributions whose supports are misaligned.
  • Framework: Diff-Instruct trains an implicit generator to match a pre-trained diffusion model’s distribution without using training data.The generator samples are differentiated with respect to model parameters, and the pre-trained model supplies distributional knowledge through marginal score functions.
  • Connections: For a generator producing a Dirac distribution, Diff-Instruct reduces to the score distillation sampling gradient used in DreamFusion.Under this assumption, no separate diffusion model is needed to estimate the generator’s marginal scores.
  • Connections: Unlike adversarial training, Diff-Instruct uses a pre-trained diffusion model instead of a discriminator and can distill knowledge without real or synthetic data.The paper also contrasts IKL with KL-based degeneration and discusses potential advantages regarding mode-drop issues.
  • Universality: Diff-Instruct supports diverse generator architectures, including CNN-, Transformer-, and U-Net-based generators, as well as GAN generators.Its experiments apply knowledge transfer to both U-Net-based and GAN generators.

4 Experiments

Experiments evaluate Diff-Instruct for data-free diffusion distillation and GAN improvement. It achieves competitive or state-of-the-art generation quality, faster convergence, and consistently improves pre-trained StyleGAN-2 generators.

  • 4.1 Single-step diffusion distillation: Diff-Instruct distills pre-trained EDM models into single-step generators on CIFAR10 and ImageNet 64×64, evaluating FID and IS.FID is lower-is-better, while IS is higher-is-better.
  • 4.1 Single-step diffusion distillation: On ImageNet 64×64, Diff-Instruct outperforms diffusion-based single-step models in FID, including methods using real data, synthetic data, or training from scratch.The comparison includes both diffusion distillation and direct-training methods.
  • 4.1 Single-step diffusion distillation: Without synthetic or real datasets, Diff-Instruct achieves competitive distillation performance across FFHQ-64, ImageNet-64, and CIFAR10.Figure 2 presents non-cherry-picked samples from one-step generators on these datasets.
  • 4.1 Single-step diffusion distillation: 1 NFE yields FID 4.19 versus 15.56 for the teacher at 10 NFEs, corresponding to at least 10+ times acceleration.The student uses the same UNet architecture as the teacher, so NFE represents inference-time cost.
  • 4.1 Single-step diffusion distillation: Diff-Instruct FID converges within 7k iterations, whereas consistency distillation does not converge in fewer than 7k iterations.The authors suggest that the one-step student avoids learning across multiple time indexes.
  • 4.2 Improving generative adversarial networks: Diff-Instruct consistently improves pre-trained StyleGAN-2 generators, reducing conditional FID from 2.42 to 2.27 and unconditional FID from 2.92 to 2.71.The experiments use pre-trained EDM models as instructors and StyleGAN-2 models initialized after adversarial training.

5 Discussion

Diff-Instruct is presented as a data-free paradigm for transferring knowledge from pre-trained diffusion models into generic generators. The discussion identifies unexplored extensions involving multiple instructors, real data, and teacher training from data alone.

  • Diff-Instruct enables data-free knowledge transfer from pre-trained diffusion models into generic generators.
  • Using multiple pre-trained diffusion models as instructors remains an uninvestigated direction.
  • The benefits of combining Diff-Instruct with real training data have not yet been explored.
  • With data but no pre-trained diffusion model, the framework could be adapted by training a teacher diffusion model from data while instructing the student.

A.1 Robustness of Integral KL divergence

This section argues that Integral KL divergence is more suitable than traditional KL divergence for gradient-based learning when distributions have misaligned supports. With suitable weighting, IKL becomes finite, differentiable, and reliably minimized at the correct parameter.

  • IKL is more robust to misaligned density support than traditional KL divergence.
  • IKL remains finite and reliable for gradient-based training where traditional KL divergence fails to guide the generator to θ = 0.
  • For the example distributions Pθ and P0, a suitable weighting makes IKL a differentiable quadratic with its sole minimum at θ = 0.
  • The appendix compares IKL with KL divergence and Wasserstein distance, concluding that IKL better supports gradient-based optimization of θ.
  • In practice, Diff-Instruct uses an inverted U-shaped weighting function also used for training the pre-trained diffusion models.

A.2 Proof of Theorem 3.3

The proof constructs the generator-induced distribution through the forward diffusion process and defines the teacher distribution using pre-trained diffusion-model scores. Under stated regularity assumptions, differentiation can pass through the spatial integral.

  • The generator distribution q(t) is represented by sampling z and Wiener-process trajectories, then evolving x0 = gθ(z) through the forward SDE.
  • The teacher distribution p(t) is defined through the pre-trained diffusion model’s score functions sp(t).
  • The proof decomposes the IKL derivative into terms and shows that term B vanishes.
  • Differentiating under the integral requires existence of the parameter derivative and an integrable dominating function.
  • Common forward diffusion processes instantiate the trajectory with Gaussian noise as xt = α(t)x0 + β(t)ϵ.

A.3 Proof of Corollary 3.4

The proof connects Diff-Instruct’s gradient to score distillation sampling when a differentiable generator outputs a single datum. With Gaussian forward diffusion and an ϵ-network, the resulting expression recovers DreamFusion’s SDS gradient.

  • When the generator distribution is a Dirac delta, its conditional and marginal distributions coincide, simplifying the score calculation.
  • The general Diff-Instruct gradient reduces to the SDS gradient when the generator outputs a single differentiable datum.
  • Using xt = α(t)x0 + σ(t)ϵ and ϵp(x,t) := −sp(t)(xt)/σ(t), the objective recovers DreamFusion’s SDS gradient estimation.
  • A fixed-view NeRF can be interpreted as a generator producing a Dirac delta distribution, linking DreamFusion to approximated Diff-Instruct distillation.

A.4 Proof of Corollary 3.5

The proof shows that, under an optimal discriminator, KL-minimizing adversarial training has the same gradient form as Diff-Instruct with a specific weight function.

  • With an optimal discriminator, adversarial training minimizes the KL divergence between the generator and data distributions.
  • The resulting gradient uses the difference between data and generator score functions.The score functions are defined as sd(x)=∇xlogpd(x) and sg(x)=∇xlogpg(x).
  • This gradient is equivalent to Diff-Instruct with w(0)=1 and w(t)>0 for all t>0.The equivalence depends on the discriminator being trained to optimality.

A.4.1 Comparison of Distillation Methods

Table 6 compares diffusion distillation methods by training efficiency and flexibility, including whether the student must match the teacher's input-output dimensions.

  • Table 6 compares diffusion distillation methods using efficiency and flexibility as evaluation dimensions.
  • Efficiency denotes the training efficiency of diffusion models.
  • Flexibility denotes whether the student model must have the same input-output dimensions.

B More on experiments

The experiments use pre-trained EDMs as instructors for transferring knowledge to implicit generators, including UNet and StyleGAN models, with evaluation procedures aligned to established codebases.

  • Experimental setup: EDMs are selected as instructors because they achieve state-of-the-art generative performance on benchmarks including CIFAR10 and ImageNet 64×64.
  • Hyperparameters: Table 7 lists the hyperparameters used for Diff-Instruct diffusion distillation.
  • Visualization: Figure 4 illustrates training with fixed latent vectors across changing generator weights and shows one-step samples distilled from a class-conditional CIFAR10 EDM.
  • Diffusion process: Forward-process samples are generated by adding Gaussian noise to generator outputs as xt=x0+σ(t)ϵ.Here ϵ∼N(0,I).
  • Experimental setup: The experiments transfer EDM knowledge to implicit generative models, specifically UNet and StyleGAN generators.
  • Evaluation: FID and IS are computed using the StyleGAN2-ADA codebase, with ImageNet 64×64 preprocessing matched to EDM.

B.1 Detailed experimental settings of diffusion distillation

The detailed settings cover diffusion distillation and GAN improvement, including initialization, alternating optimization, computational costs, and representative generated-sample results.

  • Diffusion distillation: For UNet students, Diff-Instruct provides an alternative diffusion-distillation method competing with progressive and consistency distillation.
  • Initialization: The generator can be initialized from a pre-trained diffusion model's score network through a Tweedie-based data-prediction transform at fixed time t∗.
  • Initialization: The initialized generator takes a zero-mean Gaussian latent vector with variance σ^2(t∗).
  • Optimization: Diff-Instruct alternates updates to an auxiliary diffusion model and the generator, keeping the other model fixed during each update.This makes the backpropagation graph memory cost nearly that of one model.
  • Computational costs: 10.40 over 9.55 peak GPU memory is reported for Diff-Instruct versus CD, while the method incurs only minor additional memory cost.
  • Computational costs: 0.0728 seconds versus 0.0489 seconds per 1K iterations is reported for Diff-Instruct versus CD, corresponding to about 1.5 times CD's wall-clock time per iteration.
  • Results: Figure 5 reports FIDs of 2.27, 2.71, 6.62, and 7.56 for the generated samples from left to right.
  • GAN improvement: GAN-improvement experiments use StyleGAN-2 generators and pre-trained VP-architecture EDMs on CIFAR10 under conditional and unconditional settings.
Loading 2305.18455v2…