PaperlayerView exact revision on arXiv

Source-linked AI summary

Score-based Generative Modeling in Latent Space

Arash Vahdat, Karsten Kreis, Jan Kautz

arXiv:2106.05931v3stat.MLcs.LG

TL;DR

Score-based generative models can produce high-quality samples but are costly to sample and are not readily applicable to non-continuous data. LSGM moves score modeling into a VAE latent space, achieving strong image-generation results with substantially faster sampling, including a CIFAR-10 FID of 2.10.

  • Problem

    Score-based generative models require thousands of neural-network evaluations for sampling and are not readily applicable to binary, categorical, or graph-structured data.

  • Method

    LSGM trains a score-based generative model over VAE latent embeddings, using a mixed Normal-neural score prior and variance-reduced objectives for end-to-end training.

  • Results

    LSGM delivers strong image-generation performance while reducing sampling cost, achieving a CIFAR-10 FID of 2.10 and sampling 637× faster than the original SGM’s PC method.

  • Takeaways & Limitations

    Latent score modeling supports expressive generative models, non-continuous data modeling, and faster sampling through smoother score models in a smaller space.

  • Takeaways & Limitations

    The sampling comparison is qualified because only a VESDE model was released at submission time, and ODE sampling can yield worse FID for the original SGM.

Abstract

from arXiv · show

Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution coverage. However, they are usually applied directly in data space and often require thousands of network evaluations for sampling. Here, we propose the Latent Score-based Generative Model (LSGM), a novel approach that trains SGMs in a latent space, relying on the variational autoencoder framework. Moving from data to latent space allows us to train more expressive generative models, apply SGMs to non-continuous data, and learn smoother SGMs in a smaller space, resulting in fewer network evaluations and faster sampling. To enable training LSGMs end-to-end in a scalable and stable manner, we (i) introduce a new score-matching objective suitable to the LSGM setting, (ii) propose a novel parameterization of the score function that allows SGM to focus on the mismatch of the target distribution with respect to a simple Normal one, and (iii) analytically derive multiple techniques for variance reduction of the training objective. LSGM obtains a state-of-the-art FID score of 2.10 on CIFAR-10, outperforming all existing generative results on this dataset. On CelebA-HQ-256, LSGM is on a par with previous SGMs in sample quality while outperforming them in sampling time by two orders of magnitude. In modeling binary images, LSGM achieves state-of-the-art likelihood on the binarized OMNIGLOT dataset. Our project page and code can be found at https://nvlabs.github.io/LSGM .

1 Introduction

The introduction presents LSGM, which moves score-based generative modeling into a VAE latent space to address expensive sampling and limitations with non-continuous data. It also describes end-to-end training and a denoising score-matching objective for jointly learning the latent model and its target distribution.

  • Motivation: SGMs achieve high sample quality and likelihood, but sampling requires thousands of neural network evaluations because generation solves a complex SDE or probability-flow ODE.Accurate numerical integration requires very small step sizes.
  • LSGM: LSGM maps data to latent space, diffuses and denoises there, then decodes generated latent samples back into data space.The model is trained end-to-end.
  • Motivation: SDE-based generative models are currently defined for continuous data and cannot be applied effortlessly to binary, categorical, or graph-structured data.
  • LSGM: LSGM learns the score-based generative model in latent space using a variational autoencoder framework, modeling the distribution over data embeddings.Synthesis generates embeddings from a simple base distribution and then maps them back to data space with a decoder.
  • Technical contributions: LSGMs can be trained end-to-end by maximizing the variational lower bound on data likelihood, despite jointly learning the score-based denoising model and its latent target distribution.The authors derive a new denoising score-matching objective to efficiently learn the VAE model and latent score-based model together.

2 Background

The background introduces continuous-time score-based generative models through a forward Itô diffusion designed to end in a standard Normal distribution. Generation reverses this diffusion using the marginal score, which is learned with denoising score matching and can relate to approximate maximum likelihood.

  • Forward diffusion: The forward diffusion follows an Itô SDE with scalar drift f(t), diffusion g(t), and Wiener noise, designed so z1 follows a standard Normal distribution.The process is defined over t ∈ [0, 1], with z0 as the starting variable and zt its perturbation.
  • Reverse-time generation: Sampling starts from z1 ∼ N(z1; 0, I) and integrates a reverse-time SDE involving the marginal score ∇zt log qt(zt).The reverse process uses drift f(t)z−g(t)^2∇z log qt(z) and diffusion g(t) with infinitesimal negative time steps.
  • Score learning: Because the marginal score is unavailable analytically, training uses denoising score matching on a parametric score function at times sampled uniformly from [0, 1].The objective uses a weighting coefficient λ(t) and the diffusion kernel q(zt|z0), which is closed form for certain drift and diffusion choices.
  • Likelihood connection: For λ(t) = g(t)^2/2, score-matching minimization corresponds to approximate maximum-likelihood training based on an upper bound on KL divergence.The KL divergence is between the target distribution and the distribution induced by the reverse-time SDE with the learned score.

3 Score-based Generative Modeling in Latent Space

LSGM combines a VAE with a latent-space SGM prior, diffusing encoder samples toward a standard Normal distribution and generating data through the reverse SDE and decoder. Its training uses a cross-entropy score-matching objective, a mixed score parameterization, alternative weighting mechanisms, and variance-reduction techniques to make optimization feasible and sampling efficient.

  • Framework: LSGM combines encoder qφ(z0|x), SGM prior pθ(z0), and decoder pψ(x|z0) in latent space.The prior diffuses encoder samples toward p(z1) = N(z1; 0, I), while generation reverses the SDE and decodes synthesized z0.
  • Training objective: The variational objective decomposes into reconstruction, negative encoder entropy, and cross-entropy terms, with the cross-entropy term involving the SGM prior posing the main training challenge.Reconstruction and entropy are readily estimated with an explicit reparameterizable encoder; the cross-entropy term is addressed separately.
  • Cross-entropy score matching: The proposed cross-entropy score-matching formulation avoids unavailable marginal-score terms and supports complex encoder distributions, including hierarchical and normalizing-flow constructions.Direct KL training is infeasible because ∇zt log q(zt) is unavailable analytically, and denoising score matching does not remove the problem in the conventional formulation.
  • Prior parameterization: The SGM prior uses a geometric mixture of a standard Normal and a trainable prior, with learnable mixing coefficient α, bringing latent distributions closer to a simple target before end-to-end training.Pretraining with α=0 corresponds to a VAE with a standard Normal prior and simplifies the distribution learned by the SGM prior.
  • Weighting and variance reduction: Three weighting mechanisms train the prior: wll(t) for maximum likelihood, wun(t) = 1 for the unweighted objective, and wre(t) for a reduced-weight variant.Dropping the weighting can improve sample quality at a small likelihood cost, while wre(t) facilitates variance reduction.
  • Weighting and variance reduction: Variance-reduction techniques target the high variance caused by sampling time t, and importance-sampling proposals can substantially reduce variance for the VPSDE objectives.The analysis assumes q(z0) = p(z0) = N(z0; 0, I), while the highlighted results report that techniques (ii) and (iii) significantly reduce variance and enable objective monitoring.

4 Related Work

LSGM connects denoising score matching and diffusion-based generative modeling with VAEs, extending related work on graph generation, expressive VAE priors, and latent-space diffusion. It also differs from concurrent approaches through its end-to-end training framework and analytically simplified objective.

  • Score-based generative modeling: LSGM builds on score matching, especially denoising score matching, and relates to discretized diffusion and continuous-time SDE generative models.Prior experimental work primarily focuses on image modeling.
  • Graph generation: LSGM offers a natural route to graph generation by combining score-based modeling with graph-tailored encoders and decoders.This avoids relaxing adjacency-matrix entries to continuous values as in a related graph-generation method.
  • VAE priors: As a VAE with a score-based prior, LSGM relates to VAE priors based on flows, hierarchical distributions, energy-based models, classifiers, adversarial methods, and other implicit techniques.These approaches define or improve VAE prior distributions through explicit or implicit constructions.
  • Concurrent works: Concurrent latent-space approaches include two-stage VAE and denoising-diffusion training, end-to-end training, and contrastive learning with diffusion for controllable generation.The cited symbolic-music method does not provide end-to-end training of the combined VAE and diffusion model.
  • Concurrent works: Concurrent work also studies likelihood-based training of score-based models in data space, whereas LSGM analytically marginalizes several objective terms for its diffusion SDEs.One related likelihood bound uses a denoising score-matching objective closely related to LSGM’s cross-entropy expression.

5 Experiments

Experiments show that LSGM achieves strong image-generation, binary-likelihood, and sampling-efficiency results across datasets, while its mixed-score formulation and end-to-end training improve stability and performance.

  • Implementation: LSGM uses NVAE as its VAE backbone and NCSN++ as its SGM backbone, feeding the SGM prior only the smallest-resolution latent groups.Remaining larger-resolution latent groups use standard Normal priors.
  • Unconditional color image generation: 7.22 FID: LSGM matches the original SGM on CelebA-HQ-256 while achieving state-of-the-art sample quality.This result uses wre-weighting with a linear VPSDE.
  • Modeling binarized images: ≤87.79 nat: LSGM achieves state-of-the-art likelihood on dynamically binarized OMNIGLOT, outperforming previous models including VAEs with autoregressive decoders.On MNIST, LSGM reaches a NELBO 1.09 nat lower than the state-of-the-art NVAE.
  • Sampling time: 0.07 min: LSGM samples CelebA-HQ-256 with an average of 23 NFEs and 7.22 FID, making it 637× and 56× faster than original SGM PC and ODE sampling.Original SGM requires 44.6 min. with PC and 3.91 min. with ODE-based sampling.
  • End-to-end training: 5.19 FID: End-to-end training outperforms freezing the VAE networks during the second stage, which yields 9.00 FID.The corresponding NELBO values are 2.98 for end-to-end training and 3.03 when freezing the VAE networks.
  • Mixing Normal and neural score functions: 7.60 FID: The mixed-score parametrization substantially improves the one-group LSGM over omitting it, which yields 34.71 FID and unstable end-to-end training.With the mixed score, NELBO is 3.29 versus 3.39 without it.

6 Conclusions

LSGM trains score-based generative models end-to-end in the latent space of a variational autoencoder. This shift enables more expressive models, non-continuous data modeling, and reduced sampling time through smoother SGMs.

  • LSGM is a framework for end-to-end training of score-based generative models in a variational autoencoder’s latent space.
  • Moving from data space to latent space enables more expressive generative models and supports modeling non-continuous data.
  • Smoother latent SGMs reduce sampling time compared with operating directly in data space.

7 Broader Impact · Appendix

LSGM addresses the computational cost that limits the broader use of score-based generative models while extending them beyond image generation to non-continuous data and other learning tasks.

  • 7 Broader Impact: SGMs can achieve both high sample quality and broad distribution coverage, potentially reducing generative-model bias and improving minority representation.These benefits are presented as potential consequences of better data-distribution coverage.
  • 7 Broader Impact: High sampling cost remains a barrier to the widespread use of SGMs.The broader-impact discussion identifies computational expense as a limitation despite SGMs’ quality and coverage.
  • 7 Broader Impact: LSGM is evaluated on image generation, whose potential benefits and risks have been discussed in prior work.The supplied passage refers readers to references for those benefits and risks.
  • 7 Broader Impact: LSGM is described as a generic framework that extends SGMs to non-continuous data types.This extension is stated as a property of the framework beyond its image-generation evaluation.
  • Appendix: The framework could in principle model language, music, and molecules.The passage gives these domains as examples and cites references [96] [97], [98] [10], and [100].
  • Appendix: Like other deep generative models, LSGM could potentially support semi-supervised learning and representation learning.These are identified as possible non-generative applications, with references [101] [102] [103].

A Proof for Theorem 1 … B.4.3 Variance Reduction for Reweighted Objective

The appendix proves the cross-entropy identity underlying LSGM and develops variance-reduction strategies across generic SDEs, VPSDEs, and VESDEs. It introduces mixed score parameterization and importance sampling, while showing geometric VPSDEs offer naturally low-variance objectives and VESDEs require explicit variance reduction.

  • A Proof for Theorem 1: Theorem 1 expresses cross entropy through denoising score matching under smoothness assumptions and an SDE satisfying q(z1) = p(z1).The derivation replaces the problematic marginal score with a denoising score and interprets the result as an upper bound for the learned reverse-time SGM prior.
  • A Proof for Theorem 1: A positive initial variance σ2_0 convolves the original distributions with Normal noise, so the theorem approximates their original cross entropy when σ2_0 is small.With σ2_0 = 0, the initial distribution is infinitely sharp and the integral must instead be restricted to t ∈[ϵ, 1].
  • B Variance Reduction: Variance analysis targets the aggregate-posterior cross entropy, using a Normal approximation motivated by pretraining with a standard Normal prior.The appendix organizes the analysis around generic SDEs, VPSDEs, VESDEs, and importance sampling.
  • B.1 Generic Mixed Score Parameterization for Non-Variance Preserving SDEs: Mixed score parameterization uses the score for Normal input data plus a learned residual, with the Normal diffusion behavior adjusted for generic SDEs.The standard-Normal variance remains constant only for the VPSDE case assumed in the main text; generic SDEs behave differently.
  • B.2 Variance Reduction of Cross Entropy with Importance Sampling for Generic SDEs: For generic SDEs, the minimum-variance importance-weighting distribution is derived analytically and can be sampled by inverse transform sampling.The construction uses the monotonically increasing diffused variance and aims to keep its inverse analytically tractable.
  • B.3.1 Variance Reduction for Likelihood Weighting (Geometric VPSDE): Geometric VPSDEs choose a log-linear noise schedule so the likelihood-weighting objective has constant variance contribution for Normal input data.This combines variance-preserving diffusion with minimal objective variance and empirically performs best on likelihood or NELBO.
  • B.3.2 Variance Reduction for Likelihood Weighting (Importance Sampling); B.3.3 Variance Reduction for Unweighted Objective; B.3.4 Variance Reduction for Reweighted Objective: For VPSDE likelihood-weighted, unweighted, and reweighted objectives, importance sampling selects proposal distributions that minimize Monte Carlo variance, including the original and geometric VPSDEs.The reweighted-objective derivation does not assume a particular β(t), so it applies to any VPSDE.
  • B.4 VESDE; B.4.1 Variance Reduction for Likelihood Weighting; B.4.2 Variance Reduction for Unweighted Objective; B.4.3 Variance Reduction for Reweighted Objective: VESDE objectives lack geometric VPSDEs’ naturally minimal variance, but importance-sampling proposals and inverse CDFs are derived for likelihood-weighted, unweighted, and reweighted objectives.VESDE experiments were unstable because σ2_max depends on changing latent encodings, despite the availability of these variance-reduction formulas.

B.5 Sub-VPSDE … E.1 Converting NVAE Prior to Standard Normal Prior

The paper specifies Sub-VPSDE behavior, transition-kernel assumptions, probability-flow ODE sampling and likelihood evaluation, and change-of-variables constructions that convert hierarchical VAE and NVAE priors to standard Normal priors.

  • B.5 Sub-VPSDE: Sub-VPSDE uses the same linear β(t) as regular VPSDE while slightly reducing the diffusion variance σ2_t for small t.Importance-sampling derivations are more complicated, but regular-VPSDE importance sampling may still substantially reduce objective variance.
  • C Expressions for the Normal Transition Kernel: The normal transition kernels distinguish diffusion processes by their initial and terminal variance settings, including σ2_min > 0, data-scaled VESDE σ2_max, and Geometric VPSDE σ2_max ≈1.VPSDE can set σ2_0 = 0, exactly modeling the data at t = 0, but this restricts training-time t sampling and probability-flow ODE integration.
  • D Probability Flow ODE: LSGM uses black-box ODE solvers to solve the probability flow ODE for latent-space SGM sampling and NELBO estimation.The paper states that this approach applies across all considered diffusion processes and SDEs.
  • D Probability Flow ODE: The probability flow ODE generates the same marginal distributions as the corresponding SDE when operating with the same prior and exact score function.With a learned approximate score, the ODE and SDE are formally not exactly equivalent, although they often perform similarly in practice.
  • D Probability Flow ODE: Sampling draws from the standard Normal prior at t = 1 and solves the probability flow ODE toward t = 0.The same ODE supports probability calculation through the instantaneous change-of-variables formula.
  • E Converting VAE with Hierarchical Normal Prior to Standard Normal Prior: A hierarchical VAE prior can be converted to a standard Normal prior by reparameterizing each latent variable as ϵ_l = (z_l−µ_l(z<l))/σ_l(z<l).The equivalent parameterization uses z_l = µ_l(z<l) + σ_l(z<l)ϵ_l, with ϵ_l assigned a standard Normal prior.
  • E.1 Converting NVAE Prior to Standard Normal Prior: NVAE retains the hierarchical prior form, while its residual encoder parameterization predicts residual mean and variance parameters.After reparameterization, the encoder directly predicts the mean and variance for the ϵ_l distributions.

F Bias in Importance Weighted Estimation of Log-Likelihood … G.5 Ablation Experiments

The paper analyzes bias in importance-weighted likelihood estimation and details the VAE, latent SGM, training, evaluation, and ablation setups used for LSGM. It finds that variance in the prior-likelihood estimator can bias importance-weighted estimates, while NELBO remains unbiased with small variance.

  • F Bias in Importance Weighted Estimation of Log-Likelihood: For K > 1, importance-weighted estimation can overestimate log-likelihood when log p(z) uses an unbiased estimator with variance σ2.The analysis assumes σ2 is small and uses a second-order Taylor expansion.
  • F Bias in Importance Weighted Estimation of Log-Likelihood: The resulting bias is O(σ2) and can be minimized by making the estimator variance sufficiently small.When the relevant estimates have sufficiently small variance, the bias is proportional to that variance.
  • F Bias in Importance Weighted Estimation of Log-Likelihood: In experiments, log p(z0) variance is too large for reliable test-likelihood estimation with the importance-weighted bound.Using many Hutchinson noise vectors reduces variance but makes NLL estimation computationally too expensive; NELBO with K = 1 is unbiased and has small variance through averaging.
  • G Additional Implementation Details: All hyperparameters for the main models are provided in Tab. 7.The additional implementation details cover the VAE backbone, latent SGM prior, training, evaluation, and ablation experiments.
  • G.1 VAE Backbone: All LSGM models use NVAE as their VAE backbone, with a hierarchical latent space and group-wise autoregressive latent-variable dependencies.The backbone also uses residual neural networks and depth-wise separable decoder convolutions.
  • G.1 VAE Backbone: For discrete images, LSGM uses mixtures of discretized logistics for color images and Bernoulli distributions for binary images.These decoder distributions avoid the uniform dequantization relied on by SGMs for continuous data and are available in NVAE.
  • G.2 Latent SGM Prior: The latent SGM prior uses NCSN++-based denoising networks adapted to the VAE latent dimensions, with hyperparameters including residual-cell and convolution-channel counts.All models use 0.2 dropout in the SGM prior; some also use anti-aliased FIR upsampling and downsampling.
  • G.3 Training Details: Training uses Adamax for the VAE’s encoder and decoder, then Adam for end-to-end second-stage optimization, with learning-rate annealing for the VAE and a constant rate for the SGM prior.At test time, the SGM prior uses an exponential moving average of its parameters.

G.5.1 Ablation: SDEs, Objective Weighting Mechanisms and Variance Reduction … H Additional Experiments

The supplementary experiments validate key LSGM design choices, specify alternative training procedures, and report the computational resources used. The ablations emphasize importance sampling, weighting mechanisms, the VPSDE, end-to-end training, and mixed score parametrization.

  • G.5.1 Ablation: SDEs, Objective Weighting Mechanisms and Variance Reduction: Importance sampling stabilizes training, while wll-weighting and the geometric VPSDE suit strong likelihood objectives; wun- and wre-weighting tend to improve FIDs.These trends do not translate perfectly to the large models used for the main results.
  • G.5.1 Ablation: SDEs, Objective Weighting Mechanisms and Variance Reduction: CIFAR10’s best FID uses the VPSDE, wun-weighting, importance sampling, and sample reweighting for the q-objective.For smaller ablation models, sample reweighting and a separate batch t drawn with rll(t) perform equivalently in this case.
  • G.5.2 Ablation: End-to-End Training: The end-to-end versus separate-training ablation uses the VPSDE, wun-objective weighting, and importance sampling with run(t) for the SGM prior.For end-to-end training, a second batch t drawn with rll(t) trains q.
  • G.5.3 Ablation: Mixing Normal and Neural Score Functions: Larger backbone VAEs were generally unstable without the mixed Normal and neural score parametrization.The corresponding ablation uses a smaller VAE with one latent variable group and no hierarchical latent dependencies.
  • G.6 Training Algorithms: Likelihood training with importance sampling uses a shared weighted likelihood objective, so the SGM prior and encoder need not be updated separately.Algorithm 1 updates all parameters by minimizing the combined objective.
  • G.6 Training Algorithms: Unweighted or reweighted training can use separate importance-sampled t batches for the SGM and q objectives, updating the SGM prior and VAE parameters with different weightings.Alternatively, importance sampling for the SGM objective can be combined with reweighting for the q-objective; the latter shares denoising computation but generally uses more memory.

H.1 Additional Samples

This section provides additional samples generated by the models for CIFAR-10 and CelebA-256-HQ, alongside supplementary experiments on binarized MNIST and sampling efficiency for CIFAR-10 models.

  • Additional Samples: Additional model-generated samples are shown for CIFAR-10 and CelebA-256-HQ.The samples appear in Fig. 7 and Fig. 8, respectively.
  • Additional Samples: A small VAE architecture is evaluated on dynamically binarized MNIST.The experiment is reported in Table 8.
  • Additional Samples: CIFAR-10 latent SGM prior sampling is evaluated by ODE-solver function evaluations and sampling time.Table 9 reports NFE and sampling time for the main CIFAR-10 models, averaged over 20 runs with batch size 16 on a single Titan V GPU.

H.2 MNIST: Small VAE Experiment … H.7 CelebA-HQ-256: Sampling from VAE Backbone vs. LSGM

The appendix evaluates LSGM across compact VAE settings, CIFAR-10 sampling and SDE choices, and CelebA-HQ-256 solver, sampling, and latent-resolution variants. Results highlight architecture details, dataset-dependent evaluation costs, sampling-quality tradeoffs, and faster sampling with lower latent resolution.

  • H.2 MNIST: Small VAE Experiment: The MNIST experiment uses a small non-hierarchical VAE with one group containing 64 latent variables.The encoder and decoder are small ResNets with six residual cells arranged into three blocks.
  • H.3 CIFAR-10: Neural Network Evaluations during Sampling: CIFAR-10 sampling uses an ODE solver with error tolerance 10^-5 and integration cutoff 10^-6.Its more diverse and multimodal distribution requires a more complex learned latent SGM prior and therefore more function evaluations.
  • H.4 CIFAR-10: Sub-VPSDE vs. VPSDE: The Sub-VPSDE experiment quantitatively compares it with the VPSDE under the same setup and models used for the SDE, objective-weighting, and variance-reduction ablations.Variance-reduction techniques derived for the VPSDE can also reduce variance for sample-based training-objective estimates with the Sub-VPSDE.
  • H.5 CelebA-HQ-256: Different ODE Solver Error Tolerances: CelebA-HQ-256 samples are visualized for varying ODE solver error tolerances.The paper uses the probability flow ODE for sampling from the model.
  • H.5 CelebA-HQ-256: Different ODE Solver Error Tolerances: The appendix includes additional uncurated CIFAR-10 samples generated by LSGM using probability-flow-ODE sampling in latent space.These are samples from the model with the best FID.
  • H.5 CelebA-HQ-256: Different ODE Solver Error Tolerances: Additional uncurated CelebA-HQ-256 samples are generated by LSGM with probability-flow-ODE sampling in latent space.The appendix presents these samples as supplementary qualitative results.
  • H.6 CelebA-HQ-256: Ancestral Sampling: On CelebA-HQ-256, ancestral sampling from the prior often produces higher-quality samples than probability-flow-ODE sampling, but with a slightly worse FID score.The appendix visualizes ancestral samples generated with different numbers of steps.
  • H.7 CelebA-HQ-256: Sampling from VAE Backbone vs. LSGM: The quantitative CelebA-HQ-256 model uses 32×32 latent spatial dimensions, whereas qualitative results use 64×64 dimensions.The 32×32 model achieves FID 7.22 vs. 8.53 and samples in 2.7 sec. vs. 39.9 sec., while its visual quality is slightly worse.

H.8 Evolution Samples on the ODE and SDE Reverse Generative Process

The section visualizes how latent variables evolve during reverse-time generative SDE and probability flow ODE sampling by decoding intermediate states into pixel space. ODE solver tolerance affects artifacts and subtle image details.

  • Latent evolution: Intermediate latent states are decoded into images to visualize reverse-time evolution under both the generative SDE and probability flow ODE.The visualization follows 13 stages, from t = 1.0 through t = 10−5.
  • ODE solver tolerance: High ODE solver error tolerances produce low-frequency artifacts, while reducing tolerance slightly improves subtle details.This differs from the original SGM, where high error tolerance results in pixelated images.
  • Latent evolution: 13 decoded images per row show denoising stages at t = [1.0, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.05, 0.01, 10−5].These stages span the reverse denoising diffusion process.
Loading 2106.05931v3…