Blurring Diffusion Models

• Independent Gaussian noise is commonly added to each dimension, but this process ignores images’ multi-scale properties.Source: §1 · p. 1
• Blurring retains low frequencies over higher frequencies, making it a natural image-destruction process in deterministic or near-deterministic diffusion.Source: §1 · p. 1
• Deterministic diffusion models had much worse visual quality than standard models, and whether their specifications guarantee modeling the data distribution remained an open question.Source: §1 · pp. 1–2
• Blurring Diffusion Models combine blurring with additive Gaussian noise, yield Markov transitions and diagonal covariance in frequency space, and report higher visual quality by FID.The authors describe the quality improvement as evidenced by better FID scores.Source: §1 · p. 2

2.1 DIFFUSION MODELS

• Standard diffusion gradually destroys data by adding Gaussian noise, with the information in z_t decreasing as α_t falls and σ_t rises.Source: §2.1 · p. 2
• The denoising model replaces the unknown data point with a neural-network prediction, and sufficiently small steps with an expressive prediction model can learn the data distribution exactly.Source: §2.1 · p. 2
• The epsilon parametrization predicts noise rather than x and is reported to yield better sample quality than direct x prediction.Source: §2.1 · p. 2
• The standard diffusion objective is a continuous-time variational lower bound on log likelihood, although constant weighting w(t) = 1 has been empirically found superior for sample quality.Source: §2.1 · p. 3

2.2 INVERSE HEAT DISSIPATION

• Inverse Heat Dissipation Models use heat dissipation and fixed noise to destroy information, with the heat equation’s grid-discretized solution computed diagonally in the cosine-transform frequency domain.Source: §2.2 · p. 3
• IHDMs allow a monotonically increasing dissipation schedule τ_t that can vary independently of conceptual diffusion time t.Source: §2.2 · p. 3
• The background asks whether heat dissipation can be Markov, whether its inverse process is isotropic, and whether alternatives to predicting the previous-step mean exist.Source: §2.2 · p. 3
• The following section answers that heat dissipation can be Markov and that its generative process is non-isotropic but diagonal in frequency space.Source: §2.2 · p. 3

3 HEAT DISSIPATION AS GAUSSIAN DIFFUSION

• The frequency-space marginal and its inverse are fully factorized over scalar frequency components.The change of variables uses the orthogonal DCT, with higher frequencies multiplied by smaller values.Source: §3 · p. 4
• Heat dissipation is equivalent to Gaussian diffusion in frequency space with non-isotropic, per-frequency noise schedules.The marginal variance being shared across frequencies does not reduce generality because the signal-to-noise ratio can vary by dimension.Source: §3 · p. 4
• The equivalence to Gaussian diffusion answers affirmatively whether the heat-dissipation marginal admits corresponding Markov transitions.The section derives those transitions using the standard Gaussian-diffusion results.Source: §3 · p. 4
• With lower values of dt at higher frequencies, heat dissipation destroys information in those frequencies faster per timestep than standard diffusion.The denoising process can predict ϵt with a neural network, as in standard denoising diffusion; directly predicting μt→s is reported to yield inferior sample quality, while a scalar σ²t→s schedule loses the stated s→t correctness guarantee.Source: §3 · pp. 4–5

4 BLURRING DIFFUSION MODELS

• Blurring Diffusion Models define diffusion in frequency space, combining blurring with Gaussian noise and allowing different frequencies to diffuse at different rates.The section notes that emphasizing low frequencies may avoid overfitting to high frequencies.Source: §4 · p. 5
• The model predicts noise in pixel space, then transforms predictions to frequency space to calculate the denoising mean.Training uses an unweighted squared error in pixel space; the section notes that variational-bound objectives tend to yield inferior sample quality.Source: §4 · p. 6
• The schedules combine a variance-preserving cosine noise schedule with a blur schedule, and cap the maximum blur with a tunable parameter.The blur schedule is modified to limit amplification of small errors during repeated denoising; its frequency damping bottoms out at dmin = 0.001.Source: §4 · pp. 6–7

4.1 A NOTE ON THE GENERALITY

• In any invertible basis, a diagonal diffusion process in transformed coordinates corresponds to a pixel-space process and extends the class of diffusion models with standard-diffusion guarantees.The frequency-space construction is one instance of this general basis-change framework.Source: §4.1 · p. 7

5 RELATED WORK

• Most diffusion processes use independent isotropic Gaussian noise, while super-resolution methods address image scale through explicit transitions between resolutions.Restoration methods target predefined corruptions rather than generating data from scratch.Source: §5 · p. 7
• Non-isotropic diffusion has been studied for lossy compression, where its benefit over isotropic diffusion depends on the objective.The cited comparison concerns minimal mean-squared reconstruction loss under a reconstruction model constrained to the ground-truth marginal data distribution.Source: §5 · p. 7
• Prior alternatives include heat-dissipation blurring and cold diffusion, but the cited discussion questions whether some such processes can learn the reverse process under the stated condition.Other approaches combine blur and noise with frequency-space variance preservation or use different sampling; the passage reports a performance gap from standard denoising diffusion across these approaches.Source: §5 · p. 7

6.1 COMPARISON WITH DETERMINISTIC AND DENOISING DIFFUSION MODELS

• Blurring Diffusion models achieve higher quality than IHDMs and outperform denoising diffusion on both CIFAR10 and LSUN Churches, with a smaller margin over denoising.CIFAR10 results use 50,000 samples after 2 million training steps; LSUN Churches results use 10,000 samples.Source: Table 1 · p. 8, Table 2 · p. 8, §6.1 · p. 8

6.2 COMPARISON BETWEEN DIFFERENT NOISE LEVELS AND SCHEDULES

• The sin2 schedule achieved much better FID than sin, particularly at higher maximum blur.The authors hypothesize that sin blurs too aggressively, while sin2 adds blur more gradually near t = 0.Source: Table 4 · p. 9, §6.2 · p. 9
• CIFAR10 FID was 3.17 at σB,max = 20 versus 3.60 with no blur; σB,max = 10 scored 3.26.For LSUN Churches, σB,max = 10 performed best, with performance relatively close to σB,max = 20.Source: §6.2 · p. 9, Table 3 · p. 9
• Higher maximum blur converged more slowly but outperformed lower blur after roughly 200K CIFAR10 steps and 1M LSUN Churches steps.CIFAR10 models with σB,max = 0 or 1 showed overfitting and had better FID at 1 million than 2 million steps, yet higher-blur models still outperformed them.Source: §6.2 · p. 9

7 LIMITATIONS AND CONCLUSION

• Blurring diffusion models generate images with improved perceptual quality compared with both DDPM and IHDM baselines.The class generalizes both model families.Source: §7 · p. 9
• Blurring and other deterministic transformations with fixed-variance Gaussian noise can be equivalently defined as Gaussian diffusion with non-isotropic noise.This connects the model class to non-isotropic diffusion and helps characterize its inductive bias.Source: §7 · p. 9
• Blur regularizes training and takes longer to reach convergence, which may be undesirable with very large training datasets.Its expected benefit also depends on the objective used to measure sample quality; a more exhaustive tradeoff study is left for future work.Source: §7 · p. 9

A ADDITIONAL DETAILS ON BLURRING DIFFUSION

• The appendix adds pseudocode covering essential steps for computing variables associated with the diffusion process.The section presents these details as supplementary implementation material.Source: A ADDITIONAL DETAILS ON BLURRING… · p. 12
• The additional material focuses on computation of diffusion-process variables.The passage describes the appendix's purpose without specifying the variables in this section introduction.Source: A ADDITIONAL DETAILS ON BLURRING… · p. 12
• The section is supplementary detail rather than a new model contribution.It introduces implementation steps for the previously described blurring diffusion model.Source: A ADDITIONAL DETAILS ON BLURRING… · p. 12

A.1 PSEUDO-CODE OF DIFFUSION AND DENOISING PROCESS

• Frequency scaling combines a dissipation-time schedule with per-frequency scaling, while the noise parameters use a cosine schedule.The scaling is combined with noise scaling to obtain α_t and σ_t.Source: §A.1 · p. 12
• Training samples z_t and ε from the diffusion process and minimizes mean squared error between ε and the network's prediction.The desired samples are from q(u_t|u_x); the pseudocode defines x, z_t, and ε_t in pixel space.Source: §A.1 · pp. 12–13
• Sampling repeatedly draws from p(z_{t−1/T}|z_t) across the timestep grid using a denoising mean and variance.The denoising computation uses frequency-space transforms and a sampled Gaussian term.Source: §A.1 · p. 13
• Several values are clipped to a minimum of 10^-8 to avoid numerically unstable divisions.The passage notes that DCT calls can be reduced when denoising is defined directly in frequency space, though DCTs are relatively cheap compared with the neural network.Source: §A.1 · p. 13
• A reliable way to adapt standard diffusion's clipping procedure without introducing frequency artifacts remains an open research question.The paper says clipping is known to improve standard denoising diffusion's visual quality, but its application to blurring diffusion is unclear.Source: §A.1 · p. 13

A.2 HYPERPARAMETER SETTINGS

• The experiments implement the neural network function fθ with a UNet architecture.The passage refers readers to Table 5 for specific architecture details.Source: §A.2 · p. 13, Table 5 · p. 14
• CIFAR-10 models use Adam with learning rate 2 · 10^-4 and batch size 128; LSUN models use learning rate 1 · 10^-4 and batch size 256.All methods are evaluated using an exponential moving average with decay 0.9999.Source: §A.2 · pp. 13–14
• Table 5 is identified as containing architecture settings.The supplied table passage gives its title but no settings values.Source: Table 5 · p. 14

B ADDITIONAL EXPERIMENTS

• For CIFAR10, the x −ˆx mean-squared-error parametrization reaches 23.9 FID, versus 3.17 for the epsilon parametrization.The diminished performance is also observed for standard diffusion (Ho et al., 2020).Source: B ADDITIONAL EXPERIMENTS · p. 14
• CIFAR10 Inception Scores are roughly similar across models; the best-performing model, with σB,max = 10, achieves 9.59.Table 6 presents eval-set FID and Inception Score results for CIFAR10.Source: B ADDITIONAL EXPERIMENTS · p. 14, Table 6 · p. 14
• Training the best-performing model in the frequency domain yields gray samples with checkerboard artifacts and higher loss throughout training.The authors conclude that directly learning a UNet in frequency space is not straightforward.Source: B ADDITIONAL EXPERIMENTS · p. 14
• On LSUN churches 64 × 64, Blurring Diffusion with σB,max = 20 achieves 2.62 train FID versus 2.70 for the denoising baseline.Results are described as similar to the higher-resolution case.Source: B ADDITIONAL EXPERIMENTS · p. 14, Table 7 · p. 14