PaperlayerView exact revision on arXiv

Source-linked AI summary

Correcting Guided Diffusion Trajectories with Spectral Alignment

Gihoon Kim, Taesup Kim

arXiv:2610.02753v1cs.CVcs.AI

TL;DR

CFG lacks an explicit criterion for assessing whether its guided trajectory is progressing as intended. The paper introduces training-free spectral correction based on an analytic reference spectrum, reporting gains across conditional image-generation tasks, including ImageNet.

  • Problem

    CFG’s fixed guidance scale limits its ability to provide appropriate guidance at each denoising step, and assessing guidance magnitude and direction along the trajectory is challenging.

    Source: §1 · p. 1

  • Method

    Spectral Correction Guidance corrects intermediate-state spectral deviations using a reference derived from natural-image power-law statistics, without modifying or retraining the model.

    Source: §1 · p. 2, §4 · p. 5, §7 · p. 9

  • Results

    The method improves preference-based metrics across text-to-image backbones and guidance settings, and on ImageNet reduces sFID from 7.42 to 4.92 versus CFG.

    Source: §1 · p. 2, §6.1 · p. 8

  • Takeaways & Limitations

    Spectral alignment provides a practical criterion for assessing guided trajectories and correcting them across diffusion backbones and conditional-generation tasks.

    Source: §7 · p. 9

    Power-spectrum matching captures marginal frequency statistics but not phase information or spatial structure.

    Source: D LIMITATIONS AND FUTURE WORK · p. 14

Abstract

from arXiv · show

The practical success of conditional image generation hinges on fine-grained differences in condition alignment and visual fidelity. Classifier-free guidance (CFG) is central to this success, but its lack of an explicit criterion makes it difficult to assess whether the guided trajectory is progressing as intended. To address this gap, we show that spectral alignment provides a principled criterion for understanding guidance behavior and improving guided diffusion sampling through adaptive correction. Our analysis identifies the spectra of intermediate states as an indicator of consistency with the expected spectral evolution of the forward process. Based on this observation, we introduce Spectral Correction Guidance, a method that corrects deviations from an analytic reference spectrum during sampling. The proposed method is training-free and applicable across diffusion backbones and conditional generation tasks without modifying the underlying model. Experiments demonstrate consistent gains in preference-based metrics over baseline guidance methods in text-to-image generation and improved generation quality over CFG on ImageNet. These improvements persist across a range of guidance scales and with fewer denoising steps. Our analyses and ablations provide insight into guidance behavior and how the proposed method affects generation quality.

1 INTRODUCTION

Standard CFG improves conditional generation, but its fixed guidance scale can limit step-appropriate steering. The paper uses spectral alignment as a criterion for adaptively correcting guided denoising trajectories.

1 INTRODUCTION

  • CFG is widely used to improve condition alignment and perceptual quality, but a fixed scale may not provide appropriate guidance at every denoising step.Other approaches adjust timestep strength, guidance direction, or frequency bands, but the introduction notes limited gains across settings in Section 6.1.Source: §1 · p. 1
  • Power-spectrum alignment offers a global indicator of guidance effectiveness where spatial representations encode local, sample-specific structure.The criterion captures global patterns in diffusion dynamics along the denoising trajectory.Source: §1 · p. 2
  • Spectral Correction Guidance is training-free and corrects deviations from expected spectral behavior in intermediate states.Its reference comes from natural-image power-law statistics and is not tied to individual samples or specific failure patterns.Source: §1 · p. 2
  • The correction adapts across frequencies and timesteps without imposing a prescribed correction pattern.The paper also examines how correction strength and the timestep range affect outputs.Source: §1 · p. 2
  • Across text-to-image backbones, the method improves preference-based metrics; on ImageNet it beats CFG on FID and Inception Score, even with fewer steps.The paper reports these gains across multiple guidance settings and studies their behavior through ablations.Source: §1 · p. 2

2 BACKGROUND

Diffusion models generate samples by reversing a predefined noising process. Classifier-free guidance steers this reverse sampling toward a condition, though choosing an appropriate guidance magnitude and direction along the trajectory remains challenging.

2 BACKGROUND

  • The forward process q progressively noises data, while the learned reverse process pθ models transitions from noisier to less-noisy states.The forward marginal uses time-dependent scaling factors, and the reverse process can use noise, velocity, or score prediction.Source: §2 · pp. 2–3
  • Figure 2 plots spectral distance, FID, and IS against guidance scale, with a dashed line marking the default scale.The guidance-scale values are uniformly spaced for visual clarity.Source: Figure 2 · p. 3
  • Matching reverse marginals pθ(zt) to forward marginals q(zt) across timesteps lets the diffusion model approximate the data distribution q(z0).The reverse marginal is induced by reverse sampling at time t.Source: §2 · p. 3
  • CFG combines unconditional and conditional noise predictions, scaling their difference by w; ∅ denotes the null condition.The paper derives this form from classifier guidance using Bayes’ rule and Tweedie’s formula.Source: §2 · p. 3
  • Assessing whether CFG’s guidance magnitude and direction are appropriate along the denoising trajectory remains challenging.The guidance direction is the difference between conditional and unconditional noise predictions.Source: §2 · p. 3

3 SPECTRAL DYNAMICS IN THE DIFFUSION PROCESS

The paper assesses guided diffusion trajectories by comparing intermediate-state power spectra with those expected under the forward process. Spectral discrepancy varies with guidance scale and tracks generation quality, although corresponding forward states are unavailable during sampling.

3 SPECTRAL DYNAMICS IN THE DIFFUSION PROCESS

  • Agreement between expected forward- and reverse-state power spectra is a necessary condition for matching their marginals.The spectra summarize how energy is distributed across frequency components.Source: §3 · p. 3
  • In Figure 3, the corrected spectrum follows the analytic target more closely than the baseline, reducing artifacts and improving visual quality.The figure plots power against radial frequency bins over denoising timesteps.Source: Figure 3 · p. 4
  • The spectral distance measures discrepancy across timesteps and radial frequencies between expected forward- and reverse-state power spectra.The comparison uses the same noise schedule across guidance scales.Source: §3 · pp. 3–4
  • Lower spectral discrepancy corresponds to lower FID and higher IS, while excessive guidance increases discrepancy relative to moderate guidance.These observations suggest spectral alignment indicates consistency with expected diffusion dynamics; corresponding forward states are unavailable during sampling.Source: §3 · p. 4

4 SPECTRAL CORRECTION FOR CLASSIFIER-FREE GUIDANCE

The method derives an analytic reference spectrum for forward diffusion and uses it to correct guided reverse-state spectra. Its frequency-adaptive correction preserves Fourier phase, while practical estimates and late-stage relaxation support application across backbones.

4 SPECTRAL CORRECTION FOR CLASSIFIER-FREE GUIDANCE

  • The expected intermediate-state spectrum has a power-law component plus a noise-dependent offset, yielding an analytic reference without simulating forward states.The derivation uses a power-law model for natural-image spectra, Gaussian noise, and the forward process.Source: §4 · p. 4
  • The correction interpolates the current reverse-state spectrum toward the expected target in log-spectral space, with λ controlling correction strength.Log space accommodates the wide dynamic range of spectral power.Source: §4 · p. 5
  • The correction rescales each frequency by the target-to-current spectral ratio, increasing power when the ratio exceeds 1 and decreasing it when the ratio is below 1.The adjustment is adaptive across frequencies.Source: §4 · p. 5
  • The denoising correction preserves the current latent’s Fourier phase and changes only its magnitude, retaining spatial information while aligning its spectrum with the target.Figure 3 visualizes alignment in overall energy and frequency-dependent shape.Source: §4 · p. 5
  • The target spectrum is estimated from real-image statistics in each backbone’s z0 space, and correction is relaxed late in denoising to retain fine frequencies.The procedure uses the noise schedule and does not select samples to match training data.Source: §4 · p. 5

5 A SPECTRAL VIEW OF EXISTING GUIDANCE METHODS

The paper contrasts existing guidance strategies by the signal they adjust: timestep strength, update geometry, or predefined frequency bands. Its method instead adapts correction at each frequency using deviations from an analytic reference.

5 A SPECTRAL VIEW OF EXISTING GUIDANCE METHODS

  • Scheduled guidance scaling varies guidance strength over denoising timesteps, which also modulates the Fourier-domain energy of the guidance term.Prior methods use selected intervals or predefined functional forms; this method provides fine-grained scaling across frequency and time.Source: §5 · p. 5
  • Geometry-aware methods constrain guided-update magnitude or direction, or reduce deviations from the data manifold; this method instead aligns spectra with forward marginals.Both approaches aim to keep guided trajectories consistent with the data distribution.Source: §5 · p. 5
  • Frequency-based methods apply predefined band-specific guidance rules, whereas this method adapts correction at each frequency according to deviation from an analytic reference.Existing rules include assigning different strengths to separate bands or suppressing selected frequencies.Source: §5 · p. 5

6 EXPERIMENTS AND RESULTS

Across text-to-image and ImageNet experiments, spectral correction improves guidance outcomes; ablations show that correction strength and timing must be balanced. The experiments cover multiple backbones, guidance scales, and sampling budgets.

6.1 EXPERIMENTAL RESULTS

  • Our method achieves the highest HPSv3, ImageReward, and PickScore across all three backbones, with the best or joint-best HPSv2 scores.Table 1 compares guidance methods on text-to-image models; standard CFG remains the most competitive existing baseline in most cases.Source: Table 1 · p. 6, §6.1 · p. 6

6 EXPERIMENTS AND RESULTS

  • Evaluation spans 10K COCO prompts across SDXL, PixArt-α, and SD3.5, plus ImageNet class-conditional generation with DiT-XL/2.The ImageNet evaluation uses 50 samples per class, totaling 50K samples.Source: §6 · pp. 5–6

6.1 EXPERIMENTAL RESULTS

  • In Figure 4 examples, our method alleviates CFG’s airplane artifact, improves PixArt-α brightness balance, and produces more distinct SD3.5 details.Across the text-to-image comparisons, it also better preserves intended composition and visual coherence.Source: §6.1 · p. 6, Figure 4 · p. 7
  • At higher guidance scales, our method improves preference metrics across three backbones and mitigates CFG’s over-saturation or weaker prompt following.Table 2 compares the methods at higher scales; the SD3.5 examples use w = 14.Source: Table 2 · p. 7, §6.1 · p. 8, Figure 5 · p. 8
  • At the default 250 steps on ImageNet, sFID falls from 7.42 to 4.92, and FID, Precision, and IS also improve over CFG.Figure 6 examples show a clearer chameleon depiction and brighter, sharper jellyfish and space shuttle images.Source: §6.1 · p. 8, Table 3 · p. 8, Figure 6 · p. 8
  • At 50 NFEs, our method reaches FID 2.88 and sFID 6.29, matching CFG’s 250-NFE FID while improving its sFID; recall is slightly lower across budgets.At 25 NFEs, the method remains competitive with CFG using 100 NFEs.Source: §6.1 · p. 8, Table 3 · p. 8

6.2 ABLATION STUDIES

  • Moderate correction strength improves fidelity and prompt alignment, whereas excessive correction introduces artifacts and degrades perceptual quality.Correction strength balances spectral-deviation correction against preservation of individual sample characteristics.Source: §6.2 · p. 9, Figure 7 · p. 9
  • Intermediate relaxation of the frequency constraint balances spectral correction with natural detail formation; applying it only early weakens its effect, while applying it through the end can retain artifacts.The paper adopts intermediate relaxation in its final method.Source: §6.2 · p. 9, Figure 8 · p. 9

7 CONCLUSION

The conclusion presents spectral alignment as a practical criterion for assessing and correcting guided diffusion trajectories, with generation-quality gains across architectures, guidance strengths, and reduced sampling budgets. It also identifies limits of power-spectrum matching and shared correction across conditions.

7 CONCLUSION

  • Spectral Correction Guidance uses an analytic reference from natural-image statistics to correct spectral deviations during sampling, without training or modifying the underlying model.The conclusion reports improved generation quality across model architectures, guidance strengths, and reduced sampling budgets.Source: §7 · p. 9

A REPRODUCIBILITY STATEMENT

  • Section 4 presents the method formulation and Algorithm 1 summarizes the procedure; Appendix E documents settings and resources for reproduction.Appendix F provides the proof of Proposition 1.Source: A REPRODUCIBILITY STATEMENT · p. 14

B LLM USAGE STATEMENT

  • The authors used generative AI to polish the manuscript and assist with experimental-code implementation and debugging, and reviewed the assisted outputs.Source: B LLM USAGE STATEMENT · p. 14

C ETHICS STATEMENT

  • The ethics statement notes that improved generation quality can increase misuse risk, including harmful synthetic content, and emphasizes responsible deployment policies.Source: C ETHICS STATEMENT · p. 14

D LIMITATIONS AND FUTURE WORK

  • Power spectra capture marginal frequency statistics but omit phase and spatial structure, so matching them does not fully characterize the generated distribution.Future work could use wavelets and constraints for phase and individual-sample structure.Source: D LIMITATIONS AND FUTURE WORK · p. 14
  • The correction applies a shared rule across samples and conditions, although guidance needs may depend on the prompt, input condition, or class.The authors suggest condition-aware guidance control as a direction for making spectral correction more flexible.Source: D LIMITATIONS AND FUTURE WORK · p. 14

E ADDITIONAL IMPLEMENTATION DETAILS

The additional implementation and evaluation details describe the correction setup, its proof, computational cost, and further comparisons. These experiments report preference and robustness gains, including advantages over post-generation spectral correction and results on stylized prompts.

E ADDITIONAL IMPLEMENTATION DETAILS

  • Experiments fit the reference power-law curve using COCO val 50K as a practical, nonessential choice, and implement baselines in recommended settings with model-specific correction parameters.The conceptual validation averages spectra over 1,000 COCO image-caption pairs before computing log-power-spectrum error; the method’s settings and resources are also documented.Source: E ADDITIONAL IMPLEMENTATION DETAILS · pp. 14–15

F PROOF OF PROPOSITION 1

  • The proof Fourier-transforms the forward process, removes the expected cross term using independence and zero-mean noise, then substitutes power-law clean-image and flat white-noise spectra.Source: F PROOF OF PROPOSITION 1 · p. 16

G COMPUTATIONAL COST

  • Peak GPU memory remains comparable to CFG, while correction adds about 1.7 seconds per SDXL image and 2.6 seconds per SD3.5 image.The correction requires no backpropagation or additional denoiser evaluations; fewer denoising steps could offset the overhead while maintaining quality.Source: G COMPUTATIONAL COST · p. 17, Table 4 · p. 17

H HUMAN EVALUATION

  • On SDXL, participants preferred our method over CFG for visual quality and prompt alignment, with win rates of 59.50% and 58.80%, respectively.The study used 50 prompts and 20 participants, with 1,000 votes per criterion.Source: H HUMAN EVALUATION · p. 17, Table 5 · p. 17

I ROBUSTNESS ACROSS RANDOM SEEDS

  • Across 10 SDXL seeds, our method has higher mean HPSv3, ImageReward, PickScore, and HPSv2 and lower mean FID than CFG, while CFG has slightly better mean CLIP-T and sFID.The gains in HPSv3, ImageReward, and HPSv2 are large relative to the observed standard deviations.Source: I ROBUSTNESS ACROSS RANDOM SEEDS · p. 17, Table 6 · p. 17

J COMPARISON WITH OUTPUT-ONLY CORRECTION

  • Our method achieves larger gains than output-only spectral correction in HPSv3, ImageReward, PickScore, and HPSv2, with lower FID and sFID.The comparison supports benefits from correcting spectral deviations during sampling beyond adjusting the final spectrum alone.Source: J COMPARISON WITH OUTPUT-ONLY CORRECTION · p. 18, Table 7 · p. 18

K NON-PHOTOREALISTIC AND STYLIZED GENERATION

  • For anime illustration and oil-painting prompts, our method improves HPSv3, ImageReward, PickScore, and HPSv2 over CFG; CLIP-T improves for anime and is unchanged at reported precision for oil painting.The authors attribute the results’ relevance beyond photorealistic generation to correction relaxation toward the end of sampling.Source: K NON-PHOTOREALISTIC AND STYLIZED… · p. 18, Table 8 · p. 18

L ADDITIONAL EXPERIMENTS ACROSS GUIDANCE SETTINGS

At reduced guidance on SDXL, spectral correction remains beneficial where other correction methods often do not. Qualitative examples also report better prompt alignment and visual coherence at this scale.

L ADDITIONAL EXPERIMENTS ACROSS GUIDANCE SETTINGS

  • At w = 2.5, our method performs best on all reported metrics except sFID, improving preference scores and FID over CFG while existing correction methods often degrade performance.This is half the standard guidance scale used in the main experiments.Source: L ADDITIONAL EXPERIMENTS ACROSS… · p. 18, Table 9 · p. 19
  • At reduced guidance, examples show relatively better prompt alignment and visual coherence, including clearer racket-holding and giraffe-feeding interactions.The authors note that images at this scale are not always fully satisfactory across methods.Source: L ADDITIONAL EXPERIMENTS ACROSS… · pp. 18–19, Figure 9 · p. 19

M ADDITIONAL EXPERIMENTS ON DIVERSE BACKBONE MODELS

Additional class-conditional tests on SiT and JiT indicate that spectral correction can transfer beyond the DiT setting. The JiT results also show greater sample-level variation, including artifacts in some cases.

M ADDITIONAL EXPERIMENTS ON DIVERSE BACKBONE MODELS

N ADDITIONAL ANALYSIS OF POWER SPECTRUM MODELING

The power-law model fits the latent spectra across examined models, supporting its use as a reference for spectral correction. Additional analyses show that binning resolution changes the correction’s visual effects and that ρ = 0.5 balances three evaluated metrics.

N ADDITIONAL ANALYSIS OF POWER SPECTRUM MODELING

  • R2 > 0.99 across models for the fitted power-law spectra supports characterizing the examined latent spaces with this structure.The analysis reports this fit for encoded COCO val 50K images; Figure 12 shows the spectra and fitted curves.Source: N ADDITIONAL ANALYSIS OF POWER… · p. 20, Figure 12 · p. 21
  • Coarse binning, such as 8 radial bins, mainly changes global brightness, whereas finer binning, such as 128 bins, produces more noticeable structural changes.The comparison uses a relatively strong correction strength of λ = 0.5.Source: N ADDITIONAL ANALYSIS OF POWER… · p. 20, Figure 13 · p. 21

O ADDITIONAL ANALYSIS OF CORRECTION RELAXATION

  • ρ = 0.5 yields the most balanced results across PickScore, FID, and sFID on SDXL.The authors also report that relaxation benefits can be more apparent in visual quality than in metrics, with more natural-looking samples and fewer visible artifacts.Source: O ADDITIONAL ANALYSIS OF CORRECTION… · p. 21

P ADDITIONAL QUALITATIVE RESULTS

Additional examples show improved visual qualities across several backbones, alongside cases where artifacts persist or visual improvements trade off against other image properties. The discussion notes that spectral correction does not provide explicit feedback toward a desired semantic direction.

P ADDITIONAL QUALITATIVE RESULTS

  • Across SDXL, PixArt-α, and SD3.5, correction mitigates CFG artifacts, improves brightness, sharpness, and text-aligned content, and makes only small adjustments when spectra already approach the reference.Some outputs remain visually close to CFG; in those cases, limited correction helps preserve its achieved quality.Source: P ADDITIONAL QUALITATIVE RESULTS · p. 21
  • On SDXL, ρ = 0.5 provides the most balanced results across PickScore, FID, and sFID.Figure 14 plots each metric against the relaxation factor.Source: Figure 14 · p. 22
  • Qualitative failures include residual artifacts and trade-offs, such as improved color tone accompanied by more generated people and weaker text consistency.The authors describe cases where local details improve while other image details remain unnatural or object interactions weaken.Source: P ADDITIONAL QUALITATIVE RESULTS · p. 22

Q ADDITIONAL DISCUSSION

  • Spectral correction offers no explicit feedback toward a desired semantic direction, limiting its control over generation toward a specific target.The discussion notes that more direct semantic control may require external rewards or task-specific supervision, which introduce separate challenges.Source: Q ADDITIONAL DISCUSSION · p. 22
Loading 2610.02753v1…