Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models

• High guidance scales improve quality and prompt alignment but often produce oversaturated colors and simplified compositions.Source: §1 · p. 2
• CFG updates can be decomposed into parallel and orthogonal components relative to the conditional prediction, with the orthogonal component improving quality and the parallel component primarily adding contrast and saturation.Source: §1 · p. 2
• APG combines projection, rescaling, and reverse momentum to support higher guidance scales without oversaturation or degraded image quality.Source: §1 · p. 2
• Across EDM2 and Stable Diffusion, APG uses high guidance scales without oversaturation and improves FID, recall, and saturation scores while maintaining precision similar to CFG.Source: §1 · p. 2

• Diffusion guidance improves generation quality and condition alignment but can reduce diversity and produce oversaturated outputs.Source: §2 · p. 3
• CADS and interval guidance target diversity at high guidance scales, whereas APG specifically addresses oversaturation and can be combined with them.Source: §2 · p. 3
• Dynamic thresholding is not directly applicable to latent diffusion and can reduce detail, while APG is reported to reduce oversaturation more effectively than CFG Rescale.Source: §2 · p. 3
• APG extends projection with rescaling and reverse momentum, applies across conditional diffusion models, and adds practically no sampling overhead.Source: §2 · p. 3

• The forward process adds time-dependent noise to data, and sampling reverses this process from Gaussian noise toward the data distribution.Source: §3 · p. 3
• A neural denoiser estimates clean samples or the unknown score from noisy inputs, and conditional denoisers incorporate labels or text prompts.Source: §3 · p. 3
• CFG combines conditional and unconditional denoiser predictions, with guidance scale w=1 representing unguided sampling.Source: §3 · p. 3
• CFG improves image quality but reduces diversity, paralleling truncation-based quality–diversity trade-offs in GANs.Source: §3 · p. 3

Orthogonal projection

• The orthogonal component chiefly improves image quality, whereas the parallel component primarily increases saturation in generated images.This decomposition motivates weakening the parallel component while retaining the orthogonal direction.Source: §4 · p. 4
• Reducing the parallel-component weight η toward zero significantly reduces saturation and produces more realistic generations at higher guidance scales.η ≤ 1 controls the strength of the parallel component; η = 1 recovers the unmodified CFG direction.Source: §4 · p. 4
• Parallel updates act like a gain on the conditional prediction, pushing values toward their extremes and adding saturation when guidance scale w > 1.The paper compares this effect to multiplying pixel values by a number greater than one.Source: §4 · p. 4

Adding rescaling

• APG interprets CFG as gradient ascent and rescales each update inside a sphere to control large norms and limit drift during sampling.The rescaling radius r is a hyperparameter, and the adjustment improves both FID and recall.Source: §4 · p. 4

Adding reverse momentum

• APG adds reverse momentum with β < 0, pushing updates away from previous directions and further enhancing image quality through lower FID scores.The negative momentum encourages greater focus on the current CFG update direction.Source: §4 · pp. 4–5

APG overview

• Across illustrative comparisons, APG produces high-quality generations without the oversaturation and artifacts seen with high-scale CFG, while remaining easy to implement.Projection must be applied to denoised predictions rather than model outputs such as predicted noise.Source: Figure 3 · p. 5, §4 · p. 5

5.1 MAIN RESULTS

• APG produces more realistic, less saturated images with fewer artifacts than CFG in EDM2 and Stable Diffusion XL qualitative comparisons.The reported outputs also show more globally consistent prompt-following generations.Source: §5.1 · p. 6, Figure 4 · p. 6, Figure 5 · p. 6
• APG consistently improves FID, recall, and color metrics while maintaining similar or better precision than CFG across multiple models.The comparison is reported in Table 1.Source: §5.1 · p. 7, Table 1 · p. 7
• APG samples have broader pixel and saturation distributions with less concentration at extreme values, indicating closer alignment with real-data color composition.The distributions are shown using kernel density estimates for generated and real ImageNet samples.Source: §5.1 · p. 7, Figure 7 · p. 8
• As guidance scale increases, APG achieves lower FID, higher recall, and similar or better precision while maintaining relatively constant saturation.CFG instead exhibits increasing saturation at higher guidance scales.Source: §5.1 · p. 7, Figure 8 · p. 8
• APG improves generation diversity by increasing recall while maintaining precision comparable to CFG.Additional qualitative diversity results are reported in the appendix.Source: §5.1 · p. 8
• APG remains effective with distilled models, where CFG can degrade quality, and does not degrade output quality with SDXL-Lightning.The paper reports improved generation quality when replacing CFG with APG in this setting.Source: §5.1 · p. 8, Figure 9 · p. 8
• APG improves text-rendering consistency in Stable Diffusion 3, producing more accurate spelling than standard CFG.Figure 10 reports consistently correct spellings with APG in comparison with CFG.Source: §5.1 · p. 8, Figure 10 · p. 9

5.2 ABLATION STUDIES

• Projection, rescaling, and reverse momentum each contribute to APG, with projection chiefly reducing saturation and the latter two mainly improving FID and recall.Projection is most effective when applied to denoised predictions; projecting noise predictions produces results nearly identical to CFG.Source: §5.2 · p. 10, Figure 12 · p. 10, Table 2 · p. 10

6 CONCLUSION AND DISCUSSION

• APG projects the CFG update onto the denoised prediction, removes or down-weights its parallel component, and adds rescaling and reverse momentum to mitigate oversaturation.The method is presented as a plug-and-play alternative with practically no additional computational overhead.Source: §6 · p. 10
• Experiments show that APG improves FID, recall, and saturation metrics relative to CFG while maintaining similar or better precision.The conclusion also states that APG supports higher guidance scales without oversaturation.Source: §6 · p. 10
• A remaining challenge is reducing APG’s sampling cost toward unguided sampling by eliminating the need for two diffusion-network queries per step.The paper identifies this acceleration as a direction for future work.Source: §6 · p. 10

A DETAILS ON CFG AS GRADIENT ASCENT

• CFG is equivalent to a single gradient-ascent step on an objective defined from conditional and unconditional denoised predictions.The update is written as the conditional prediction plus a scaled gradient of the CFG objective.Source: A DETAILS ON CFG AS GRADIENT ASCENT · p. 16
• Because the CFG objective has no maximum, rescaling constrains updates within a limited-radius ball, while reverse momentum pushes updates away from previous predictions.These modifications target arbitrary drift during sampling.Source: A DETAILS ON CFG AS GRADIENT ASCENT · p. 16

B DENOISED PREDICTION FOR DIFFERENT DIFFUSION MODELS

• APG converts model outputs into denoised predictions Dθ(zt, t, y), enabling compatibility with denoising-based samplers including EDM and DPM++.The paper provides separate conversions for DDPM, EDM, and rectified-flow models.Source: B DENOISED PREDICTION FOR DIFFERENT… · p. 16
• High-guidance CFG drifts toward low-probability regions and reduces mode coverage in the Gaussian toy problem, whereas APG corrects the drift and improves coverage.The authors argue that the analogous image-level phenomenon produces artifacts and oversaturation.Source: B DENOISED PREDICTION FOR DIFFERENT… · p. 17, Figure 13 · p. 18

C.1 COMPATIBILITY WITH CADS AND IG

• Replacing CFG with APG improves FID, recall, and saturation for both CADS and interval guidance.Table 4 describes combining APG with these diversity-enhancing methods as producing better FID than either method in isolation.Source: §C.1 · p. 17, Table 4 · p. 18

C.2 APG VS NUMBER OF SAMPLING STEPS

• Across different numbers of EDM2 sampling steps, APG consistently achieves better FID, recall, and saturation while maintaining the same precision as CFG.The comparison is reported in Figure 14.Source: §C.2 · p. 18, Figure 14 · p. 18

C.3 USING APG WITH DISTILLED MODELS

• With PIXART-δ, SDXL-Lightning, and SDXL-Flash, APG improves generation quality and saturation without degrading output quality relative to CFG.The distilled-model results also show APG outperforming CFG at fewer sampling steps, such as 8–16.Source: §C.3 · p. 18, Figure 15 · p. 19

C.4 COMPATIBILITY WITH DIFFERENT SAMPLERS

• Using APG with different diffusion samplers improves FID, recall, and saturation compared with CFG.The experiment uses class-conditional ImageNet generation with DiT-XL/2, and the result is summarized in Table 5.Source: §C.4 · p. 19, Table 5 · p. 20

C.5 COMPATIBILITY WITH ICG

• Combining APG with ICG improves FID, recall, and saturation while maintaining similar precision to CFG.This compatibility is reported for class-conditional ImageNet generation with EDM2-S.Source: §C.5 · p. 20, Table 6 · p. 20

C.6 COMPATIBILITY WITH TSG

• Applying APG to TSG improves FID, recall, and saturation while maintaining similar precision to TSG.The experiment uses class-conditional ImageNet generation with DiT-XL/2.Source: §C.6 · p. 20, Table 7 · p. 20

C.7 ALIGNMENT WITH THE CONDITION

• CFG and APG achieve comparable condition-alignment metrics, while APG reduces saturation and improves FID.Alignment is measured using ImageNet classification accuracy and Stable Diffusion CLIP score.Source: §C.7 · p. 20

C.8 EXTENDED ABLATION STUDIES

• Increasing the parallel-component strength raises saturation and FID, so the paper recommends η = 0 unless more saturation is desired.The ablation is reported in Table 9a.Source: §C.8 · p. 20
• Midrange rescaling radii yield better FID because excessive rescaling degrades quality while large radii approach the identity function.The paper recommends choosing r comparable on average to the norm of ∆D_t during inference.Source: §C.8 · p. 21
• Negative momentum strengths outperform positive momentum for FID, while excessive momentum degrades image quality.The reported effective range is β ∈ [−0.75, −0.25] in most setups.Source: §C.8 · p. 21

Implementation

• APG adds only a few lines of code compared with CFG, without noticeable sampling overhead.The implementation converts model predictions to Dθ(zt, t, y), computes the guided prediction, and converts back at each step.Source: D IMPLEMENTATION DETAILS · p. 21

Evaluation

• FID, precision, and recall are computed using the ADM evaluation suite.Class-conditional ImageNet FID uses 10,000 generated images and the full training set, while text-to-image FID uses the MS COCO 2017 evaluation subset.Source: D IMPLEMENTATION DETAILS · p. 21

Implementation

• Algorithm 1 computes the conditional–unconditional prediction difference, optionally applies momentum, clips its norm, and separates parallel and orthogonal components.The resulting normalized update combines the orthogonal component with an eta-weighted parallel component before forming the guided prediction.Source: E MORE VISUAL RESULTS · p. 22
• The APG implementation accepts conditional and unconditional predictions, a guidance scale, an optional momentum buffer, eta, and a norm threshold.These inputs define the configurable guidance, momentum, and update-rescaling behavior.Source: E MORE VISUAL RESULTS · p. 22

Visual comparisons

• APG examples show reduced oversaturation and improved overall compositional diversity compared with CFG.The diversity comparison is presented for EDM2, while additional comparisons cover Stable Diffusion 2.1, Stable Diffusion XL, and other models.Source: Figure 16 · p. 23, Figure 17 · p. 24, Figure 18 · p. 24
• APG produces more consistent spellings than CFG in Stable Diffusion 3 text-rendering examples.The comparison is shown in Figure 20.Source: E MORE VISUAL RESULTS · p. 21, Figure 20 · p. 25
• Additional visual comparisons evaluate APG with PIXART-δ, SDXL-Flash, and SDXL-Lightning distilled models.These examples appear in Figures 21–23.Source: E MORE VISUAL RESULTS · p. 21, Figure 21 · p. 26, Figure 22 · p. 26