Opening the Black Box of Classifier-Free Guidance via Information Bottleneck

1 Introduction

• Most CFG methods use fixed or heuristic schedules, leaving unclear how conditioning information's relevance to the score function changes during generation and when guidance should activate.The introduction notes that fixed schedules implicitly assume conditional information matters equally across denoising stages.Source: §1 · pp. 1–2
• A time-dependent information-bottleneck analysis links mutual-information evolution with Fisher information to derive a balance condition for CFG's transition region.The analysis finds conditional information becomes significantly more effective after a critical stage, making uniform CFG application suboptimal.Source: §1 · p. 2
• The proposed activation criterion estimates guidance gain from conditional–unconditional score-norm differences and uses a rescaled threshold to identify model-dependent activation regions.The rescaling is described as improving numerical conditioning.Source: §1 · p. 2
• The training-free dynamic CFG strategy activates guidance online by threshold crossing and connects adaptive activation with fixed-interval guidance in the large-batch concentration regime.The introduction describes the strategy as plug-and-play and applicable to batch-level and single-sample inference.Source: §1 · pp. 2–3
• Experiments across SiT, DiT, SD3, and FLUX report improved generation quality under varied settings, with multi-seed stability analyses and high-resolution single-sample experiments.The introduction also describes the approach as effective and broadly applicable across SiT, DiT, FLUX, and Stable Diffusion.Source: §1 · pp. 2–3

2 Background

• Diffusion models generate data by reversing a gradual noising process, described in continuous time by forward and reverse stochastic differential equations.The reverse-time formulation involves a score function that the network aims to predict.Source: §2 · p. 3
• CFG combines conditional and unconditional noise predictions, with guidance strength ω ≥ 1 controlling their contribution.The conditional predictor is contrasted with an unconditional predictor in the guidance expression.Source: §2 · p. 3
• The background introduces mutual information and Fisher information, including Fisher mutual information, before framing the information bottleneck as compression balanced against target-relevant information.In this formulation, β controls the balance between compressing input X and preserving information relevant to target C.Source: §2 · p. 3

3.1 Information Bottleneck on Conditional Diffusion Model

• The conditional diffusion trajectory is modeled as C → X0 → Xt, and an interior optimum of the bottleneck objective satisfies a Fisher-information balance that identifies a candidate CFG activation time.Source: §3.1 · p. 4

3.2 Extension and limitation on Flow Models

• For stochastic interpolation with an independent Gaussian prior endpoint, the bottleneck optimality condition extends to flow models, but deterministic couplings such as n-rectified flow (n ≥2) lack the same justification.Equation 20 remains a practical heuristic for those deterministic models.Source: §3.2 · pp. 4–5

3.3 Justification via Hyperparameter Rescale

• The simplified parameterization is better conditioned: its steeper log-parameter-to-t* curve reduces sensitivity to tuning perturbations.It serves as a computationally accurate proxy for the original bottleneck, with asymptotic consistency under the mapped scale.Source: Figure 1 · p. 5, §3.3 · pp. 5–6

• The method replaces a preset schedule with per-batch activation: it builds a stochastic process from score statistics and switches CFG when estimated guidance gain crosses a threshold.Source: §4 · pp. 6–7
• A backward scan from T selects the latest threshold-crossing time, and CFG remains active for subsequent denoising steps.Source: §4 · p. 7
• Near the information bottleneck, data fidelity and conditional relevance are balanced, and guidance is described as most effective at this critical time.Source: §4 · p. 7
• As batch size approaches infinity, the activation time approaches a deterministic boundary like Interval Guidance, while finite-batch activation is inferred from score statistics.Source: §4 · p. 8

5.1 Experimental Setup

• The evaluation covers ImageNet and MS-COCO using SiT, DiT, Stable Diffusion, and FLUX models with official pretrained weights.Metrics include FID, sFID, and IS for class-conditional tasks and FID, CLIP Score, HPSv2, and Aesthetic Score for text-to-image tasks.Source: §5.1 · p. 8

5.2 Experimental Results

• 2.06 on ImageNet-256 and 6.49 on ImageNet-512 are the reported DiT-XL/2 FID scores; SiT-XL/2 reaches FID 1.41 with SDE and 1.47 with ODE.The authors report consistent improvements across metrics for DiT and SiT, especially FID and IS.Source: §5.2 · p. 8, Table 1 · p. 9
• Across tested batch sizes B ∈ {4, 8, 16, 32}, the method outperforms the baseline on FID in every setting.Table 2 compares evaluation metrics across batch sizes, holding other settings identical to Table 1 unless otherwise specified.Source: Table 2 · p. 9, §5.2 · p. 9
• On SD3, the method improves FID, HPSv2.1, CLIP Score, and Aesthetic Score over standard CFG; on FLUX.1 at 256 × 256, FID and HPSv2.1 improve while CLIP Score slightly declines.The Table 3 caption lists these metrics where available.Source: Table 3 · p. 9, §5.2 · p. 9
• At 1024 × 1024 with FLUX.1 and ω = 7.5, average HPSv2.1 is 30.73 for B = 16 and 30.29 for single-sample inference (B = 1).The authors report 20.13 for standard CFG at B = 16.Source: Table 3 · p. 9, §5.2 · p. 9
• Across three random seeds, the method reports FID 4.98 ± 0.06 and IS 298.1 ± 2.6, outperforming Interval Guidance.The multi-seed comparison is on SiT-XL/2.Source: §5.2 · p. 9, Table 4 · p. 10
• Mean batch-wise activation time remains approximately 0.68 as batch size changes, while its standard deviation decreases with larger batches.The Fisher-derived score-norm difference achieves better FID and IS than the conditional–unconditional score-discrepancy MSE under tested configurations.Source: §5.2 · p. 9, Table 5 · p. 10, Table 6 · p. 10
• Visual comparisons on Stable Diffusion 3 and Flux.1 show the approach generating semantic details that are missing or incorrect in baseline results.Figure 3 marks prompt details in red and highlights method differences with boxes.Source: Figure 3 · p. 10, §5.2 · p. 10

6 Conclusion and Future Work

• Conditional information is not uniformly effective during generation but becomes critical after a stage characterized by a Fisher-information balance.The conclusion frames this finding through an information-bottleneck perspective on CFG.Source: §6 · p. 10
• The proposed training-free dynamic CFG strategy adaptively activates guidance by monitoring score statistics, with experiments reporting improved sample quality and robustness across architectures.Source: §6 · p. 10
• Future work includes fully characterizing deterministic couplings, developing more robust sample-specific criteria, and extending the framework to heterogeneous conditioning.The authors also note that batch averaging can improve activation-statistic stability despite support for single-sample inference.Source: §6 · p. 10

A Extended Related Work

• Classifier guidance [5] uses an external classifier, while CFG [8] uses conditional and unconditional score estimates and is described as dominant because of its simplicity and empirical performance.The text notes that fixed CFG rules across reverse steps can be suboptimal.Source: A Extended Related Work · p. 15
• Prior dynamic approaches vary guidance strength or parameterization, including β-adaptive guidance [11], Frequency-Decoupled Guidance [17], and normalization methods [42] and [12].Other approaches modify guidance dynamics through control [16], momentum [43], or a temporal-prediction signal [44].Source: A Extended Related Work · p. 15
• Interval Guidance [15] restricts CFG to a fixed intermediate range because guidance can harm diversity under high noise and have limited effect at very low noise.The passage says existing activation intervals still largely rely on hand-designed schedules.Source: A Extended Related Work · p. 15
• Information-theoretic studies include conditional mutual-information maximization [45] and DGIB [46], while other work studies information-theoretic learning objectives [47].This paper instead focuses on conditional information during inference, deriving a Fisher-information balance for dynamic CFG activation.Source: A Extended Related Work · pp. 15–16
• The paper focuses on adaptive activation based on generation dynamics rather than prescribing a schedule or interval, and interprets interval guidance as a limiting special case.Source: A Extended Related Work · p. 16

B.1 Proof of Lemmas

• The lemma proofs establish entropy evolution for diffusion processes and then derive mutual-information evolution under conditioning.The appendix specializes the discussion to an Ornstein–Uhlenbeck process.Source: §B.1 · pp. 16–17

B.2 Proof of Theorem 1

• Theorem 1 applies the lemmas to the information-bottleneck objective and derives its interior-minimizer condition using first-order optimality.Source: §B.2 · p. 17

B.3 Proof of Theorem 2

• Theorem 2 follows the same objective-based proof approach, applying the flow-model information-evolution lemma to derive the optimality condition.The flow-model lemma is stated for differentiable α_t > 0 and σ_t ≥ 0.Source: §B.3 · p. 18

C.1 Asymptotic Consistency Near the Gaussian Endpoint

• For stochastic Gaussian interpolation paths with independent endpoints, score–velocity conversion is exact.For deterministic n-rectified flows with n ≥2, the conversion is generally inexact and the surrogate is a practical approximation induced by the learned velocity field.Source: §C.1 · pp. 18–19
• Near t →1, assuming bounded velocity and a Gaussian endpoint, the surrogate score converges to the true score.This makes it an asymptotically accurate proxy near the Gaussian endpoint.Source: §C.1 · p. 19

C.2 Consistency of Score Differences via Local PDE Dynamics

• The local analysis compares conditional–unconditional score differences because the true scores and their surrogates share the limit −x_t.Source: §C.2 · p. 19
• Under the stated local conditions, the discrepancy vanishes asymptotically, preserving the rate of change of the guidance score-norm difference under the surrogate.The analysis assumes a local score approximation and locally affine velocity near t = 1.Source: §C.2 · pp. 19–20

D Implementation Details and Code

• The implementation is a training-free, plug-and-play wrapper for flow-matching SDE samplers, with analogous procedures for ODE, DDPM, and DDIM samplers.Source: D Implementation Details and Code · p. 20
• The code converts the model velocity to a score and combines conditional and unconditional updates using the CFG scale.Source: D Implementation Details and Code · p. 21
• The sampler checks whether log_beta exceeds the threshold before enabling guidance, then advances the sample with an Euler–Maruyama update.Source: D Implementation Details and Code · p. 21

E.1 Properties of Stopping Time

• The stopping time has mean τ̄ ≈0.680 and standard deviation στ ≈0.050, indicating a central tendency with moderate trajectory-to-trajectory variability.Source: §E.1 · p. 22
• The fitted Weibull has the highest log-likelihood (L ≈713.24), versus Exponential (L ≈−268.89) and Gamma (L ≈675.64), with shape k ≈16.78 and scale λ ≈0.70.Source: §E.1 · p. 22
• Figure 4 compares the empirical τ histogram with the fitted Weibull PDF and KDE, which closely align with the observed distribution.Source: §E.1 · p. 22, Figure 4 · p. 22
• Although activation peaks around 0.7, the Weibull survival function indicates a long tail of activation events down to 0.4.This variability supports per-sample timing rather than treating the mean as a fixed activation point.Source: §E.1 · pp. 22–23
• The stochastic stopping strategy achieves the best overall FID, while the mean stopping-time threshold improves IS over Interval Guidance but remains inferior.The comparison uses the empirical mean stopping time of approximately 0.68 versus the Interval Guidance threshold of 0.7.Source: §E.1 · p. 23, Table 7 · p. 23

E.2 Ablation on Parameter log β∗

• As log β∗ rises from 5.9 to 6.3, both FID and IS improve; beyond the peak around 6.3, fidelity degrades at higher thresholds.Figure 5 depicts the quality–diversity trade-off across threshold values.Source: §E.2 · p. 24, Figure 5 · p. 24
• At log β∗ = 6.9, FID slightly degrades despite continued IS improvement, as overly delayed activation reduces the guidance window for refining samples.Source: §E.2 · p. 24
• The results support activating guidance neither too early nor too late, with an appropriate threshold important for performance.Source: §E.2 · p. 24
• FID reaches 4.82 and IS improves significantly over baseline around log β∗ ∈ [6.3, 6.7].Source: Figure 5 · p. 24, §E.2 · p. 24

E.3 Additional visual results

• Figure 6 presents additional results using the method on Flux and SiT, including generated images from FLUX.The figure caption identifies the FLUX images.Source: §E.3 · pp. 24–25, Figure 6 · p. 25

NeurIPS Paper Checklist

• The method is a training-free, plug-and-play inference wrapper with lightweight statistical estimation and dynamic thresholding; code is provided in Appendix D.The checklist notes that no complex retraining code is required.Source: NeurIPS Paper Checklist · p. 28
• The authors direct readers to Section 5 for experimental settings and report analyzing the distribution of τ and validating its effectiveness in Appendix E.They state that other settings are the same as the baseline and mark the checklist item on limitations as covered in Section 3 and future work in Section 6.Source: NeurIPS Paper Checklist · pp. 26, 28