PaperlayerView exact revision on arXiv

Source-linked AI summary

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment

Denis Suchkov

arXiv:2609.39531v1math.STcs.LGstat.TH

TL;DR

Estimating diffusion guidance is difficult because the sampler needs the gradient of the log conditional expectation, not just an accurate estimate of that function. The paper estimates the guidance using a weak harmonicity-residual penalty computable from the pretrained score, and proves high-probability error bounds that can be faster than generic derivative estimation in favourable scenarios.

  • Problem

    Guidance estimation requires controlling the gradient of the log conditional expectation, whose error is not guaranteed by small function-estimation error and is harder to estimate than the function itself.

    Source: §1 · p. 2

  • Method

    The estimator uses a weak form of the h-function’s harmonicity equation to penalize its residual, yielding a first-order penalty computable from the pretrained model’s score.

    Source: §1 · p. 3

  • Results

    The paper gives a non-asymptotic high-probability squared L2 error bound for guidance that is faster than generic first-derivative estimation in favourable scenarios.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The guidance error bound transfers to the total variation distance between marginals of the estimated and exactly guided samplers.

    Source: §1 · p. 3

    The theory treats the pretrained score as exact, and the guarantees’ degradation under score error remains to be quantified.

    Source: §8 · p. 12

Abstract

from arXiv · show

We study the problem of estimating the guidance that steers the distribution learned by a diffusion generative model toward a tilted target $q_0 \propto w\,p_0$ at inference time. Relying on the stochastic optimal control approach, we observe that the exact drift correction is the gradient of the logarithm of Doob's $h$-function, and we study the problem of estimating it from a sample. In the present paper, we assume that the score of the pretrained model is available, that the tilting weight is bounded and positive, and that the reference distribution has a bounded support, no smoothness of the weight is required. Introducing a penalized least-squares risk in which the penalty is the residual of the space-time harmonicity equation satisfied by the $h$-function, measured in a dual Sobolev norm, we derive high-probability bounds on the squared error of the resulting guidance estimate. Since the penalty vanishes at the target, the estimator is free of regularization bias, and in favourable scenarios its rate of convergence is faster than the minimax rate of estimating first-order derivatives of a smooth regression function. Assuming that $w$ is bounded and positive with $\mathbb{E}_{p_0}[w^{-\mathrm{s}}] < \infty$ for some $\mathrm{s} \in (0,\infty]$, and that the reference data are compactly supported, we prove that the guidance is estimable in squared $L^2$ at rate $\varepsilon_n^{\mathrm{s}/(\mathrm{s}+4)}$, where $\varepsilon_n = n^{-2(\beta-1)/(2(\beta-1)+d)}.$ We also transfer the obtained bounds to the total variation distance between the marginals of the estimated and the exactly guided samplers, and illustrate the performance of the suggested approach with numerical experiments.

1 Introduction

The paper studies exact inference-time alignment by estimating the Doob h-function guidance from data, using its space-time harmonicity to regularize estimation. It gives high-probability guidance-error bounds, with faster-than-generic derivative-estimation rates in favourable cases, and transfers the bounds to sampler marginals in total variation.

1 Introduction

  • Inference-time alignment leaves pretrained weights untouched and uses the exact drift correction, given by the gradient of the logarithm of Doob’s h-function, to target a tilted distribution.The h-function is estimated from data because the reference distribution is unknown.Source: §1 · p. 2
  • Estimating h* by regression does not ensure accurate guidance, because the sampler uses its log-gradient and derivative estimation is slower than function estimation.For a β-smooth regression function observed in noise, the cited squared L2 rates are n^-2β/(2β+d) for the function and n^-2(β−1)/(2β+d) for its first derivative.Source: §1 · p. 2
  • The space-time harmonicity residual supplies a penalty that is computable from the sample and pretrained score, yet vanishes at h*, so it introduces no population-level bias.A weak formulation moves one derivative from h* onto the test function, making the penalty first order in both arguments.Source: §1 · pp. 2–3
  • The paper derives a non-asymptotic high-probability squared L2 error bound for guidance, faster than the generic first-derivative minimax rate in favourable scenarios when w is not too close to zero.The bound holds on all but an arbitrarily small end-portion of the estimation window.Source: §1 · p. 3
  • The guidance-error bound transfers to total variation distance between the marginals of the estimated and exactly guided samplers on the estimation window.The proof uses an exact reverse-diffusion energy identity to control the gradient through value error and weak residual, without an inf-sup condition.Source: §1 · p. 3
  • Related h-transform estimators either lack rates, use penalties that do not vanish at the target, or bound drift error as a parameter rather than estimating it at a rate.The paper frames its question as the attainable rate for a bias-free penalty and the assumptions needed to obtain it.Source: Related work · p. 4

2 Setup

The setup treats the pretrained score as known and estimates the Doob h-function guidance for a tilted target using reverse-diffusion samples on an interior time window. The theory assumes a bounded positive weight with an inverse-moment condition and compact reference support.

2 Setup

  • The pretrained score is treated as known, separating score-model approximation error from error in estimating h*; image experiments instead use the model’s learned score.The reverse process has marginals ρ_t=p_(T−t), from which the estimation sample is drawn.Source: §2 · pp. 4–5
  • The target is q0=wp0/Z, and the Doob h-transform adds 2∇log h* to the reverse-process drift.Here h* is the conditional expectation of the terminal weight given the reverse process at time t, and its gradient of the log is the guidance.Source: §2 · p. 5
  • The weight is measurable, positive, bounded above, and has a finite inverse moment E_p0[w^-s] for some s>0; the reference distribution is supported in a bounded cube.The order s quantifies how close w may come to zero and enters the rate; no smoothness of w is assumed.Source: §2 · p. 5
  • Estimation is restricted to an interior reverse-time window because recovery is ill-posed near T and there is nothing to recover near 0.The window is I=[t0,t1]⊂(0,T), with positive noise level at its late endpoint.Source: §2 · p. 5

3 Harmonicity and its weak form

The population excess least-squares risk measures value error, while a weak harmonicity residual adds a computable, gap-free penalty. Its first-order approximation floor supports convergence without a population-level bias trade-off.

3 Harmonicity and its weak form

  • The weak form is computable from first derivatives of the candidate and test functions and the pretrained score.Integration by parts against the marginal produces its logarithmic derivative, the score already available from the pretrained model.Source: §3 · p. 6
  • At the target h*, the weak penalty vanishes, so every penalty weight leaves the population minimizer unchanged.The excess least-squares risk is the population value error because its fixed-time minimizer is the conditional mean h*.Source: §3 · p. 6
  • The weak residual’s first-order continuity bound yields a first-order approximation floor for suitable network classes.The paper contrasts this with alternative residual penalties in Table 1; only the last is both gap-free and informative at β = 2.Source: §3 · p. 6, Table 1 · p. 6

4 From residual to gradient

An exact reverse-diffusion energy identity links the weak residual to gradient error, without an inf-sup condition. Endpoint averaging makes the identity usable for statistical guarantees, but those guarantees are restricted to an interior portion of the time window.

4 From residual to gradient

  • Endpoint averaging introduces a weight that vanishes at the window’s upper endpoint, so guarantees are stated on Iδ, not the full window.On Iδ the weight is at least δ; this covers a (1 − δ) fraction of the window at a δ^-1 cost in the constant.Source: §4 · p. 7
  • A finite test class exposes only a projected residual, not the full dual-norm residual needed for the coercivity bound.Recovering the full bound conventionally requires a discrete inf-sup condition, which the paper avoids by imposing structural class properties.Source: §4 · p. 7
  • The test class is a symmetric cone, allowing its inner supremum to split into radial and directional parts.The radial part is a scalar quadratic, and the deviation bounds place the empirical inner supremum around the projected residual without an additive term outside the ridge.Source: §4 · p. 7
  • Absorption requires the test class to contain weighted differences of trial functions; a difference of two networks can be represented by a larger network.The stated construction uses a test network with twice the width and one additional layer.Source: §4 · p. 7
  • Theorem 4.1 uses an exact energy identity to control gradient error through value error and the weak residual, without an inf-sup condition.Instead, the trial and test classes must satisfy two structural properties; the classes need not be linear or convex.Source: §4 · p. 7
  • The coercivity theorem applies under stated smoothness and boundedness conditions on the trial functions and a bound on weighted differences.For the classes of Definition 5.2, the regularity conditions hold and C0 = 1.Source: §4 · p. 7

5 Estimator and main result

The section defines trial and test classes for estimating guidance and analyzes an estimator that penalizes a projected residual. Its main result gives a high-probability guidance-error bound, including an n^-2/(d+2) rate up to logarithmic factors when β=2 and s=∞.

5 Estimator and main result

  • The trial class uses clipped, retracted network functions, while the test class uses scaled clipped network functions.The classes are built with a smooth spatial clip and a smooth retraction.Source: §5 · p. 8
  • At β=2 and s=∞, the guidance-error bound is of order n^-2/(d+2), up to logarithmic factors, with probability at least 1−ζ.This is the boundary case where the weight has a uniform positive lower bound.Source: §5 · p. 9
  • The estimator minimizes empirical fit with a ridge-secured projected-residual penalty; gap-freeness leaves no bias–variance tradeoff in calibration.The inner supremum is tied to the projected residual, and the ridge secures the bound.Source: §5 · p. 9
  • The weak residual is decisive at β=2: unlike the strong penalty, it permits the rate balance because its approximation floor does not block it.The strong-penalty floor would make the balance empty at β=2.Source: §5 · p. 10

6 From guidance error to sampling error

The corollary connects guidance-estimation error to differences between the exactly and approximately controlled samplers over a time interval. It also explains how bounds under the uncontrolled marginals can yield guarantees under the controlled ones.

6 From guidance error to sampling error

  • The corollary transfers the guidance-estimation guarantee into a bound on distances between controlled sampler marginals over J.The exactly and approximately guided path laws are equivalent.Source: §6 · p. 10
  • For the boundary case s = ∞, the stated bound reads δ^-1ℓ̄BB^-5σ^-c log^8(en/ζ)εn.Source: §6 · p. 10
  • Projection keeps the Novikov constant independent of n, while reweighting transfers a guarantee under sampleable uncontrolled marginals to controlled marginals.The reweighting bound is Υ≤B̄/Z.Source: §6 · p. 10

7 Numerical Experiments

The experiments test the estimator's rate and its use for image-model alignment. They report guidance-error rates, reward shifts and FID costs, alongside comparisons with other steering methods.

7 Numerical Experiments

  • In the rate experiment, measured guidance-error exponent is 0.842±0.026, against 0.857 predicted and 0.667 for noisy first-derivative recovery.At β=3, the strong-residual approximation floor decays as N^-1.0, versus N^-1.8 for the weak residual.Source: §7 · p. 10
  • At c=20, animal share rises from 0.653 to 0.831, versus 0.854 for 32-fold importance resampling at 32× the cost.Figure 1b plots held-out reward shift against FID cost; the caption reports that DOIT is off-panel at 4.8 and 21 times its base.Source: Figure 1 · p. 11, §7 · p. 11
  • The CIFAR-10 experiment tilts a pretrained DDPM toward animals using a bounded positive classifier-based weight and convolutional trial and test classes.The score is the pretrained network; samples are also evaluated by a classifier head not used to define the weight.Source: §7 · pp. 10–11
  • At c=1, window-only guidance achieves 11.7% of the exact tilt’s reward shift at FID 24.7, against base FID 24.5.The authors describe this as a measure of practical steering, not the guarantee itself.Source: §7 · p. 11
  • At fixed departure, ours reaches 62.5% of the exact tilt’s shift versus 62.7% at FID 28, and 77.0% versus 79.2% at FID 30; DOIT reaches 70.6% at FID 116.6.Only DEFT receives the gradient of the log weight; the proposed method does not differentiate the reward model.Source: §7 · p. 11
  • On MNIST, odd-digit share rises from 0.523 to 0.819 at FID 13.8, or 176% of the exact tilt’s shift; DEFT reaches 84% at that FID.DEFT needs FID 29.5 to reach 155%; DOIT reaches 296% at FID 153, with its cap binding at every step.Source: §7 · pp. 11–12

8 Discussion and limitations

The discussion weighs the convergence-rate benefit of increasing β against factorially growing constants and identifies the smallest admissible β as operative in moderate samples and high dimensions. The theory treats the pretrained score as exact, while image experiments use the base model’s output and leave robustness guarantees to score error for future work.

8 Discussion and limitations

  • At moderate n and large d, factorial growth in Aβ can make the smallest admissible β the better trade, despite improved exponents at larger β.The discussion identifies β = 2, yielding n^-2/(d+2), as the operative case; it is also where the strong residual is non-informative.Source: §8 · p. 12
  • The theory treats the pretrained score as exact; image experiments instead use the base network’s output, and quantifying guarantee degradation from score error remains future work.The paper describes the image results as evidence that the construction tolerates pretrained-score error, not as a quantified robustness guarantee.Source: §8 · p. 12
  • The authors expect the theorem’s ambient dimension d could be replaced by intrinsic dimension k, but present this as an expectation rather than a proved result.They attribute any such gain to the approximation side, since the bilinear form is integrated against the d-dimensional marginal.Source: §8 · p. 12

A Notation recap

  • The Doob h-function is the conditional expectation of the terminal weight given the reverse-process state, and the guidance is its log-gradient.The recap also expresses h* as a conditional expectation given the forward-process state at time T−t.Source: A Notation recap · p. 15
  • β is a free construction parameter: increasing it improves approximation order while the associated constant grows factorially.The approximation analysis uses uniform W^β,∞ regularity on the time window, which the text says follows from the assumptions rather than being an added assumption.Source: A Notation recap · p. 15

B Regularity of the Doob h-function

Under the paper’s assumptions, the Doob h-function is smooth and its derivatives admit quantitative bounds. The section relates guidance to a posterior covariance and establishes boundedness and negative-moment control relevant to estimation.

B Regularity of the Doob h-function

  • The Doob h-function is C∞ on the interior time interval under Assumptions 1–2, with derivative bounds that quantify its smoothness.The approximation analysis can select β as a free parameter, trading against a constant that grows factorially.Source: B Regularity of the Doob h-function · pp. 16–17
  • The guidance is a rescaled posterior covariance between the weight and clean signal, with σ^-2 and σ^-4 scalings that quantify endpoint singularity.The stated analysis excludes the endpoint region through its time window.Source: B Regularity of the Doob h-function · pp. 16–17
  • The guidance is bounded under Assumptions 1–2 without requiring a lower bound on the tilting weight.The proof uses averages over posteriors supported on the compact support of the reference distribution.Source: B Regularity of the Doob h-function · pp. 17–18
  • Negative moments of h* transfer from the weight: Eρt[(h*)^-s] is bounded by Ep0[w^-s], uniformly in time.The passage derives this by conditional Jensen’s inequality and the tower property.Source: B Regularity of the Doob h-function · p. 18

C Harmonicity, the weak form, and gap-freenesss

The target h* is space-time harmonic for the reverse generator, making its PDE residual vanish. The section uses this property to motivate a weak, first-order residual penalty that remains gap-free and avoids the strong residual’s second-derivative bottleneck.

C Harmonicity, the weak form, and gap-freenesss

  • The target h* is space-time harmonic for the reverse generator, with terminal condition h*(T, ·) = w.The martingale property of h*(t, reverse-process state) underlies the harmonicity result.Source: C Harmonicity, the weak form, and… · p. 18
  • The value-fidelity term is the excess least-squares regression risk for estimating the conditional mean h*, and the residual penalty vanishes at that target.Consequently, h* uniquely minimizes the penalized population risk for every nonnegative penalty weight, with no regularization gap.Source: C Harmonicity, the weak form, and… · p. 19
  • The strong residual contains second derivatives, whose approximation control can leave a floor even at β = 2; removing that floor motivates the weak form.The stated approximation control for a resolution-N approximation of a W^β,∞ function is of order N^-(β−2).Source: C Harmonicity, the weak form, and… · p. 19
  • The weak residual tests the PDE against ψ and integrates by parts, yielding a first-order bilinear form computable from the covariates and pretrained score.The integration-by-parts weight is the marginal’s logarithmic derivative, equal to the pretrained score; the weak form vanishes at h* for every test function.Source: C Harmonicity, the weak form, and… · pp. 19–20
  • The weak penalty is measured in the H^-1 dual norm, and the analysis must show that it still controls the gradient while improving the approximation floor.The section gives a continuity bound for the residual in terms of time and spatial derivatives of h−h*.Source: C Harmonicity, the weak form, and… · p. 20

D The energy identity and coercivity

An exact energy identity connects value error, gradient energy, and the residual cross term. Weighting that identity yields coercivity and supports control of the clipped, projected guidance on a specified portion of the time window.

D The energy identity and coercivity

  • The time weight ω is the survival function of the endpoint measure; its properties enable absorption in the weighted energy bound.The guarantee is stated on the half-window because converting the weighted bound to an unweighted one requires ω ≥ 1/2 there.Source: D The energy identity and coercivity · pp. 21–22
  • The energy identity is exact and relates endpoint value error, integrated gradient energy, and a residual cross term.Averaging the endpoint over the uniform time measure makes the terminal value term equal to the value-fidelity risk D(h).Source: D The energy identity and coercivity · pp. 20–22
  • The weak residual’s dual norm bounds the cross term and yields coercivity; the projected test-class quantity suffices without an inf-sup constant.The estimator controls a projection of the weak penalty rather than the full penalty, and Section G establishes sufficiency for the chosen test class.Source: D The energy identity and coercivity · pp. 21–23
  • The guidance bound applies to a clipped, projected estimate and uses a lower-tail bound for h* when h* falls below the clipping level.On that low-h* event, both estimated and true guidance are bounded by G*, and the contribution is bounded using the negative moment condition.Source: D The energy identity and coercivity · pp. 23–24

E The hypothesis classes

This section develops neural-network hypothesis classes for the analysis, establishing their regularity, target inclusion, stability, and complexity properties. It applies these bounds to the trial, test, generator, and model classes.

E The hypothesis classes

  • The network classes have a parameter-to-function Lipschitz bound and finite covering bounds for joint function-gradient jets on the evaluation domain.The entropy control uses realized hidden-activation bounds, which the paper says are checkable and yield a polylogarithmic entropy bound.Source: E The hypothesis classes · pp. 24, 30
  • The clipped trial and gated test classes are constructed from rectified-cubic networks, and the target h* satisfies the trial constraints and is unchanged by the output retraction.The classes are C1,2 with bounded space-time derivatives, meeting the stated hypotheses of Theorems D.1 and D.2.Source: E The hypothesis classes · pp. 25–26
  • The test class requires two scales: its cone scale controls global size, while its V-norm can be small even when its supremum norm or gradient is large.This distinction motivates two-norm deviation bounds and a ridge at the cone scale.Source: E The hypothesis classes · p. 27
  • The unit V-ball of the test class is not totally bounded in the gradient supremum norm under the stated conditions, and the ratio of cone scale to V-norm is unbounded.Consequently, its gradient covering numbers are infinite at every radius.Source: E The hypothesis classes · p. 28
  • When P0 ≲KN d, WN ≲KN d, 2 ≤ N ≤ n, L ≲ log N, and Λ,A ≤ poly(n), the entropy bound satisfies VN ≲ KN d log N log n.With K ≤ σ^−c(log n)^(3/2), this gives VN ≲ Nd polylog(n).Source: E The hypothesis classes · pp. 30, 33
  • The network Jacobian is stable under parameter perturbations: its sup-norm difference is at most 12L^2m^(9L+3)δ throughout Z.For function values, the corresponding bound is |fϑ(z) − f˜ϑ(z)| ≤ 2Lm^(3L+1)δ.Source: E The hypothesis classes · p. 31
  • For generic sparse networks, the jet covering number is controlled by the number of nonzero parameters and parameter magnitude, via an ε-net of logarithmic size bounded in (E.17).The argument covers sparse parameter supports and transfers parameter-space nets to function-value and gradient jets.Source: E The hypothesis classes · p. 32
  • The trial class HN embeds in the sparse network class, yielding log N(ε, JHN, ||·||∞) ≤ e(ε) for its value-gradient jets.The same covering bound applies to generator jets JΦN; scaling and covering the scalar coefficient gives the corresponding ΨN bound.Source: E The hypothesis classes · p. 32

F The approximation floor

This part constructs bounded neural-network approximations to the space-time h-function, using spatial smoothness and time analyticity to control the weak residual at a quantified approximation scale. It also establishes projected-residual coercivity under stated regularity and function-class conditions, with slack for empirical and population adversary norms.

F The approximation floor

  • Cubic B-splines provide an exact partition of unity, enabling localized approximation with W^k,p error bounded by C∥f∥W^n,pN^-(n-k).The spline family is an exact 3-partition of unity of radius 2 for the rectified cubic activation.Source: F The approximation floor · pp. 35–36
  • For integer β ≥ 2, a ϱ₃-network approximates W^β,∞ functions simultaneously in W^0,∞ and W^1,∞ at rates N^-β and N^-(β-1), respectively.The network uses at most C(N + 1)^d nonzero weights and can be encoded with ⌈Cβ log₂N⌉ bits.Source: F The approximation floor · p. 37
  • The space-time construction achieves weak-residual approximation error e_N = 8Q_nA_βN^-(β-1) under the stated network-budget conditions.On I × Q_R, the approximation also satisfies |H_N − h*| ≤ Q_nA_βN^-β and first-derivative errors bounded by Q_nA_βN^-(β-1).Source: F The approximation floor · pp. 38, 40
  • Analyticity of h* in time permits Chebyshev truncation, so temporal approximation contributes only polylogarithmic factors rather than an additional polynomial resolution dimension.The resulting bound uses A_β, which is polylog(n) at fixed β but grows factorially in β.Source: F The approximation floor · pp. 33, 38, 39
  • The clipped approximation preserves derivative bounds, and its network realizes HN exactly with depth L, width at most C♯K(N + 1)^d, and at most C♯K((N + 1)^d + L) nonzero parameters.The construction clips outside the approximation region and uses the identity clip within it.Source: F The approximation floor · pp. 40–41

G Removing the inf-sup assumption

  • The slack τ makes empirical and population adversary norms comparable multiplicatively, while the projected-residual approximation input is available for every τ.The population uses τ = 0; the estimator uses τ = δn.Source: G Removing the inf-sup assumption · p. 41
  • Theorem G.4 establishes coercivity without an inf-sup constant, bounding weighted energy by residual, projected-residual, slack, and approximation terms under stated conditions.For the classes in Definition E.3, the stated hypotheses other than the approximation condition hold with C0 = 1.Source: G Removing the inf-sup assumption · pp. 42, 44

H Statistical guarantee

Under the paper’s assumptions, this section develops high-probability statistical guarantees for the guidance estimator, including a simultaneous deviation bound and a squared-L2 error rate governed by ε_n and the inverse-weight moment condition. The guarantees have no regularization-gap term and include envelope and entropy controls for the empirical-process argument.

Envelopes and truncation

  • The empirical-process class has an envelope controlled by the cone scale and a sub-Gaussian drift envelope; truncating the drift yields bounded surrogates.On the event that all sampled drift envelopes are at most T, truncation is inactive on the sample.Source: H Statistical guarantee · p. 45

Entropy of the index classes

  • The index classes are bounded and admit uniform L2 covering-number control, supporting the entropy-based deviation analysis.The stated entropy bound applies to the truncated empirical-process class, the discrepancy class, and the quadratic adversary class.Source: H Statistical guarantee · p. 46

Deviation bound

  • With probability at least 1 − ζ, Theorem H.7 gives a deviation bound simultaneously for all h ∈ HN and ψ ∈ ΨN, yielding a purely multiplicative comparison.The theorem assumes Assumptions 1–2 and specifies a sufficiently large constant C0σ−c.Source: H Statistical guarantee · p. 49

H Statistical guarantee

  • At fixed β, the guidance bound’s n exponent is 2(β−1)/(2(β−1)+d); for β = 2 and s = ∞, its rate is n^(-2/(d+2)) up to logarithmic factors.For any fixed κ, the bound contains no regularization-gap term.Source: H Statistical guarantee · p. 51
  • The bound contains no regularization-gap term for any fixed κ; only the ratio μ/α matters, so its choice does not trade bias against variance.The proof relates the estimator’s risk to approximation error and the penalty scale.Source: H Statistical guarantee · pp. 51–52
  • Moment conditions from test functions control the fitted h-function’s first derivatives at complexity V_N ≍ N^d, avoiding the N^2 variance amplification of pure value estimation.The resulting balance is N^(2(β-1)+d) ≍ n.Source: H Statistical guarantee · p. 53

I From guidance error to sampling error

The paper transfers guidance-estimation error bounds to sampling-law error for exactly and approximately guided diffusions. The guarantee is restricted to a time window, and the exactly guided process must use a tilted initial law.

I From guidance error to sampling error

J.1 General implementation details

The experiments use frozen unconditional diffusion checkpoints and evaluate an empirical objective alongside synthetic-instance comparisons. They report penalty diagnostics, define reward and distribution metrics, and compare weak-residual estimation with value least squares.

J.1 General implementation details

  • The experiments use frozen unconditional DDPM checkpoints, sampled deterministically in 100 DDIM steps, with animal-superclass and odd-digit targets.The checkpoints use ϵ-prediction and 1000 training steps.Source: §J.1 · p. 55

J.2 The empirical objective

  • The reported network penalty is a lower bound on NΨ(ĥ); cross-fitted cosine-feature checks yielded 2.1·10^-3, 1.8·10^-3, and 1.1·10^-3 across the stated settings.The corresponding value-only least-squares fit measured 5.7·10^-3, while the linear estimator measured 5.1·10^-4.Source: §J.2 · p. 57

J.3 Metrics used in the experiments

  • Reported metrics include normalized held-out reward gain, target-class share, FID against the model’s own reference sample, and transfer ratio across classifier heads.Source: §J.3 · p. 57

J.4 The synthetic instance

  • The synthetic sweep found the weak-residual estimator ahead of value least squares in all 20 cells, with the margin growing with dimension.The table compares relative squared guidance error at equal budget; its largest sample size is 10^6, and 3.2 · 10^5 at d = 3.Source: §J.4 · p. 57, Table 4 · p. 58
  • Table 3 compares relative guidance error at the smallest and largest sample sizes; a rate is predicted for the weak residual and value least squares, but not the other penalties.The caption attributes a regularization bias to the Sobolev penalty and an approximation floor to the strong residual.Source: Table 3 · p. 58

J.5 Evaluation on CIFAR-10

The evaluations compare guidance methods on CIFAR-10 and MNIST, examine the effects of the shared correction cap, and test approximation floors and convergence behavior. Comparisons are interpreted along reward-shift/FID frontiers rather than by matching guidance scale alone.

J.5 Evaluation on CIFAR-10

  • 0.842 ± 0.026 is the weak-residual fitted exponent, close to its predicted 0.857; least squares fits 0.909 ± 0.038 against its predicted 0.667.The least-squares slope reflects climbing from a worse error level over the measured range, not a faster rate.Source: Figure 2 · p. 59
  • −43.8% held-out reward shift is DOIT’s CIFAR-10 result at γ = 0.01; its best small-γ point reaches 116% at FID 222.On MNIST, γ = 0.01 reaches 88.5% at FID 14.1, while the proposed method reaches 176% at FID 13.8.Source: §J.5 · p. 59

J.6 Evaluation on MNIST

  • At c = 20 on MNIST, the proposed method reaches 176.4% of the exact tilt’s shift, versus about 84% for DEFT at the same FID.The proposed method’s transfer ratio rises from 1.37 to 1.54, while DEFT’s rises from 1.56 to 1.66.Source: §J.6 · p. 60

J.7 Final hyperparameter values

  • Comparisons use the reward-shift/FID frontier, not fixed-c table rows, because methods at the same c can have different FIDs.The sweeps use 100 DDIM steps, a correction cap of 10% of the predicted-noise norm, and three seeds of 2,000 samples.Source: Table 5 · p. 60, Figure 3 · p. 61, §J.7 · p. 61

J.8 What the guidance cap does

  • The shared correction cap does not explain the CIFAR-10 frontier match: it binds on none of DEFT’s steps and at most one twentieth of the proposed method’s steps there.Removing the cap costs the proposed method nine points of share and ten points of FID; uncapped DOIT at γ = 20 collapses to FID 719 with no animals left.Source: §J.8 · p. 62

J.9 Approximation floors at the operative smoothness

  • At β = 2, the strong-residual approximation floor stops decaying while the weak-residual floor keeps falling, with the gap widening from 7 to 311 to 8500.The fitted slopes support the ordering and flattening, not the predicted exponents, because the grid binds early at β = 3 and 4 and h* is smoother than assumed at β = 2.Source: §J.9 · p. 63

J.10 The finite-s regime

The finite-s experiment tests whether errors worsen as the weight approaches zero, as the theorem’s envelope predicts. Clipping separates regimes: the estimator continues converging while the bound remains informative, then stalls when the bound goes flat.

J.10 The finite-s regime

  • The experiment lowers w on 80% of the atoms through 1, 0.3, 0.1, 0.03, and 3 × 10^-3, while the rest retain weight 3.The finite-s ladder is compared with the s=∞ boundary case, where clipping never binds; the experiment moves to I=[0.96, 0.995] because the earlier window smooths away the low-weight region.Source: §J.10 · p. 63
  • At min w=0.1 clipping does not bind at n=106 and error matches the reference rung’s 2 × 10^-5; at 0.03 it binds at every sample size and error is two orders worse.Table 9 reports the finite-s ladder across seven sample sizes, with each cell at its own oracle (N, µ).Source: Table 9 · p. 63, §J.10 · p. 63
  • At min w=3 × 10^-3 there is no convergence over three decades of n, and the bound goes flat where the estimator stops converging.The measured slope is at least as steep as the envelope at every rung.Source: §J.10 · pp. 63–64

Related papers

Loading 2609.39531v1…