Equal Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models

1 Introduction

• Perturbations arise from score or denoising errors, deliberate guidance, and computational approximations, creating a mismatch between trajectory-level measures and output-level consequences.Because perturbations alter inputs to later denoising steps, their effects depend on subsequent sampling dynamics.Source: §1 · p. 1
• Prior work bounds generated-distribution deviations or studies accumulated prediction errors, but does not fully characterize how downstream dynamics mediate perturbation magnitude and final output effects.The cited endpoint-observability analysis concerns recursively trained diffusion models and measures pathwise score-error energy detectable in the terminal distribution.Source: §1 · pp. 1–2
• The framework combines information-theoretic analysis and dynamical output-response characterization to explain why equal-strength perturbations can have unequal output effects.It measures strength by the KL divergence between perturbed and reference trajectory distributions, called path cost.Source: §1 · p. 2
• The framework decomposes path cost into output-distribution change and trajectory-distribution change conditional on the output, and derives a propagation law for accumulating local perturbations.It also bounds feature-mean responses relative to local output-distribution changes, explaining why a selected feature mean may fail to detect them.Source: §1 · p. 2
• Equal-path-cost interventions produce output responses that differ across sampling stages and spatial directions, while first-order propagation captures temporal patterns but can mispredict magnitudes.Smaller absolute feature-mean responses can still correspond to larger changes relative to baseline fluctuations.Source: §1 · p. 2
• Propagation analysis identifies cache-acceleration sampling intervals with larger image deviations more reliably than local error magnitude and path cost.The diagnostic remains informative even when predicted and observed error magnitudes differ substantially.Source: §1 · p. 2

Error propagation in diffusion sampling

• Score-error analyses provide likelihood and sampling-error bounds, while propagation equations relate local and cumulative errors and endpoint-observability work measures terminal visibility of pathwise errors.This paper instead studies perturbations of a fixed sampler and connects output visibility to downstream propagation and selected feature means.Source: §2 · p. 2

Conditional information and observable response

• Information and response work characterizes retained Fisher information and bounds relating mean sensitivity to variance and path information; this paper links feature-mean capture to propagation.The cited foundations include conditional-score projections, fluctuation–response bounds, and smooth-test-function bounds for diffusion sampling errors.Source: §2 · pp. 2–3

Spectral generation dynamics

• Prior spectral work derives Gaussian spectral transfer dynamics and uses data spectra to design schedules, whereas this paper compares temporal and directional responses at matched path cost.It distinguishes absolute response from response relative to baseline feature fluctuations.Source: §2 · p. 3

Caching and sensitivity-based acceleration

• Caching methods reuse features or calibrate reuse from sensitivity, feature shifts, or terminal deviations; this paper tests reference-trajectory propagation for multi-step cache image errors against local-defect and path-cost measures.The cited approaches include DeepCache, SenCache, ERTACache, and EpaCache.Source: §2 · p. 3

3.1 How path cost quantifies perturbation strength

• Path cost is the KL divergence between perturbed and reference trajectory distributions; under isotropic transition noise, each mean-shift cost scales inversely with noise variance.For Λ_i = ν_iI, the conditional KL at step i is ε^2∥u_i∥^2/(2ν_i).Source: §3.1 · pp. 3–4

3.2 Why path cost does not determine output changes

• The exact KL decomposition splits path cost into output-distribution change and nonnegative bridge cost, so equal path costs need not yield equal output changes.A Gaussian example shows that a cancelling perturbation can have positive path cost while output KL remains zero.Source: §3.2 · pp. 4–5

3.3 How local perturbations propagate and accumulate

• Downstream updates transform each local perturbation, and their propagated contributions sum; cross-time directions can reinforce or cancel the total response.The propagation map includes the remaining sampling updates and output transformation.Source: §3.3 · p. 5

3.4 What feature mean responses reveal about output changes

• Theorem 1 factors captured path information into output visibility and feature–score alignment, so output change can leave a selected feature mean unchanged to first order.Output visibility measures the fraction of path cost reflected in output-distribution change; feature–score alignment measures how much local output information the chosen feature mean captures.Source: §3.4 · pp. 6–7

4.1 Experimental setup

• The controlled experiments perturb transition means within one of eight 16-step windows and vary spatial direction with unit-norm DCT templates, using equal path cost within each model.The study covers class-conditional ImageNet-256 generation with pixel-space JiT-B/L and ADM, and latent-space DiT; it measures feature means for JiT/ADM and decoded-image RMS for DiT.Source: §4.1 · p. 7

4.2 Spatiotemporal sensitivity to perturbations

• At equal path cost, the most sensitive sampling stage varies with both model and spatial frequency: peaks shift across windows in ADM and DiT, while JiT-B peaks in the first window for all four frequencies shown.Two DiT image examples also show different output changes when equal-cost perturbations are applied at different stages.Source: §4.2 · pp. 7–8

4.3 Local predictions and finite perturbation responses

• Local and finite DiT responses peak in the same windows for DC, DCT-4, and DCT-31, but finite-response peaks are 36.8%, 47.7%, and 87.9% of local predictions, respectively.The local predictions identify responsive windows but overestimate peak magnitudes at the tested finite amplitude.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 Feature responses relative to baseline fluctuations

• For JiT-B/L, DCT-16 has a smaller absolute response than DC but a larger normalized response; in ADM, DCT-16 has a smaller normalized response as well.The comparison is at the first sampling window and accounts for baseline fluctuations.Source: §4.4 · p. 8

• Each candidate caches the prediction at the start of one eight-step window and reuses it for the remaining seven steps, while other steps use fresh predictions.The setup divides 64 sampling steps into eight windows and treats cached-versus-fresh prediction differences as local update perturbations.Source: §5 · pp. 8–9
• The propagation-based predictor captures broad variation in image MSE across caching windows, although predicted and observed magnitudes can differ substantially.For JiT-B/L, predictions track observed magnitudes more closely, particularly in middle and later windows.Source: §5 · p. 9, Figure 4 · p. 9
• Across all four models, the propagation-based predictor achieves Spearman correlations of 0.9524–0.9762 with observed mean image MSE, substantially exceeding local-defect and path-cost baselines.The predictor identifies sampling windows where caching causes larger output errors.Source: §5 · p. 9

6 Conclusion

• Output responses are sums of propagated local perturbations, so downstream dynamics, cross-time accumulation, and observation shape measured effects beyond path cost.The framework separates changes in output distributions from changes in conditional trajectory distributions and characterizes how much local output information a selected feature mean captures.Source: §6 · pp. 9–10
• The framework is used to identify temporal and frequency-dependent sensitivity under equal path costs and caching windows with larger image deviations in the evaluated models.Source: §6 · p. 10

A Proofs and derivations

• The proofs derive the exact path-cost decomposition, propagation identity, local Fisher decomposition, feature-response bound, and main response identity.The proof setup denotes reference and perturbed trajectory distributions by P and Qε, respectively.Source: A Proofs and derivations · p. 13

A.1 Derivation of the path cost for local perturbations

• The path likelihood ratio factors into conditional transition ratios, whose log expectations sum to trajectory-level KL by the relative-entropy chain rule.Each conditional KL compares updates at the same history, averaged over histories reached by the perturbed process.Source: §A.1 · p. 13
• For Gaussian transitions with shared covariance, each local cost is the squared mean displacement weighted by inverse transition covariance.The covariance normalization terms cancel when the Gaussian kernels are compared.Source: §A.1 · pp. 13–14
• At finite perturbation strength, equal displacements cost more at lower noise levels because the path cost weights them by inverse covariance.For Λ_i = ν_iI, each contribution is ε^2∥u_i∥^2/(2ν_i), with expectation under the perturbed process.Source: §A.1 · p. 14

A.2 Proof of Proposition 1: Decomposition of Path and Output Changes

• The path KL decomposes into output KL and conditional-history KL averaged over the perturbed output law, so output KL lies between zero and path cost.The decomposition assumes a measurable output and an absolutely continuous perturbed trajectory law with finite KL.Source: §A.2 · p. 14

A.3 Proof of Proposition 2: Perturbation propagation

• First-order output changes accumulate transported local perturbations, whose alignment across steps is retained in cross-time terms.An adjoint recursion transports the final statistic's gradient backward to pair it with each local injection.Source: §A.3 · p. 15

A.4 Path Fisher information and output visibility

• The path score separates into an output-resolvable component and a residual, yielding path Fisher information as output Fisher information plus conditional variance.Coarsening the output cannot increase its Fisher information; the output-visible fraction is F_Y/F_path when F_path > 0.Source: §A.4 · p. 15

A.5 Proof of Proposition 3: Feature response and output information

• A feature mean response is its covariance with the output score and is bounded by feature variance times output Fisher information.This links the response of one statistic to the information retained in the output distribution.Source: §A.5 · p. 16

A.6 Proof of Theorem 1: Perturbation propagation and measured output response

• The response bound is attained by the centered, unit-variance output score, while a selected statistic measures only one projection of the response geometry.Optimizing over unit-variance observables recovers the full local output information when the class contains such an observable.Source: §A.6 · p. 16
• A selected feature's mean response can vanish when output changes are orthogonal to its centered feature, even if output Fisher information is positive.This limitation concerns the feature mean, not all changes in its distribution.Source: §A.6 · p. 16
• A smaller absolute feature response can correspond to a larger variance-relative response when the reference feature variance is sufficiently smaller.The normalized quantity B_f is unchanged by nonzero rescaling or shifting of the feature.Source: §A.6 · p. 16
• Predicting a finite mean shift by εχ_f requires the higher-order remainder to be negligible.The first-order identity alone does not ensure this finite-perturbation approximation.Source: §A.6 · p. 17

B Gaussian construction for equal-cost perturbations

• The construction fixes noise-normalized intervention strength while changing temporal alignment with downstream gains.The reverse Gaussian chain uses M = 24 steps, α = 0.9, and independent standard Gaussian innovations.Source: B Gaussian construction for equal-cost… · p. 17
• At ε = 2, the reinforcing control has endpoint/bridge KL (1.957579, 0.042421), while the cancelling control has (0, 2).The cancelling control is orthogonal to downstream gains, preserving the endpoint Gaussian law while changing the intermediate path law.Source: B Gaussian construction for equal-cost… · p. 17

C.1 Measuring output responses

• Responses are estimated by comparing outputs from +ε and −ε runs that share class, initial noise, step noise, and spatial template.The output difference is divided by 2ε; JiT and ADM use a DCT projection, while DiT uses decoded-image RMS response.Source: §C.1 · p. 18
• Frequency comparisons use baseline DCT-projection variance estimated from unperturbed images, with the same baseline variance across injection windows for each feature.Response-mean standard errors are computed from per-sample responses.Source: §C.1 · p. 18

C.2 Reference samplers and perturbation design

• Unit-norm DCT modes set the spatial perturbation directions, while each injection window has path Fisher cost one and path KL ε^2/2.The modes include k = 0, 1, 4, and 16 for all models, plus k = 31 for DiT.Source: §C.2 · p. 19

C.3 Models and sampling configurations

• The controlled experiments use public class-conditional ImageNet-256 checkpoints and compare eight injection windows over 128 stochastic steps at path Fisher Fpath = 1.Window comparisons hold classes, initial states, and stepwise noise fixed; reported variability includes class and noise variation.Source: Table 3 · p. 19, §C.3 · p. 19

C.4 Caching protocol and local defects

• Caching stores the prediction at the first step of a candidate window and reuses it for the next seven steps, omitting seven network evaluations.Cached and reference runs are paired with the same class, initial state, and noise at every step.Source: §C.4 · p. 19
• A first-order tangent recursion propagates local prediction defects along the reference trajectory to predict the final decoded-pixel response.The predictor includes the learned covariance derivative and propagates through the time-zero map, latent scaling, and VAE decoder.Source: §C.4 · pp. 19–20
• For DiT, the local cache defect includes changes to both the transition mean and log covariance from reusing the complete noise-and-variance prediction.For JiT and ADM, the local update defects are update differences at the fixed reference state.Source: §C.4 · p. 20

C.5 Caching error prediction and evaluation

• The caching evaluation compares predicted-response rankings across windows with observed mean image MSE, alongside local-state and path-cost baselines.The path-cost baseline is a local approximation, not the finite path KL under caching.Source: §C.5 · p. 20

C.6 Image visualization protocol

• Image-response visualizations use unclipped decoded pixels for differences and annotate full-image central-response RMS.The display clips images to [−1, 1], and the difference-map color scale saturates at the pooled 99.5th percentile.Source: §C.6 · p. 20

D.1 DiT image examples at equal path cost

• 19.88× and 11.23× larger central-response RMS in window 4 than window 8, respectively, for two DiT samples with equal path cost 0.32.Both examples use DCT-4 perturbations at ε = 0.8; the shared-scale difference maps show per-pixel RGB RMS relative to the reference.Source: §D.1 · pp. 20–21, Figure 5 · p. 21, Figure 6 · p. 21

D.2 Feature responses across sampling windows

• Across windows, baseline variance is fixed for each feature, so temporal variation in absolute mean response comes from the response itself; baseline fluctuations shape normalized responses across features.Tables 4 and 5 extend the feature measurements across all eight sampling windows.Source: §D.2 · p. 21, Table 4 · p. 22, Table 5 · p. 22