Source-linked AI summary
The Noise-Transmission Burden of Diffusion Network: Why Image Prediction Can Outperform Noise Prediction
Gyeonghun Kang, Pengxi Liu, Bingshuo Qian, Xiang Cheng
TL;DR
Finite diffusion networks can perform differently across prediction targets even when their population-optimal denoisers are equivalent. The paper analyzes the information noise prediction must retain, derives bottleneck and width results, and tests them with JiT on ImageNet. The results connect denoising floors and their preconditioning response to generation differences, while relating low-noise widening gains to the spread of nonlinear corrections.
Problem
Prior work links prediction-target preference to data geometry and architecture, but how architectural constraints on retained input information affect denoising remains unquantified.
Source: §1 · pp. 1–2Method
The paper analyzes the clean-image estimate induced by noise prediction, derives bottleneck and width results, and evaluates denoising and generation with JiT experiments on ImageNet.
Source: §1 · p. 2Results
A rank-r input bottleneck imposes a σ_t^2(d-r) noise-prediction denoising floor, EDM attenuates it at high noise, exact preconditioning cancels it, and correction covariance spectra characterize width needs above patch dimension.
Source: §1 · p. 2Takeaways & Limitations
Denoising provides an account of generation-quality differences across prediction targets, preconditioning choices, and model widths within the studied settings.
Source: §1 · p. 2The width characterization assumes exact affine recovery of the noisy patch, a representation strategy rather than a necessary property of all successful noise predictors.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Clean-image prediction ($x$-prediction) and noise prediction yield equivalent population-optimal denoisers, yet can perform very differently in finite diffusion networks. For noisy input $Y_t$ with noise standard deviation $\sigma_t$, a noise predictor $f_{\varepsilon,t}$ induces the clean-image estimate $\widehat X_1=Y_t-\sigma_t f_{\varepsilon,t}(Y_t)$. This reveals a noise-transmission burden: the predictor must preserve information needed to cancel the input corruption. Under Gaussian corruption of $d$-dimensional data, a linear input bottleneck of rank $r<d$ imposes a denoising-error floor $\sigma_t^2(d-r)$ even with unrestricted downstream computation, while the corresponding optimal risk for image prediction remains bounded. EDM attenuates this bottleneck contribution at high noise; exact preconditioning cancels it at every noise level. Without an input bottleneck, assuming exact affine recovery of the noisy patch from the model representation, we characterize how the covariance spectrum of corrections beyond linear denoising determines the width required above the patch dimension. ImageNet experiments with JiT show that measured denoising losses reflect the predicted bottleneck floor and its attenuation or cancellation through preconditioning. Further widening yields larger relative denoising gains at low noise, where corrections beyond linear denoising spread across more directions, consistent with our spectral characterization of the required model width. Together, these results provide a denoising-based account of practical generation-quality gaps across prediction targets, preconditioning choices, and model widths.
1 INTRODUCTION
The paper explains why equivalent population-optimal prediction targets can diverge in finite diffusion networks: accurate noise prediction must retain input information needed to cancel corruption. It links input bottlenecks, preconditioning, and model width to denoising accuracy through theory and ImageNet experiments.
1 INTRODUCTION
- The motivating gap is that prior work links prediction-target preference to data geometry and architecture but does not quantify how input information constraints affect denoising.The paper asks how bottlenecks and noise level shape error, how explicit input paths change it, and what width is needed beyond patch dimension.Source: §1 · pp. 1–2
- A noise predictor induces a clean-image estimate by subtracting its predicted noise contribution from the noisy input, creating a requirement to preserve cancellation-relevant information.The paper names this requirement the noise-transmission burden.Source: §1 · p. 2
- A rank-r linear input bottleneck creates a noise-prediction denoising-error floor of σ_t^2(d-r), even with unrestricted downstream computation, while image-prediction risk remains bounded.The floor requires only finite second moment for clean data and no manifold assumption.Source: §1 · p. 2
- EDM suppresses the bottleneck contribution at high noise, while exact preconditioning removes it at every noise level.In the reported experiment, FID changes from 412.11 for noise prediction to 52.28 with EDM and 9.15 with exact preconditioning, close to 9.13 for image prediction.Source: §1 · p. 2
- Without an input bottleneck, the covariance spectrum of corrections beyond linear denoising determines the additional width needed above patch dimension, assuming exact affine recovery of the noisy patch.Widening yields larger relative denoising gains at low noise, where nonlinear corrections spread across more directions.Source: §1 · p. 2
2 PROBLEM SETUP
The setup considers Gaussian corruption at a fixed interior noise level and compares image, noise, and general target prediction. Their optimal population predictors determine the same clean-image estimate, but finite networks may represent them differently because noise prediction must cancel corruption carried by the input.
2 PROBLEM SETUP
- The normalized observation is Y_t = X_1 + σ_tε, with ε standard Gaussian and independent of the clean image; the analysis fixes an interior time and treats high noise as σ_t → ∞.The underlying corruption path is X_t = a_tX_1 + b_tε, with positive deterministic coefficients.Source: §2 · p. 2
- Under squared loss, image and noise predictors are conditional-mean estimators, and either population-optimal predictor determines the same clean-image estimate as the other.The equivalence is at the population optimum; the section distinguishes it from finite-network representability.Source: §2 · p. 2
- For a finite network, noise prediction must supply the contribution that cancels Gaussian noise carried by the input, potentially requiring more capacity than directly estimating the clean image.The passage contrasts ambient Gaussian noise with clean images that may concentrate near low-dimensional structure.Source: §2 · pp. 2–3
- The setup extends to targets T_t = α_tX_1 + β_tε, including velocity prediction, and studies how target conversion and output preconditioning affect the induced clean-image estimate.Table 1 collects notation for direct target prediction at a fixed interior time.Source: §2 · p. 3, Table 1 · p. 3
- An input bottleneck occurs when model width or an intermediate patch embedding is narrower than the patch dimension, discarding noisy-input directions from subsequent computation.The section asks how this bottleneck affects recovery and denoising, how explicit input contributions help, and how much width is needed without a bottleneck.Source: §2 · p. 3
3 THEORY
The theory relates denoising accuracy to information retained from the noisy input and to how the network represents corrections beyond linear denoising. It derives bottleneck error floors, shows how output preconditioning changes them, and characterizes the extra width needed under affine recovery of the noisy patch.
3.1 INPUT BOTTLENECKS IMPOSE DENOISING-ERROR FLOORS ON NOISE PREDICTION
- A rank-r input bottleneck under noise prediction imposes a denoising-error floor of σ_t^2(d-r), while optimal image-prediction risk remains bounded across noise levels.The noise-prediction floor holds even with unrestricted downstream computation.Source: §3.1 · pp. 3–4
- The bottleneck arises when a linear input map discards noisy-image directions, whose Gaussian components remain inaccessible to any decoder and limit noise recovery.A narrow patch embedding can create this bottleneck even when the model width exceeds the intermediate embedding dimension.Source: §3.1 · pp. 3–4
3.2 OUTPUT PRECONDITIONING CONTROLS THE NOISE-TRANSMISSION BURDEN
- Exact preconditioning eliminates the discarded-noise contribution at every noise level, whereas EDM makes that contribution vanish at high noise and keeps optimal risk bounded.The preconditioned skip term determines how much intact noisy input is supplied directly.Source: §3.2 · pp. 5–6
- EDM’s raw network target approaches a scaled clean image at high noise, while exact preconditioning removes the discarded-noise contribution at every noise level.The paper relates this high-noise limit to JiT’s observation that EDM-style preconditioning improves over direct noise and velocity prediction but remains worse than image prediction in its setting.Source: §3.2 · p. 5
3.3 THE RESIDUAL BEYOND LINEAR DENOISING DETERMINES MODEL WIDTH
- Assuming the noisy patch is affinely recoverable from the representation, the best affine clean readout provides a baseline, and added width captures covariance directions of the residual beyond it.The residual can reflect nonlinear dependence on the noisy patch or information from other patches.Source: §3.3 · pp. 6–7
- Additional width reduces approximation error by capturing leading residual-covariance directions; zero approximation error requires width dp + rank(Cov(Rt)), so width 2dp always suffices in this representation class.Fewer extra coordinates are needed when residual variance concentrates in fewer directions.Source: §3.3 · p. 7
4 EXPERIMENTS
ImageNet experiments test how input bottlenecks, preconditioning, and model width affect denoising and generation. The results show bottleneck-related denoising floors, their attenuation or cancellation through preconditioning, and width-dependent gains consistent with residual-spectrum predictions.
4.1 PRECONDITIONING LOWERS DENOISING FLOORS UNDER INPUT BOTTLENECKS
- FID was 412.11 for noise prediction, versus 9.13 for image prediction; EDM variants reached approximately 52.28, while exact preconditioning reached 9.15.The comparison uses JiT-B/16 on ImageNet256 with a 768 →128 →768 bottleneck patch embedding.Source: §4 · p. 7, §4.1 · p. 8
- At t = 0.05, noise prediction’s clean-image MSE was 300.836 against a 300.833 floor; at t = 0.5, it was 0.83387 against 0.83333.EDM’s measured MSE at t = 0.5 was approximately 0.04747 against a 0.03333 floor, while at t = 0.05 it was approximately 0.1103 as its floor became negligible.Source: §4.1 · p. 8, Figure 2 · p. 8
- All 78 affine-probe scores stayed below the 1/6 ceiling; final-block noise recovery rose from 0.0833 at t = 0.95 to 0.1665 at t = 0.1.Figure 3 shows final-block recovery, which approaches but remains below the ceiling at high noise.Source: §4.1 · p. 9, Figure 3 · p. 9
4.2 WIDENING IMPROVES DENOISING AND AFFINE INPUT RECOVERY
- On ImageNet256, widening from 768 to 896 reduced FID from 404.03 to 93.11; width 1536 reached 54.71.Figure 4 compares generation quality against width, with a dotted reference at dp = 768.Source: §4.2 · p. 9, Figure 4 · p. 9
- At t = 0.5, widening from 768 to 896 reduced clean-image MSE from 0.01959 to 0.01597 and raised noisy-input recovery from 78.34% to 93.60%.At width 1536, MSE was 0.01524 and input recovery was 99.18%.Source: §4.2 · p. 9, Figure 5 · p. 9
- At t = 0.9, relative MSE reduction from width 768 grew from 9.8% at width 896 to 17.4% at width 1536, unlike the smaller further gain at t = 0.05.The residual’s 95%-variance direction count increased from 6 at t = 0.1 to 416 at t = 0.9, consistent with low-noise corrections spanning more directions.Source: §4.2 · p. 10
5 CONCLUSION
The width characterization has explicit scope limits: it assumes affine recovery of the noisy patch and does not explain every widening-related generation gain. The theory also treats noise levels separately and does not establish how denoising errors affect sampling outcomes.
5 CONCLUSION
- The width characterization assumes exact affine recovery of the noisy patch, a representation strategy that is not necessary for all successful noise predictors.Within the unrestricted representation class, width 2dp suffices for the population-optimal denoiser; gains beyond that are not characterized by the denoising-risk theory.Source: §5 · p. 10
- The theory treats each noise level separately, leaving open how parameter sharing across noise levels changes attainable tradeoffs.Source: §5 · p. 10
- The analysis characterizes denoising accuracy but does not establish how errors propagate through sampling or change generated and target distributions.Source: §5 · p. 10
REPRODUCIBILITY STATEMENT
The authors provide code, numerical results, and scripts to regenerate tables and Figures 2–9. Appendices document proofs and experimental protocols.
REPRODUCIBILITY STATEMENT
- The repository provides training and evaluation code, saved numerical results, and scripts to regenerate all tables and Figures 2–9.It also includes run configurations, checkpoint identities, and instructions for supplying external data and model weights.Source: REPRODUCIBILITY STATEMENT · p. 11
- Appendix A contains proofs of the theorems and corollaries, including supporting lemmas and derivations.Source: REPRODUCIBILITY STATEMENT · p. 11
- Appendix B documents training and generation settings, evaluation protocols, probe procedures, residual-spectrum estimation, and protocol qualifications.Source: REPRODUCIBILITY STATEMENT · p. 11
AI USE STATEMENT
The appendix proves bottleneck risk bounds for noise and preconditioned targets, then develops an affine denoising baseline and a feature decomposition under exact affine recovery. It also records the authors’ use of generative AI tools.
AI USE STATEMENT
- The authors report using generative AI for literature review, presentation, code, experiments, proofs, feedback, and editing, while retaining responsibility for the paper.They say they independently reviewed and verified AI-assisted results and made final scientific and methodological decisions.Source: AI USE STATEMENT · p. 11
A.1.1 PROOF OF THEOREM 1
- Theorem 1’s proof separates noise-recovery risk into a discarded-direction term of d−r at every noise level and a retained-direction term.The retained term is bounded using projected clean-image risk; the discarded Gaussian components are independent of the bottleneck observation.Source: §A.1.1 · p. 14
A.2.1 PROOF OF THEOREM 3
- Theorem 3’s proof gives the discarded-noise contribution as (βt − cskipσt)^2(d−r), alongside two nonnegative clean-image uncertainty terms.The formula makes the bottleneck contribution depend on the target and skip coefficient.Source: §A.2.1 · pp. 14–15
A.2.2 CONVERGENCE OF EDM’S RAW NETWORK TARGET
- EDM’s raw target converges in mean square to scaled image prediction as σt grows.The first expansion coefficient tends to 1/σdata and the second tends to zero.Source: §A.2.2 · p. 15
A.2.3 PROOF OF COROLLARY 4
- With bounded converted skip coefficients, the clean-image estimate’s high-noise expansion has an O(1) remainder.The proof applies Theorem 3 to the converted image-target coefficients.Source: §A.2.3 · p. 15
A.3.1 DERIVATION OF THE BEST AFFINE DENOISING BASELINE
- The best affine denoising baseline is the best affine approximation to the population-optimal denoiser, with coefficients determined by clean-patch moments and noise level.Under Assumption 1, this baseline is available as an affine readout of the representation.Source: §A.3.1 · pp. 15–16
A.3.2 FEATURE DECOMPOSITION UNDER AFFINE INPUT RECOVERY
- Under affine recovery, the representation separates into a linear function of the noisy patch and dh−dp additional feature coordinates.The additional coordinates lie in the kernel of the affine recovery map.Source: §A.3.2 · p. 16
A.3.3 PROOF OF THEOREM 5
Theorem 5’s proof separates exact noisy-patch recovery from additional features, whose output directions determine how much denoiser residual variance can be captured. The minimum approximation error is attained by retaining leading residual-covariance directions.
A.3.3 PROOF OF THEOREM 5
- After affine recovery, the dh−dp extra features can affect the clean readout only within an output subspace of dimension at most dh−dp.The residual component outside that subspace cannot be canceled by the additional features.Source: §A.3.3 · pp. 16–17
- The uncaptured residual variance is bounded below by the covariance spectral tail after the largest min{k, dp} eigenvalues, where k=dh−dp.The proof bounds captured variance by the sum of the largest eigenvalues available to a k-dimensional subspace.Source: §A.3.3 · pp. 16–17
- The minimum approximation error is attained by storing the noisy patch and projecting the residual onto its leading covariance eigenvectors.The first dp coordinates recover the patch and provide the affine baseline; remaining coordinates capture residual directions.Source: §A.3.3 · p. 17
- The spectral tail gives the smallest excess clean-image loss above irreducible Bayes risk within the stated representation class.The Bayes-risk term is independent of the representation.Source: §A.3.3 · p. 17
A.3.4 PROOF OF COROLLARY 6
Corollary 6 converts the residual-covariance spectral tail into a minimum width for a chosen approximation tolerance.
A.3.4 PROOF OF COROLLARY 6
- For zero approximation error, the required width is dp + rank(Cov(Rt)); width 2dp always suffices in this representation class.For a fixed tolerance, the minimum width is dp plus the smallest k whose residual spectral tail is at most that tolerance.Source: §A.3.4 · p. 18
B EXPERIMENTAL PROTOCOLS AND ADDITIONAL RESULTS
The appendix specifies the held-out evaluation protocol for its denoising, probe, and residual-spectrum diagnostics.
B EXPERIMENTAL PROTOCOLS AND ADDITIONAL RESULTS
- Fixed-time denoising, probe, and residual-spectrum evaluations use the official ImageNet validation split, held out from backbone training.Fitting, tuning, and test images are disjoint within each diagnostic.Source: B EXPERIMENTAL PROTOCOLS AND… · p. 18
B.1.1 TRAINING, PRECONDITIONING, AND GENERATION
The appendices document training and evaluation protocols, then probe learned preconditioning, bottleneck denoising losses, and layerwise noise recovery. The measurements include a high-noise loss floor and learned skip coefficients that approach exact preconditioning over part of the noise range.
B.1.1 TRAINING, PRECONDITIONING, AND GENERATION
- Training used 12 Transformer blocks, width 768, and effective batch size 1024, with a single run per configuration and endpoint clamps in the implemented image-loss conversion.Generation used Heun’s method with 50 steps and 10,000 images; FID measurements are single-run results without run-to-run uncertainty estimates.Source: §B.1.1 · pp. 18–19
- The additive shortcut leaves the clean-image skip at t, giving a discarded-noise contribution of (1 − t)^2(d − r).The learned branch receives AXt with cin = t.Source: §B.1.1 · p. 18, Table 2 · p. 18
B.1.2 LEARNED VERSUS EXACT PRECONDITIONING COEFFICIENTS
- Across 0.05 ≤ t ≤ 0.5, all three learned variants have |c̃skip| < 0.0045; at t = 0.9, their values are 0.00566, 0.899, and 0.834 for Learned 1, Learned 2, and Learned 3.The output coefficient can differ in sign and scale because the learned branch can absorb those changes.Source: Figure 6 · p. 19, §B.1.2 · p. 19
B.1.3 FIXED-TIME DENOISING LOSSES
- At t = 0.05, noise-prediction MSE is [300.794, 300.879] and contains the 300.833 predicted floor, while image-prediction and exact-preconditioning intervals are [0.10810, 0.11223] and [0.10813, 0.11229].The evaluation used 2,000 common validation images without guidance.Source: Figure 7 · p. 20, §B.1.3 · p. 20
B.1.4 LAYERWISE NOISE-RECOVERY PROBES
- All layerwise affine noise-recovery scores are below 1/6, using separate affine noise decoders for each layer and evaluation time.The scores include prediction bias and use finite-test noise-variance normalization.Source: §B.1.4 · p. 20, Table 3 · p. 21
B.2.1 MODEL CONFIGURATIONS AND GENERATION RESULTS
The appendix describes matched width, input-recovery, and denoising evaluations at two ImageNet resolutions. Its residual-spectrum measurements show a much larger 95% direction count at low noise, while the probes and single-run generation results have explicit scope limits.
B.2.1 MODEL CONFIGURATIONS AND GENERATION RESULTS
- The width sweep compares noise predictors from dp to 2dp and image predictors including widths below dp, using 16 × 16 patches at 256 resolution and 8 × 8 patches at 128 resolution.Generation reports single-run FID-10K results without run-to-run uncertainty estimates.Source: §B.2.1 · p. 21, Figure 8 · p. 22
B.2.2 AFFINE INPUT-RECOVERY PROBES AND CONTROLS
- At t = 0.5, affine input-recovery probes measure finite-sample accessibility, not exact recovery, its necessity for noise prediction, or a causal effect on denoising.Image-prediction controls show that affine input preservation is not a universal requirement for accurate denoising.Source: §B.2.2 · p. 22, Figure 9 · p. 23
B.2.3 DENOISING LOSSES ACROSS WIDTHS AND NOISE LEVELS
- The width tables report within-target denoising comparisons, which do not establish an ordering between trained image and noise predictors.The tables give per-coordinate native clean-image MSE at each resolution using common images and corruptions across widths.Source: §B.2.3 · p. 23, Table 4 · p. 23, Table 5 · p. 23
B.2.4 ADM RESIDUAL SPECTRA AND REFERENCE CHECKS
- At ImageNet256, the residual’s 95% direction count is 5–6 at t = 0.1 and 388–416 at t = 0.9; the count measures relative concentration, while absolute approximation tolerance also depends on residual magnitude.Counts vary across five patch positions, and finite-sample checks preserve the qualitative contrast while changing exact low-noise counts.Source: §B.2.4 · p. 24
C RELATED LITERATURE
Related work links prediction-target preferences to data geometry and architecture, while leaving open how input information loss affects denoising and how much width preserves that information. The paper positions its input-bottleneck bounds, width characterization, and preconditioning analysis as complements to these earlier results.
C RELATED LITERATURE
- Prior studies connect data geometry to target choice, including a jointly learned clean-data/noise mixture, and show that architectural locality and intrinsic dimension can also change which target performs best.Jin & Wang (2026) propose k-Diff; Gagneux et al. (2026) report small-patch Transformers favor velocity prediction and larger-patch Transformers favor image prediction in their tested settings.Source: C RELATED LITERATURE · p. 25
- Earlier output-rank bounds constrain prediction directions, whereas this paper studies denoising error from information discarded at the input and width needed to retain noisy inputs.The width result assumes exact affine recovery of the noisy patch.Source: C RELATED LITERATURE · p. 25
- Prior linear-denoising observations motivate retaining an affinely accessible noisy patch so additional representation coordinates can encode corrections beyond the best affine estimate.The analysis formalizes exact affine recovery of the noisy patch through a readout of the representation.Source: C RELATED LITERATURE · pp. 25–26
- Related analytic input paths and EDM-style output preconditioning provide context for the paper’s analysis of how preconditioning changes attainable risk under architectural information constraints.The paper distinguishes EDM’s high-noise attenuation from exact cancellation at every noise level.Source: C RELATED LITERATURE · p. 26