Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss

Yucheng Zhou; Hao Li; Jianbing Shen

Source-linked AI summary

Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss

Yucheng Zhou, Hao Li, Jianbing Shen

arXiv:2602.07022v1Feb 2, 2026eess.IVcs.CVcs.LG

TL;DR

The paper studies how autoregressive image generation with diffusion loss compares with conditional diffusion, focusing on condition errors and inconsistency. It analyzes patch denoising and autoregressive condition refinement, introduces an Optimal Transport method formulated as Wasserstein Gradient Flow, and reports superior experimental results over competing methods.

Problem
The paper addresses the need to compare conditional diffusion with autoregressive image generation using diffusion loss and to resolve condition inconsistency during autoregressive condition generation.
Method
The paper theoretically analyzes patch denoising and autoregressive condition refinement, then formulates Optimal Transport-based condition refinement as a Wasserstein Gradient Flow.
Results
Experiments show superiority over diffusion and autoregressive models with diffusion loss, while the theory establishes convergence toward the ideal condition distribution.
Takeaways & Limitations
Patch denoising mitigates condition errors, autoregressive refinement causes condition influence to decay exponentially, and Wasserstein Gradient Flow addresses condition inconsistency.
Takeaways & Limitations
The theoretical patch-denoising analysis assumes a Markov reverse diffusion process and sufficiently small conditional-distribution variance, approximated by a common σ2.

Abstract

from arXiv · show

Recent studies have explored autoregressive models for image generation, with promising results, and have combined diffusion models with autoregressive frameworks to optimize image generation via diffusion losses. In this study, we present a theoretical analysis of diffusion and autoregressive models with diffusion loss, highlighting the latter's advantages. We present a theoretical comparison of conditional diffusion and autoregressive diffusion with diffusion loss, demonstrating that patch denoising optimization in autoregressive models effectively mitigates condition errors and leads to a stable condition distribution. Our analysis also reveals that autoregressive condition generation refines the condition, causing the condition error influence to decay exponentially. In addition, we introduce a novel condition refinement approach based on Optimal Transport (OT) theory to address ``condition inconsistency''. We theoretically demonstrate that formulating condition refinement as a Wasserstein Gradient Flow ensures convergence toward the ideal condition distribution, effectively mitigating condition inconsistency. Experiments demonstrate the superiority of our method over diffusion and autoregressive models with diffusion loss methods.

1 INTRODUCTION

The paper compares autoregressive image generation with diffusion loss against conditional diffusion, showing how patch denoising and autoregressive condition generation reduce condition errors. It also proposes Optimal Transport-based refinement to mitigate condition inconsistency and proves convergence under Wasserstein Gradient Flow.

Motivation: The study addresses the lack of comparative analysis between conditional diffusion and autoregressive image generation with diffusion loss.Prior work identified quantization errors in VQ-based generation and introduced diffusion loss as an alternative.
Theoretical analysis: Patch denoising optimization theoretically mitigates condition errors and stabilizes the condition distribution under Markov and Gaussian-noise assumptions.The analysis also characterizes attenuation of the conditional probability gradient as the condition stabilizes.
Theoretical analysis: The condition’s influence on the outcome decays exponentially toward a stationary value as autoregressive iterations progress.The influence is quantified by the gradient norm of the conditional probability distribution.
Condition refinement: Condition inconsistency arises when autoregressive conditions accumulate extraneous information that hinders optimal patch generation.The proposed refinement method uses Optimal Transport theory and Wasserstein Gradient Flow to converge toward the ideal condition distribution.
Experiments: Experiments on ImageNet show the method’s superiority over diffusion and autoregressive models with diffusion loss, with analysis supporting its effectiveness in condition refinement.The experiments compare the method against other diffusion and autoregressive diffusion-loss models.

2 PRELIMINARIES

The preliminaries describe diffusion models as Markov processes that transform data into Gaussian noise and reverse denoising processes that reconstruct data. They also introduce autoregressive models, which sequentially predict each element conditioned on preceding elements.

Diffusion Modeling: Diffusion models use a forward Markov chain of Gaussian transitions to transform data x0 into Gaussian noise xT.The forward process is followed by a reverse denoising process.
Diffusion Modeling: The reverse diffusion process reconstructs x0 from xT by modeling pθ(x_t−1|x_t) as a Gaussian distribution.Its mean and covariance are predicted by a neural network.
Diffusion Modeling: Because the true reverse posterior is intractable, training approximates it using q(x_t−1|x_t,x0).The model is trained through a variational lower bound that can be reformulated as score matching.
Autoregressive Modeling: Autoregressive models sequentially predict each data element by conditioning on all preceding elements.They assume each x_i depends only on prior points x_<i, and generation proceeds from x1 through xn.

3 THEORETICAL ANALYSIS ON AUTOREGRESSIVE IMAGE MODELING WITH DIFFUSION LOSS

This section compares conditional diffusion with autoregressive modeling using diffusion loss, showing how evolving patch conditions address conditional errors. It develops theoretical error measures, refinement dynamics, and convergence-related results under stated assumptions.

Model comparison: Conditional diffusion uses one static condition for every denoising step and image patch, whereas autoregressive diffusion loss evolves conditions from preceding conditions.The autoregressive formulation guides each patch with its dynamic condition and uses generated patches to predict subsequent conditions.
Conditional error analysis: Conditional score matching loss upper-bounds standard score matching loss, so minimizing the conditional loss constrains the standard loss from above.The analysis decomposes score-matching losses to isolate contributions from conditioning.
Conditional error analysis: The conditional error term ϵc measures the change in the expected squared norm of the true score caused by conditioning relative to the unconditional case.The simplified term focuses directly on the magnitude of the conditional true score.
Conditional control analysis: The conditional control term adds σ2_t∇x_t log p(c|x_t) to the reverse-process mean, and the expected score-norm difference isolates its contribution.The analysis defines f(c_i) as the squared norm of the conditional score and uses its expected difference from the unconditional score norm.
Condition refinement: Patch-related autoregressive updates c_i+1 = T(c_i) refine conditions during denoising and are theoretically linked to improved conditional generation quality.The condition update is modeled as a discrete-time Markov chain under Markov, Gaussian, and small-variance assumptions.
Condition refinement: Under the autoregressive-process assumptions, the conditional probability gradient norm has a descent result with constants M > 0 and β ∈ (0, 1).The state-vector construction also yields a strong Markov chain for the condition sequence.

4 AUTOREGRESSIVE CONDITION OPTIMIZATION

Autoregressive condition generation can accumulate extraneous information, creating condition inconsistency that perturbs denoising. The proposed OT-based refinement formulates correction as a Wasserstein gradient flow that converges toward the ideal condition distribution.

4.2 OPTIMAL TRANSPORT FOR CONDITION REFINEMENT VIA WASSERSTEIN GRADIENT FLOW: Optimal Transport is used to correct distributional drift because it measures geometric transformation cost and supports a Wasserstein Gradient Flow formulation.The approach is presented as a principled solution to condition inconsistency while preserving valid semantic information.
4.1 CONDITION INCONSISTENCY IN AUTOREGRESSIVE GENERATION: Autoregressive conditions accumulate extraneous information beyond the minimal sufficient subspace, perturbing the conditional distribution and denoising score.The resulting deviation from the ideal denoising trajectory is O(∥η_i∥2), and accumulation across patches yields total inconsistency.
4.2 OPTIMAL TRANSPORT FOR CONDITION REFINEMENT VIA WASSERSTEIN GRADIENT FLOW: The refinement process minimizes an energy functional incorporating inverse-process regularization and admits an implementable JKO iteration scheme.The update balances proximity to previous iterates with movement toward the ideal condition and inverse-process consistency.
4.2 OPTIMAL TRANSPORT FOR CONDITION REFINEMENT VIA WASSERSTEIN GRADIENT FLOW: Under the stated assumptions, the Wasserstein gradient flow converges to the ideal condition distribution from any initial distribution.The proof uses contractive properties of the flow for convex energy functionals, with contraction rate ρ < 1.
4.2 OPTIMAL TRANSPORT FOR CONDITION REFINEMENT VIA WASSERSTEIN GRADIENT FLOW: The refinement monotonically reduces Wasserstein distance to the ideal condition distribution, with smaller ρ indicating faster convergence.The inverse process regularization directly counters extraneous information accumulation.

5 EXPERIMENTS

Experiments evaluate the method on ImageNet conditional generation at 256 × 256 and 512 × 512 resolutions, including model-size scalability and denoising behavior. The method achieves strong quantitative performance and remains effective as model capacity and resolution increase.

5.2 PERFOMANCE COMPARISON: The method achieves a best FID of 1.52 on ImageNet 256 × 256 conditional generation, compared with 1.55 for MAR, 1.58 for MDTv2-XL/2, and 1.73 for DiffiT.It also achieves the highest Inception Score and reaches 1.31 when based on MAR.
5.3 SCALABILITY ANALYSIS: Across 208M, 479M, and 943M parameter models, the method consistently outperforms MAR, with the performance gap widening at larger scales.The evaluation uses ImageNet 256 × 256 and tests whether condition optimization benefits larger capacities.
5.3 SCALABILITY ANALYSIS: On ImageNet 512 × 512, the method achieves FID 1.58 versus 1.73 for MAR, indicating effective condition refinement at higher resolution.The evaluation uses a model of approximately 481M parameters.
5.2 PERFOMANCE COMPARISON: Denoising analysis reports higher SNR for the method than the baseline, with the gap widening in later denoising stages.Both methods reduce noise intensity as denoising progresses.

6 CONCLUSION

The paper concludes that autoregressive patch denoising stabilizes conditions, while OT-based refinement addresses condition inconsistency and improves patch generation. Experiments support the method’s effectiveness, though large-scale empirical validation remains outstanding.

Figure 3 compares SNR and noise intensity during image-space denoising after VAE decoding for the proposed method and a baseline.
Patch denoising mitigates condition errors, stabilizes the condition distribution, and enhances generation quality.
Autoregressive condition generation further refines conditions by exponentially reducing the influence of condition errors.
OT-based condition refinement is formulated as a Wasserstein Gradient Flow that converges toward the ideal condition distribution.
The experimental evaluation was not conducted on large-scale models because validation required substantial computational resources.

C PROOF OF THEOREM 1 (CONDITIONAL SCORE MATCHING UPPER BOUND)

This proof derives a conditional score-matching upper bound using Bayesian score decomposition, Jensen’s inequality, and expectation identities. The equality condition corresponds to independence between the noisy variable and the condition.

The proof begins with foundational lemmas for Bayes’ theorem applied to conditional scores and Jensen’s inequality for convex functions.
The marginal distribution is expressed by conditioning on c, then differentiated under conditions justified by the Dominated Convergence Theorem.
Bayes’ theorem decomposes the marginal score into a conditional score component and the marginal score.
Jensen’s inequality bounds the unconditional loss by an expectation of conditional score-matching errors.
Equality holds when the conditional score is constant almost surely, corresponding to statistical independence between x_t and c.

D.2 DERIVATION OF ERROR DIFFERENCE ϵc (DEFINITION 1)

The derivation decomposes the difference between conditional and unconditional score-matching losses. It identifies the additional conditional score energy as the excess induced by conditioning.

The loss-difference analysis subtracts the unconditional objective from the conditional objective after expanding both squared norms.
Cross-terms cancel through the tower property of expectation and the identity relating marginal and conditional scores.
The resulting expression quantifies the excess score energy induced by conditioning.
A law-of-total-variance decomposition separates intrinsic unconditional score energy from the additional energy associated with conditioning.

E CONDITIONAL CONTROL TERM UNIQUENESS PROOF

This proof establishes the uniqueness of the conditional control term by decomposing conditional scores with Bayes’ rule and showing that the cross-term vanishes. The remaining contribution isolates conditional control.

The proof proceeds through three stages: Bayesian score decomposition, cross-term cancellation, and variance propagation.
Bayes’ rule decomposes the conditional score into the gradient of log p(c|x_t) and the marginal score.
The cross-term vanishes because the conditional expectation of the gradient of log p(c|x_t) is zero.
The reverse-process parameterization supplies an explicit σ_t^2 multiplier in the conditional mean adjustment, isolating the conditional control term’s contribution.

F PROOF OF CONDITION REFINEMENT VIA PATCH DENOISING (PROPOSITION 1)

The proof analyzes how patch denoising affects the conditional distribution by examining its conditional mean, variance, and gradient. Under stationary refinement, the mean stabilizes, the variance remains unchanged, and gradient magnitude decays.

The conditional probability density is analyzed through its conditional mean and conditional variance, which determine gradient behavior.The proof separately examines the mean and variance before assessing the squared gradient norm.
As the condition reaches its stationary distribution, the conditional mean converges to a constant µstable.The mean of c converges as the autoregressive condition iterates.
The conditional variance is determined by the joint covariance structure and remains unchanged in the stationary distribution.Because the stationary variance does not depend on c, it is constant during refinement.
As x approaches the conditional mean while variance stays constant, the conditional probability gradient attenuates over iterations.The resulting reduction in gradient magnitude indicates diminishing conditional influence.

G PROOF OF DESCENT OF GRADIENT NORM IN AUTOREGRESSIVE PROCESS (THEOREM 2)

The proof represents the autoregressive condition process as a Markov chain and establishes geometric convergence toward a stationary distribution. This convergence yields an exponential decay estimate for the conditional log-likelihood gradient norm.

State-space representation: A state vector converts the autoregressive process into a first-order vector-valued Markov representation.The conditional distribution of c_i+1 then depends only on c_i.
Stability assumptions: A spectral radius below 1 ensures stability of the represented autoregressive process.The stability condition is imposed on matrix A.
Gradient analysis: The log-likelihood gradient is decomposed into stationary and non-stationary terms, which are bounded using Lipschitz and norm inequalities.The proof separately estimates both terms before combining them.
Gradient descent: The combined estimates establish exponential decay of the gradient norm uniformly for all x_t ∈ X.The decay estimate holds as the autoregressive process iterates.
Stationary convergence: Geometric ergodicity implies that the condition state converges geometrically toward the stationary mean cπ.The distance ∥c_i − cπ∥ is bounded by a geometrically decaying term.

L.1 IMPLEMENTATION DETAILS •

The implementation combines latent-distribution estimation, entropy-regularized optimal transport, stochastic optimization, and stability controls. These components support the framework’s theoretical convergence guarantee.

Target distribution estimation: Generated latents are tracked with an exponential moving average using ν = 0.1 and a buffer size B = 2048.The implementation uses these settings for target distribution estimation.
Optimal transport: Entropy regularization is adaptively scheduled during Sinkhorn iterations.The schedule adjusts ϵ during the OT computation.
Stochastic optimization: Adam optimization is used with a learning-rate warmup controlled by k_warm = 100.The implementation specifies stochastic optimization and warmup behavior.
Convergence and stability: The algorithm maintains the convergence guarantee through Sinkhorn error bounds and gradient clipping.Sinkhorn approximation error is bounded, while clipping ensures Lipschitz continuity.
Regularity conditions: Bounded second-order derivatives on a fixed interval imply bounded first-order derivatives and a bounded conditional density.These bounds provide regularity conditions used in the theoretical analysis.