Scale-Wise VAR is Secretly Discrete Diffusion

• Prior autoregressive diffusion formulations mask tokens in a fixed order, while visual generation lacks an explicit degradation signal and can underperform with diffusion-style training.The introduction also identifies an inductive bias: early generated pixels or regions lack information about the image’s distribution or semantics.Source: §1 · p. 2
• Diffusion models generate high-quality images through iterative denoising but are slow and computationally demanding at larger scales; VAR instead predicts progressively higher-resolution scales.VAR has also shown improved generation quality in terms of FID as model parameters increase.Source: §1 · p. 2
• SRDD combines a Markovian next-scale VAR formulation with a discrete-diffusion perspective, retaining strong scaling behavior as model size increases.It conditions on the immediately previous scale rather than all earlier scales and aims to replace handcrafted design choices with theory-grounded ones.Source: §1 · p. 2, Figure 2 · p. 2
• SRDD works better with classifier-free guidance, token resampling, and distillation, and improves zero-shot super-resolution, inpainting, and outpainting over VAR.The introduction points to Figure 1 for these results.Source: §1 · p. 2
• The authors present the diffusion-based explanation as a potential guide for design choices when scaling language models toward joint visual-language generation.They describe this as a possibility opened by the paper’s explanatory perspective.Source: §1 · p. 3

• VAR generates all tokens at one resolution jointly, then refines progressively to higher resolutions, conditioning each scale on all previously generated scales.The discrete tokens come from a multi-scale VQ-VAE, and residuals across resolutions reconstruct the image.Source: §2 · p. 3
• Discrete diffusion transforms categorical states through transition matrices and learns a reverse process conditioned on the degradation level.The transition matrix determines the degradation pattern; training uses cross-entropy to predict categorical distributions.Source: §2 · p. 3
• A Markovian diffusion formulation can train the model to reconstruct the preceding state directly from the current degraded state.The background describes the model as iteratively restoring degraded samples toward the real-data token distribution.Source: §2 · pp. 3–4
• Prior work identified redundant conditioning on all preceding scales, but lacked a theoretical explanation for why Markovian VAR variants perform better.The paper connects the Markovian formulation to discrete diffusion as that explanation.Source: §2 · p. 4, Figure 4 · p. 4

3.1 AUTOREGRESSIVE MODELS AS DISCRETE DIFFUSION MODELS.

• Autoregressive generation is a discrete-diffusion special case when a deterministic forward process masks tokens one by one in fixed order, reducing the only nontrivial divergence to standard cross-entropy.Source: §3.1 · p. 4

3.2 RETHINKING VAR VARIANTS THROUGH THE DISCRETE DIFFUSION LENS

• VAR’s current restoration scale is embedded with the class embedding, and under deterministic transitions with a fixed start token its loss matches VAR’s cross-entropy up to scaling when conditioned on the previous scale.Source: §3.2 · p. 5
• At each scale, VAR predicts the residual from the upsampled previous scale; SNR rises with scale, so progressive downsampling can be viewed as deterministic diffusion.The scale-dependent downsampling maps the original image to coarser, noisier resolutions.Source: §3.2 · p. 5
• Markovian attention conditions VAR on the previous scale alone; with fixed start tokens and transitions, this defines SDD, which outperforms VAR across multiple datasets and has a higher ELBO.The paper frames this variant as an efficient discrete-diffusion formulation.Source: §3.2 · pp. 4–5

3.3 HOW TO IMPROVE THE GENERATION PERFORMANCE AND EFFICIENCY

• Unlike naive VAR’s inconsistent ad-hoc CFG behavior, Markovian SDD exhibits a CFG behavior pattern and can achieve higher performance across CFG values.Source: §3.3 · p. 6
• SRDD resamples tokens predicted below 0.01 probability at each resolution and can add multiple sampling steps per scale for deeper refinement.Source: §3.3 · p. 6
• Progressive distillation starts from pretrained SDD and skips scales, upsampling previous-resolution latents to increase the SNR gap between consecutive scales.Source: §3.3 · p. 6

4.2 METHOD-WISE ANALYSIS

• At presample = 0.01, five MR iterations yield FID 16.76 at cfg 5.0 and 16.81 at cfg 3.5, while more aggressive thresholds mildly regress FID.The reported setting refines about 65% of tokens per pass, leaving 35% of high-confidence tokens to guide attention.Source: §4.2 · p. 8
• With presample fixed at 0.01, most quality gains occur in the first 15–25 refinement passes, after which FID curves flatten on MiniImageNet and SUN.The authors suggest most tokens reach the acceptance threshold within that range.Source: §4.2 · p. 8
• Simple resampling recovers high-frequency details, while confidence-aware MR progressively restores boundaries and textures and suppresses background noise.The qualitative comparison describes MR as producing the sharpest, most faithful images among the illustrated settings.Source: §4.2 · p. 8, §4.4 · p. 10
• For SDD, FID falls to 17.99 and IS peaks at 63.28 at cfg 5; VAR instead worsens from FID 20 to 27 and IS 60 to 51 as guidance increases.The authors attribute VAR’s decline under strong guidance to over-conditioning without an explicit noise schedule; SR and MR maintain near-optimal performance at cfg 6–10.Source: §4.2 · pp. 8–9
• A six-scale student preserves similar FID/IS to the ten-scale teacher while cutting inference cost by 20%; SDD is also reported to deliver a 1.75× speedup and 3× lower memory use.The pruning schedule retains the two highest scales, which encode high-frequency details.Source: §4.2 · p. 9

4.3 ZERO-SHOT PERFORMANCE

• On zero-shot editing, SDD improves in-painting LPIPS from 0.26 to 0.23 and FID from 29.92 to 28.79, with gains also reported for out-painting and super-resolution.Evaluation follows RePaint on 300 AFHQ validation images, without task-specific fine-tuning.Source: §4.3 · pp. 9–10

4.4 ABLATION STUDY

• Replacing causal masking with Markovian masking reduces memory cost and improves visual quality on all four benchmarks; MiniImageNet FID drops from 21.01 to 18.03.The reported qualitative comparison describes sharper object boundaries and fewer artifacts.Source: §4.4 · p. 10
• 20.2% and 15.6% FID improvements over VAR on MiniImageNet and SUN, respectively, accompany IS gains ranging from +6.7% to +31.1% with token mask resampling.Unconditional FFHQ reaches an IS of 4.05; MR selectively sharpens salient regions without over-sharpening artifacts.Source: §4.4 · p. 10

5 CONCLUSION

• SDD is mathematically equivalent to a structured discrete diffusion process, and the authors report faster convergence, lower inference cost, and improved zero-shot performance across multiple benchmarks.The conclusion says the perspective permits use of classifier-free guidance, token resampling, and scale distillation.Source: §5 · p. 10

6 ETHICS STATEMENT

• The study uses publicly available Mini-ImageNet, SUN397, FFHQ, and AFHQ datasets and involves no human subjects or private data.The authors state that the work is intended to advance scientific understanding and efficiency of visual generation, while acknowledging possible misuse of generative models.Source: §6 · p. 11

7 REPRODUCIBILITY STATEMENT

• The authors report documenting model architecture, training details, hyperparameters, protocols, ablations, and evaluation metrics, and plan to release code, pretrained checkpoints, and reproduction instructions.Datasets are publicly available and described in the appendix.Source: §7 · p. 11

A FULL ALGORITHM OF FORWARD SAMPLING PROCESS IN VAR

• At scale n, the VAR transformer operates with the previous scale as input.Source: A FULL ALGORITHM OF FORWARD SAMPLING… · p. 15
• The appendix presents this procedure as VAR's inference algorithm.Source: A FULL ALGORITHM OF FORWARD SAMPLING… · p. 15
• The output is downsampled to scale n−1, and N denotes the largest scale in the sampling process.Other details are referred to the section “Rethinking VAR Variants Through the Discrete Diffusion Lens.”Source: A FULL ALGORITHM OF FORWARD SAMPLING… · p. 15

Classifier free guidance

• At cfg 5, mask-resampling reaches FID 13.26 and IS 17.97, staying within ±0.3 FID across cfg 1.5–10.VAR peaks at cfg 2 with FID 15.55 and IS 16.76, then degrades as guidance increases; SDD remains flat until cfg 6.Source: B ANALYSIS · p. 15

Distillation of VAR variants

• Dropping two of every three scales reduces decoder passes by 2.5×, but FID rises to 31.21 at cfg 3.5 and 29.83 at cfg 5.0.IS falls to 48.25/48.61, and the student fails to reconstruct both low-frequency layout and high-frequency details.Source: B ANALYSIS · p. 15
• The early-heavy schedule yields FID 39.91 and IS 38.95 at cfg 5.0, with blurry textures and colour bleeding.A single high-scale pass is insufficient to overwrite errors accumulated during densely sampled low-scale stages.Source: B ANALYSIS · p. 15
• The random sparse 1-4-8-16 schedule performs worst, with FID 38.76 at cfg 5.0.The passage attributes its failure to poorly overlapping receptive fields that disrupt iterative error correction.Source: B ANALYSIS · p. 15
• A 1-3-5-8-13-16 schedule cuts inference time by 20% while preserving perceptual quality within 8% of the teacher.SDD remains strictly better than VAR across settings; consecutive low-resolution scales are largely redundant, but at least two high-resolution scales are indispensable.Source: B ANALYSIS · p. 15

Compute budget

• Experiments used four NVIDIA A6000 GPUs for 200 epochs, forcing reduced training datasets and limiting the largest model explored.Larger-scale training might reveal failure modes or gains not tested in this setting.Source: C LIMITATIONS · p. 16

Dataset scope

• Evaluation covers four medium-scale image collections at 256 × 256 resolution, leaving performance on very high-resolution images, complex scenes, and video unclear.Source: C LIMITATIONS · p. 16

Codebook expressiveness

• Like VAR, SRDD relies on one VQ-VAE codebook, whose discrete bottleneck can limit fine detail and color accuracy versus continuous-latent diffusion models.Source: C LIMITATIONS · p. 16

Larger-scale pre-training and scaling laws

• Systematic pre-training across model sizes, sequence lengths, and token vocabularies could establish scaling laws and compute-efficient regimes.The authors say current results hint at the parameter–quality trend observed in VAR.Source: D FUTURE WORK · p. 16

Learned resampling policies

• A learned policy could replace mask-resampling's fixed probability threshold by predicting which tokens to resample from decoder uncertainty.The proposed direction might improve results while reducing refinement passes.Source: D FUTURE WORK · p. 16

Continuous–discrete hybrid diffusion

• A hybrid could run SRDD at coarse scales, then use a lightweight continuous decoder for final touch-ups.The proposed combination aims to pair SRDD speed with continuous diffusion photorealism.Source: D FUTURE WORK · p. 16

Leveraging advances in discrete diffusion theory

• Advances in discrete diffusion theory, including refined noise schedules and tighter ELBO bounds, could transfer to SRDD.The authors present this as a low-cost path for SRDD to inherit future developments.Source: D FUTURE WORK · p. 16