PaperlayerView exact revision on arXiv

Source-linked AI summary

The Design Space of Tri-Modal Masked Diffusion Models

Louis Bethune, Victor Turrisi, Bruno Kacper Mlodozeniec, Pau Rodriguez Lopez, Lokesh Boominathan, Nikhil Bhendawade, Amitis Shidani, Joris Pelemans, Theo X. Olausson, Devon Hjelm, Paul Dixon, Joao Monteiro, Pierre Ablin, Vishnu Banna, Arno Blaas, Nick Henderson, Kari Noriy, Dan Busbridge, Josh Susskind, Marco Cuturi, Irina Belousova, Luca Zappella, Russ Webb, Jason Ramapuram

arXiv:2602.21472v1cs.LG

TL;DR

Native multimodal masked diffusion pretraining and its design space remain underexplored. This paper extends masked discrete diffusion to a unified tri-modal model and analyzes its scaling and inference design space. It derives empirical scaling laws and finds that inference tradeoffs vary by modality.

  • Problem

    Multimodal masked diffusion research has largely adapted pretrained models, leaving native multimodal pretraining and its broad design space underexplored.

    Source: §1 · p. 2

  • Method

    The paper extends masked discrete diffusion to one tri-modal model with a unified token space and uses SDE-based reparameterization to eliminate optimal-batch-size search.

    Source: §1 · p. 2

  • Results

    The authors derive tri-modal scaling laws, finding the Kaplan form fits better than the additive form, and identify modality-dependent inference tradeoffs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings offer empirical guidance for compute-optimal tri-modal training and indicate that inference settings should be chosen separately by modality.

    Source: §1 · p. 2

    At sufficiently large token budgets, scaling GPUs risks reaching critical batch size, where added tokens lose marginal utility or increase wall-clock time, preventing both FLOP-efficient and fast training.

    Source: §5.1 · p. 9

Abstract

from arXiv · show

Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and fine-tuning a base unimodal model for bimodal generation. Diverging from previous approaches, we introduce the first tri-modal masked diffusion model pretrained from scratch on text, image-text, and audio-text data. We systematically analyze multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects, and we provide optimized inference sampling defaults. Our batch-size analysis yields a novel stochastic differential equation (SDE)-based reparameterization that eliminates the need for tuning the optimal batch size as reported in recent work. This reparameterization decouples the physical batch size, often chosen based on compute constraints (GPU saturation, FLOP efficiency, wall-clock time), from the logical batch size, chosen to balance gradient variance during stochastic optimization. Finally, we pretrain a preliminary 3B-parameter tri-modal model on 6.4T tokens, demonstrating the capabilities of a unified design and achieving strong results in text generation, text-to-image tasks, and text-to-speech tasks. Our work represents the largest-scale systematic open study of multimodal discrete diffusion models conducted to date, providing insights into scaling behaviors across multiple modalities.

1 Introduction

The paper introduces a tri-modal masked diffusion model pretrained from scratch in a unified discrete token space, and studies the training and inference choices that shape its scaling and efficiency.

1 Introduction

  • The introduction motivates bidirectional denoising for conditional generation with evidence scattered across positions and modalities, while noting that MDM sampling latency remains a challenge.Source: §1 · p. 1
  • Concatenated text, image, and audio tokens are partially masked and jointly denoised in one unified discrete space, enabling infilling and arbitrary conditioning without modality-specific heads or adapters.The single model supports text-to-image generation, image captioning, text-to-speech, and automatic speech recognition.Source: §1 · p. 2
  • The work targets multimodal MDM pretraining, where compute is concentrated and latent spaces are shaped, and seeks reliable rules for transferring hyperparameters from small to larger models.Source: §1 · p. 2
  • SDE reparameterization makes training loss invariant to batch size up to Bcrit, eliminating the need to search for Bopt; CompleteP + SDE scaling validates per-module hyperparameter transfer.Source: §1 · p. 2
  • The Kaplan et al. (2020) scaling form fits better than Hoffmann et al. (2022), with compute-optimal D*(N) ≈ 7754 · N^0.84 and greater asymptotic data efficiency than autoregressive models.Source: §1 · p. 2
  • Controlled inference ablations find that optimal noise schedules and sampling parameters, including guidance and temperature, differ significantly across text, image, and audio generation.Source: §1 · p. 2

2 Background and Related Work

This section introduces masked diffusion models (MDMs), surveys their use for text, images, and audio, and situates the paper’s tri-modal model against prior multimodal MDMs. It highlights the shift to training from scratch across three modalities while accounting for the total token budget.

2.1 Masked Diffusion Models

  • MDMs progressively replace data tokens with [MASK] until all are masked, then learn a reverse process to reconstruct the original data.The framework originated in discrete diffusion work by Austin et al. (2021), building on Sohl-Dickstein et al. (2015) and Hoogeboom et al. (2021).Source: §2.1 · p. 3
  • Continuous-time extensions formulate forward and reverse diffusion as continuous-time Markov chains, while another approach embeds discrete inputs in continuous space.These directions are attributed to Campbell et al. (2022) and Dieleman et al. (2022), respectively.Source: §2.1 · p. 3
  • Text MDMs progressed from small-scale datasets to LLaDA’s 8B-parameter training on 2.3T tokens, while Dream fine-tuned pretrained Qwen2.5-7B on 580B tokens.The passage notes that Dream does not account for the initial autoregressive model’s pretraining budget; both methods use the same training objective.Source: §2.1 · p. 3
  • Direct categorical diffusion on pixels was limited to low-resolution datasets, while pretrained image tokenizers enabled stable high-resolution discrete diffusion on patch-wise tokens.Examples of the tokenizer-based approaches include MaskGIT and VQ-Diffusion, using VQ-GAN or VQ-VAE tokenizers.Source: §2.1 · p. 3
  • Prior audio work includes discrete diffusion for audio generation and a SoundStream-tokenizer approach combining masking and unmasking.The cited examples are Yang et al. (2023) and Borsos et al. (2023), respectively.Source: §2.1 · p. 3

2.2 Multimodal Masked Diffusion Models

  • The work extends prior bi-modal text–image MDMs by adding audio, pretraining from scratch, and jointly optimizing all modalities while accounting for the total token budget.MMaDA, by contrast, is initialized from LLaDA’s weights; the authors encourage reporting model and data size together as (N, D_total) for fair comparisons.Source: §2.2 · pp. 3–4

3 Method

The method represents text, image, and audio tokens in a shared-vocabulary masked diffusion model, trained with modality-specific corruption and a denoising objective. Generation iteratively unmasks tokens under modality constraints, using a bidirectional transformer architecture.

3 Method

  • The model unifies text-only, image-text, and audio-text sequences using modality-specific discrete vocabularies within a shared vocabulary, with modality and task tokens.A PADtext token is also introduced for short multimodal sequences; task tokens can signal the task and support adding tasks during later training or fine-tuning.Source: §3 · p. 4
  • Training sequences have maximum length L★: text-only sequences are packed and truncated, while shorter mixed-modality sequences are right-padded after EOStext.Boundary tokens wrap modality tokens; text-only sequences therefore need no padding.Source: §3 · pp. 4–5
  • Each position is independently masked with a modality-specific MASK token according to a monotonic corruption schedule from t=0 to t=1, while task tokens remain unmasked.The process interpolates from the original sequence to a fully masked sequence; once masked, a token stays masked.Source: §3 · p. 5
  • A denoiser predicts unified-vocabulary logits, and training minimizes loss over masked, non-padding positions using w(t)=1/t for an unbiased ELBO estimator under Bernoulli masking.Cut-cross-entropy avoids materializing the full probability distribution and enforces modality constraints; a z-loss regularizer stabilizes logits (Wijmans et al., 2024; de Brébisson and Vincent, 2016).Source: §3 · pp. 5–6
  • Generation conditions on a prompt and target modality, samples masked positions from modality-constrained predictions, and reveals a scheduled subset at each reverse step until none remain.The process starts from a fully masked sequence and follows a predefined linear unmasking schedule.Source: §3 · p. 6

3.1 Architecture

  • Experiments use a bidirectional prenormalized RMSNorm transformer with SwiGLU, RoPE, and QK-norm; the 3B model is pretrained from scratch for 1M steps at batch size 3072 and sequence length 3256.Specialized encoders tokenize images, audio, and text: SBER-MoVQGAN, Higgs Audio v2, and Tiktoken, respectively.Source: §3.1 · p. 6

4 Hyperparameter Transfer

The section transfers hyperparameters from small to larger models and introduces SDE-based batch-size parametrization to reduce dependence on compute-constrained batch size. It also defines scaling rules that balance SDE drift and horizon as token count increases.

4 Hyperparameter Transfer

  • The work adopts Mlodozeniec et al. (2025)'s extension of CompleteP and provides the first empirical validation of per-module hyperparameter tuning for multimodal masked diffusion models.The tuning adjusts AdamW learning rate, weight decay, momenta, and epsilon across modules; preliminary results are in Appendix D.Source: §4 · p. 6

4.2 Isonoise and Isohorizon Scaling Rules

  • The proposed scaling rule uses γ∈[0, 1] to conserve SDE drift at γ=0, conserve SDE horizon at γ=1, or interpolate between them as token count increases.The resulting SDE-scaling factor κ is used to rescale AdamW hyperparameters for target model and batch sizes.Source: §4.2 · p. 7

4.1 Eliminating 𝐵opt with SDE Parametrization

  • Below the critical batch size B_crit, SDE parametrization maintains constant loss across batch sizes; above it, discretization breaks and training is no longer FLOP-efficient.Within the SDE regime, larger batches require fewer iterations.Source: Figure 4 · p. 7, §4.1 · p. 7

4.2 Isonoise and Isohorizon Scaling Rules

  • Critical iteration count S_crit remains constant with model size under the SDE regime, consistent with Bergsma et al. (2025), whose study was outside that regime.Figure 5 reports this model-size relationship.Source: Figure 5 · p. 7

5 Scaling Behavior of MDM under the SDE Transfer Rule

Under the SDE transfer rule, critical batch size governs when training remains FLOP-efficient, while token-horizon scaling determines the compute-optimal model and data allocation. The section finds a nonstandard optimal drift–horizon tradeoff and fits scaling laws for tri-modal MDMs, with important limits on cross-family interpretation.

5.1 Scaling Rules for Critical Batch Size

  • Under SDE, all batch sizes below Scrit yield identical results at fixed token budget, and runs below Scrit are FLOP-efficient while runs above it are wall-clock-inefficient.The physical batch size can be selected for available compute as long as B≤Bcrit; Bcrit = D/(L Scrit), and too few integration steps break the SDE approximation and sharply reduce performance.Source: §5.1 · p. 8
  • Scrit is independent of model size but grows sublinearly with token horizon, so Bcrit also grows sublinearly and proportional GPU scaling eventually risks reaching it.Beyond that point, additional tokens have declining marginal utility or total wall-clock time increases; over-trained models cannot be both FLOP-efficient and fast to train.Source: §5.1 · pp. 8–9, Figure 6 · p. 8, Figure 7 · p. 8

5.2 Optimal Drift–horizon Tradeoffs

  • The fitted drift–horizon optimum is γ*=0.44, and neither the γ=0 nor γ=1 setting used in prior work is optimal.The fit gives α=0.18 and β=0.23; γ allocates added tokens between reducing drift via virtual batch size and extending the SDE horizon, and its optimum could shift under a different learning-rate schedule.Source: §5.2 · p. 9, Figure 9 · p. 9, Figure 8 · p. 9

5.3 Scaling Laws for Tri-modal MDM

  • A 3B tri-modal MDM requires at least 480B tokens, versus 60B tokens reported by Chinchilla (Hoffmann et al., 2022) for autoregressive language models.The fitted compute-optimal token count is D★ = 7754 · N^0.84; the gap from Chinchilla holds at realistically reachable model sizes.Source: §5.3 · p. 10
  • The compute-optimal rule D★ = 7754 · N^0.84 implies D★/N ∝ N^-0.16, and tri-modal curves cross Quokka around 20B parameters (≈2T tokens).Below the crossing, at equal FLOPs, tri-modal models should be smaller and trained longer than Quokka models; above it, the trend reverses.Source: §5.3 · pp. 10–11
  • Across 262 tri-modal models, the scaling-law fit yielded a≈0.14, b≈0.17, R2=99.3%, and MRE=0.5%.Models spanned token-per-parameter ratios of 1–2000 and 24 logarithmically distributed isoFLOPs from 5 × 10^18 to 1 × 10^22.Source: §5.3 · p. 10, Figure 11 · p. 11
  • Scaling exponents alone do not establish cross-method superiority, and ELBO can mislead across diffusion families because their forward processes induce different likelihood bounds.The data coefficient b is also sensitive to data composition: repetitions reduce effective token count and can deflate apparent scaling efficiency.Source: §5.3 · p. 11

6 Data

The experiments use balanced text, image-text, and audio-text data, with each modality assembled from filtered corpora. The token horizon stays below total dataset size, while some small high-quality components are repeated up to four times.

6 Data

  • Experiments use a 33% text, 33% image-text, and 33% audio-text mix, except for the Section 7.2 modality-mixing ablation.Within each modality, mixture components are reweighted to balance quality and diversity; the token horizon is below total dataset size, and some small high-quality components are repeated up to four times within estimated tolerance (Prabhudesai et al., 2025).Source: §6 · p. 12
  • The text corpus combines public, licensed, and synthetic data, with additional cleaning and filtering for PII and licensing-related concerns.Sources include Nemotron-CC (Su et al., 2025), DCLM (Li et al., 2024), subsets of The Pile (Gao et al., 2020), licensed StackOverflow data, and synthetic reasoning traces; validation uses different dataset splits with identical mixture ratios.Source: §6 · p. 12
  • Audio-text training uses 2M hours of web-scraped audio transcribed by Whisper (Radford et al., 2023), with speech, dialogue, and production-quality filters.The source data were PII-filtered and filtered for production complexity (Tjandra et al., 2025).Source: §6 · p. 12
  • Image-text training aggregates image and recaptioned-text datasets, including CC3M, CC12M, and COYO, with PII filtering.The aggregation also includes other licensed datasets from Manzano (Li et al., 2025).Source: §6 · p. 12

7 Results

The results evaluate a pretrained-only 3B tri-modal model and ablate modality mixing, generation hyperparameters, and anti-masking. They report no cross-modal mixing synergies, modality-specific sampling trade-offs, and improvements from anti-masking.

7.1 Unified 3B Tri-modal MDM

  • The pretrained-only 3B tri-modal model is evaluated on text, image, and audio benchmarks, including language tasks, image FID and GenEval, and speech FAD, WER, and Audiobox Aesthetics.Image FID uses DINOv2-L and Inception-v3 features; speech evaluation uses ground-truth prompts and durations.Source: Figure 15 · p. 12, §7.1 · p. 13

7.2 Modality Mixing Ratios

  • At 320M parameters and 13B tokens, increasing a modality’s mixture weight lowers its corresponding loss, but the other modalities’ ratios show no synergy, making equal thirds a reasonable default.The 15 experiments constrain each modality to at least 20% of the mixture.Source: §7.2 · p. 13, Figure 16 · p. 13

7.3 Best Generation Hyperparameters

  • For text-to-image generation, FID improves with more steps but with diminishing returns, while CFG, temperature, and top-p have intermediate optima except that higher top-p is preferred.The ablations generate 10,000 images using CC12M and train (eval seed) prompts, varying one hyperparameter at a time.Source: §7.3 · p. 13, Figure 17 · p. 14
  • For text-to-speech, more steps improve quality with diminishing returns; stronger CFG improves WER but worsens FAD, and optimal settings vary across metrics.Trends broadly hold on train (eval seed) and LibriSpeech-PC; evaluation uses ground-truth durations and samples filtered to at most 30 seconds.Source: §7.3 · pp. 13–14, Figure 18 · p. 14

7.4 Anti-Masking

  • Anti-masking creates two versions of each sample with complementary masks to reduce gradient-estimate variance, and comparisons use a fixed unique-token horizon.The anti-mask variations process each sample twice within one epoch, while standard training revisits unique samples over two epochs.Source: §7.4 · pp. 14–15
  • Anti-masking improves multimodal performance across modalities, with the largest gains in audio, and improves most text-only tasks.The multimodal comparison reports image FID and audio FAD; the text-only comparison reports mean accuracy ± standard deviation.Source: Table 1 · p. 15, Table 2 · p. 15, §7.4 · p. 15
  • Anti-masking improves benchmark performance in the multiple-epochs setup without additional computational cost.For compute matching at a fixed token horizon, standard masking trains for two epochs while anti-masking trains for one.Source: §7.4 · p. 15

8 Conclusion

The paper extends masked discrete diffusion to unified tri-modal generation, sharing one token stream and transformer across text, images, and audio. It also identifies design choices that shape stable, efficient training and modality-specific inference at scale.

8 Conclusion

  • A single token stream and transformer support flexible text, image, and audio conditioning without modality-specific heads or bespoke factorizations.The framework covers captioning, text-to-image, ASR, and TTS.Source: §8 · p. 15
  • SDE-based reparameterization reduces expensive tuning, while empirical scaling behavior guides compute-optimal training.The conclusion also notes that inference guidance, temperature, and sampling steps should be chosen differently for different modalities.Source: §8 · p. 15
  • Anti-masking yields consistent improvements under compute-matched comparisons.The paper also finds a strongly modality-dependent inference landscape for sampling hyperparameters.Source: §8 · p. 15

Appendices · A General formulation of weighting and the masking process · B Tokenizer Ablations

The appendix specifies masked diffusion as a per-token Markov corruption process and explains how cumulative masking and inverse-probability weighting shape its training objective. The weighting analysis connects timestep contributions to corruption, while the importance-weighting analysis corrects for masked-token subsampling.

A General formulation of weighting and the masking process

  • Each token follows a Markov corruption process that either leaves it unchanged or replaces it with MASK, with masking probability β_t and retention probability α_t = 1 − β_t.The stated transition rule excludes transitions to other tokens, though the text notes that more general formulations could allow them.Source: A General formulation of weighting and… · p. 24
  • The cumulative retention probability ᾱ_t is the product of per-step retention probabilities, so a token remains unchanged with probability ᾱ_t and is masked by time t with probability 1 − ᾱ_t.This marginal distribution permits direct sampling of s_t from s_0 at any timestep.Source: A General formulation of weighting and… · p. 24

A.1 Connection of Loss Weighting and Cumulative Corruption

  • The timestep loss weight w(t) is tied to the masking schedule because early timesteps leave more context and are easier, whereas late timesteps mask most tokens and are harder.The weighting makes each timestep contribute proportionally to the rate at which masking destroys information about s_0.Source: §A.1 · pp. 24–25
  • The ELBO-based weighting analysis relates timestep contributions to the incremental probability that masking occurred at that timestep.The appendix frames this weighting as compensating for differences in corruption across timesteps.Source: §A.1 · pp. 24–25

A.2 Unbiasedness of the Importance Weighting

  • Under linear scheduling, each token position is independently masked with probability t, so averaging reconstruction loss over masked positions randomly subsamples tokens.This sampling causes smaller-t timesteps to contribute less without reweighting.Source: §A.2 · p. 25
  • Multiplying by 1/t corrects masked-token subsampling and gives each token equal expected contribution across timesteps.The passage characterizes this as inverse-probability weighting, analogous to time-dependent loss rescaling in denoising score matching.Source: §A.2 · p. 25

B.1 Audio Tokenizer Ablations … C.3 Hyperparameters for the Unified 3B Tri-modal MDM

The appendix selects compact audio and image tokenizers under sequence-length constraints, then reports the hyperparameter-search procedure, batch-size runtime trade-offs, and training details for the unified 3B model.

B.1 Audio Tokenizer Ablations

  • Higgs Audio v2 with 4 codebooks is the default audio tokenizer, offering a rate–distortion trade-off that fits the 30-second clip and fixed-context constraints.The fixed context length is L★=3256; only two configurations fit the token budget for 30-second audio.Source: §B.1 · pp. 25–26
  • Training with more codebooks and decoding with fewer can preserve perceptual quality better than directly training a low-rate model.Increasing the number of codebooks improves reconstruction but quickly becomes impractical under the sequence-length constraint.Source: §B.1 · p. 26

B.2 Image Tokenizer Ablations

  • MoVQGAN is the default image tokenizer, selected for balancing reconstruction performance, compression, and vocabulary size.The comparison evaluates reconstruction FID on ImageNet at 256 and 512 resolution and on CC12M.Source: §B.2 · p. 26, Table 4 · p. 26

C.1 Optimal Global Hyper-Parameter Search

  • A Gaussian Process fit to 2,900 small-scale trial runs identifies optimal global hyperparameters and seeds the per-module multiplier search.The search uses 13B tokens and 320M parameters in total, including 80M non-embedding parameters.Source: §C.1 · p. 27

C.2 Runtime as Function of Batch Size

  • Adding nodes sharply reduces runtime but sacrifices FLOP efficiency because cluster throughput scales sub-linearly with node count.The runtime declines more slowly as per-GPU batch size increases; doubling node count approximately halves runtime, but does not recover the efficiency of fewer nodes.Source: §C.2 · p. 27, Figure 20 · p. 27
  • Small per-GPU batches can leave GPUs idle, while saturated GPUs use FLOPs more efficiently than non-saturated ones.This GPU-utilization effect is distinct from the runtime reduction achieved by increasing node count.Source: §C.2 · p. 27, Figure 20 · p. 27

C.3 Hyperparameters for the Unified 3B Tri-modal MDM

D MDM with Per-module Hyperparameters

The section moves from global transfer settings to per-module AdamW tuning, finding structured multipliers and a token-efficiency gain.

D MDM with Per-module Hyperparameters

  • Each weight receives a unique multiplier, factored into shared depth-dependent and module-type terms.AdamW multipliers cover learning rate, weight decay, 𝛽1, 𝛽2, and 𝜖; this analysis builds on CompleteP (Dey et al., 2025) global hyperparameters and the local search method from Mlodozeniec et al. (2025).Source: D MDM with Per-module Hyperparameters · p. 29
  • The local random search began at the best global settings and used an eight-block, width-1024 model with 320M parameters, trained for 13B multimodal tokens.The model included 80M non-embedding parameters.Source: D MDM with Per-module Hyperparameters · p. 29
  • 1.81× fewer tokens were needed to reach equivalent loss with per-module tuning on a 350M model trained for 13B tokens.Figure 21 reports average final ELBO for the sweep using batch size 256.Source: Figure 21 · p. 29
  • Embedding and unembedding weights favor effective learning rates up to ∼4×, while attention projections and MLP gates use more conservative settings.Later blocks favor smaller steps and stronger stabilization; some module settings increase 𝜖 for numerical damping.Source: D MDM with Per-module Hyperparameters · p. 29

E Extended Scaling Laws Results

This appendix explains how embedding-excluded parameter counts affect compute-optimal scaling-law fits and reports a separate uni-modal text scaling-law experiment.

E Extended Scaling Laws Results

  • Scaling-law fits count non-embedding parameters as N, but count all FLOPs as C, so the usual FLOP/token = 6N formula does not directly apply.For small models, embedding parameters can make total model size up to five times larger than the non-embedding count.Source: E Extended Scaling Laws Results · p. 30
  • The compute-optimal N*(C) and D*(C) curves are found by minimizing parametric loss with the actual FLOP/token function; FLOP/token ≈ 6 works for larger models.The appendix uses a linear search over N, noting that embedding size has a more substantial effect for small models.Source: E Extended Scaling Laws Results · p. 30
  • Uni-modal text MDM scaling-law runs use CompleteP without SDE scaling rules, with batch sizes chosen for GPU occupancy and packed, truncated sequences of length 4,096.Training curves and scaling-law predictions are shown in Figures 22 and 23, respectively.Source: E Extended Scaling Laws Results · p. 30, Figure 22 · p. 31, Figure 23 · p. 31

F Masking Schedules for Image and Audio Generation · G Extended generations · H Contributions

Across the tested schedules, polynomial performs best for both image and audio generation, with different optimal CFG ranges. The appendix also presents extended generations and documents author contributions.

F Masking Schedules for Image and Audio Generation

G Extended generations

  • Extended generations appear in Figures 26–29 with prompts in Tables 7–10; the prompts are synthetic captions, and samples were selected for quality and diversity.Source: G Extended generations · pp. 31–32

F Masking Schedules for Image and Audio Generation

  • Polynomial also achieves the best audio quality across all six evaluated metrics, with an optimal CFG range of 1–3.Ground-truth-duration audio was evaluated using 512 diffusion steps and 10,000 samples; metrics included FAD, WER, and AudioBox Aesthetics.Source: F Masking Schedules for Image and… · p. 31, Figure 24 · p. 32
  • Polynomial achieves the best image quality across both metrics and datasets, with optimal CFG scales of 7–9.Image generation was evaluated at 256×256 using 1,024 diffusion steps and 8,192 samples; quality was measured with FID-Inception and FID-DINOv2.Source: F Masking Schedules for Image and… · p. 31, Figure 25 · p. 32

H Contributions

  • All authors contributed to writing, experiment design, and discussion; named responsibilities span experiments, theoretical framing, literature grounding, and project leadership.Source: H Contributions · p. 41
  • Code, data, and infrastructure contributions cover training and evaluation code, data loading and preprocessing, and an optimized Docker-container pipeline.The passage names contributors for each responsibility, including Nick Henderson for the infrastructure pipeline.Source: H Contributions · p. 41

Related papers

Loading 2602.21472v1…