PaperlayerView exact revision on arXiv

Source-linked AI summary

MMaDA: Multimodal Large Diffusion Language Models

Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, Mengdi Wang

arXiv:2505.15809v2cs.CV

TL;DR

Unified multimodal diffusion models have largely focused on architecture and pretraining, leaving post-training methods for non-autoregressive settings underexplored. MMaDA combines a shared diffusion framework with mixed long-CoT fine-tuning and UniGRPO, and reports strong performance across textual reasoning, multimodal understanding, and text-to-image generation.

  • Problem

    Unified multimodal foundation models have focused mainly on architecture and pretraining, leaving post-training methods for non-autoregressive settings underexplored.

    Source: §1 · p. 2

  • Method

    MMaDA uses a shared modality-agnostic diffusion architecture, mixed long-CoT fine-tuning across modalities, and UniGRPO with diversified rewards for post-training.

    Source: §1 · p. 2, §2.3.2 · p. 7

  • Results

    MMaDA achieves superior, balanced performance across textual reasoning, multimodal understanding, and text-to-image generation, outperforming autoregressive and diffusion baselines in accuracy, efficiency, and task adaptability.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The experiments highlight the potential of diffusion models as a foundation paradigm for multimodal intelligence.

    Source: §7 · p. 17

    MMaDA is limited by its current 8B-parameter model size, and the authors plan to use a larger model for better performance.

    Source: §7 · p. 17

Abstract

from arXiv · show

We introduce MMaDA, a novel class of multimodal diffusion foundation models designed to achieve superior performance across diverse domains such as textual reasoning, multimodal understanding, and text-to-image generation. The approach is distinguished by three key innovations: (i) MMaDA adopts a unified diffusion architecture with a shared probabilistic formulation and a modality-agnostic design, eliminating the need for modality-specific components. This architecture ensures seamless integration and processing across different data types. (ii) We implement a mixed long chain-of-thought (CoT) fine-tuning strategy that curates a unified CoT format across modalities. By aligning reasoning processes between textual and visual domains, this strategy facilitates cold-start training for the final reinforcement learning (RL) stage, thereby enhancing the model's ability to handle complex tasks from the outset. (iii) We propose UniGRPO, a unified policy-gradient-based RL algorithm specifically tailored for diffusion foundation models. Utilizing diversified reward modeling, UniGRPO unifies post-training across both reasoning and generation tasks, ensuring consistent performance improvements. Experimental results demonstrate that MMaDA-8B exhibits strong generalization capabilities as a unified multimodal foundation model. It surpasses powerful models like LLaMA-3-7B and Qwen2-7B in textual reasoning, outperforms Show-o and SEED-X in multimodal understanding, and excels over SDXL and Janus in text-to-image generation. These achievements highlight MMaDA's effectiveness in bridging the gap between pretraining and post-training within unified diffusion architectures, providing a comprehensive framework for future research and development. We open-source our code and trained models at: https://github.com/Gen-Verse/MMaDA

1 Introduction

MMaDA addresses the post-training gap in unified multimodal diffusion models with a modality-agnostic architecture, cross-modal CoT fine-tuning, and unified reinforcement learning. The paper reports balanced performance across textual reasoning, multimodal understanding, and text-to-image generation.

Introduction

  • MMaDA uses a shared probabilistic formulation and modality-agnostic diffusion architecture, eliminating modality-specific components while supporting strong performance across tasks.The paper presents this architecture as part of a framework addressing the gap between pretraining and post-training in unified multimodal diffusion models.Source: §1 · p. 2
  • Mixed long-CoT fine-tuning curates a unified cross-task format that aligns textual and visual reasoning, enabling cold-start training and intermediate reasoning before output generation.The approach is intended to foster cross-modal synergy.Source: §1 · p. 2
  • UniGRPO is a diffusion-centric reinforcement-learning algorithm for multimodal generation that uses diversified rewards to support complex reasoning and factual consistency.The paper motivates this post-training focus by noting that existing unified multimodal models leave non-autoregressive post-training methods underexplored.Source: §1 · pp. 2–3
  • MMaDA achieves superior, balanced performance across textual reasoning, multimodal understanding, and text-to-image generation, outperforming autoregressive and diffusion-based baselines.The introduction characterizes its advantages in accuracy, efficiency, and task adaptability.Source: §1 · p. 3

2 MMaDA: Multimodal Large Diffusion Language Models

MMaDA combines a shared masked-token diffusion objective for text and images with mixed long-CoT finetuning and UniGRPO, whose task-specific rewards support post-training across reasoning and generation.

2.1 Pretraining with Unified Diffusion Architecture and Objective

  • A shared diffusion formulation aligns noise corruption and semantic recovery across text and images, predicting masked tokens simultaneously with loss computed only on masked tokens.The model is formulated as a mask-token predictor under a unified cross-entropy objective.Source: §2.1 · p. 4

2.2 Post-Training with Mixed Long-CoT Finetuning

  • A curated long-CoT dataset spans textual reasoning, multimodal reasoning, and text-to-image generation, using a unified reasoning/result format and verifier-filtered trajectories.The shared structure bridges modality-specific outputs and supports knowledge transfer; the selected samples are diverse, complex, and accurate.Source: §2.2 · p. 5
  • Mixed long-CoT finetuning jointly optimizes heterogeneous tasks and initializes RL by preserving prompts, masking result tokens, and reconstructing them from prompt and corrupted-result context.The objective computes loss over masked result positions while maintaining alignment with the original prompt and task-specific reasoning logic.Source: §2.2 · p. 5

2.3.1 Unified GRPO for Diffusion Foundation Models

  • UniGRPO adapts policy-gradient RL to diffusion by sampling varied response mask ratios and approximating likelihood from masked-token predictions, capturing multi-step denoising efficiently.Its design avoids the computational inefficiency of numerous mask-ratio samples and the oversimplified prediction strategy of prior approaches.Source: §2.3.1 · pp. 6–7

2.3.2 Diversified Reward Modeling

  • A unified rule-based reward combines task-specific correctness, output-format, image-text alignment, and human-preference signals under a KL-penalized objective.Textual GSM8K [29] uses correctness 2.0 and format 0.5; GeoQA and CLEVR use the same rewards for mathematical tasks, with 0.1 · CLIP(image, text) for captioning; generation uses 0.1-scaled CLIP and Image Reward.Source: §2.3.2 · p. 7

3 Flexible Sampling Strategies at Inference Time

MMaDA uses semi-autoregressive diffusion denoising for text generation and parallel non-autoregressive denoising for image generation. Text sampling produces more detailed descriptions than fixed-length generation in the reported qualitative comparison.

Semi-Autoregressive Sampling for Text Generation

  • Text generation follows LLaDA [22] by generating left-to-right blocks and denoising a selected subset of masked positions using a linear masking schedule.Tokens are selected either randomly or by confidence scores, and denoising repeats for a prescribed number of steps.Source: §3 · p. 8
  • Semi-autoregressive text sampling produced more intricate, detailed descriptions, while fixed-length generation often produced very short responses.The evaluation used length 1024, 512 denoising steps, and blocks of 64 tokens; directly applying lowest-confidence remasking without blocks can yield unnaturally frequent |EOS| tokens in instruction-tuned models.Source: §3 · p. 8

Parallel Non-Autoregressive Sampling for Image Generation

  • Image generation uses parallel low-confidence remasking over one output block, following MAGVIT-v2 [24] with a cosine noise schedule.Evaluation generates length-1024 sequences for 512×512 images over 50 denoising steps, with classifier-free guidance scale 3.5.Source: §3 · p. 8

4 Experiments

Across multimodal understanding, text-to-image generation, and textual reasoning, MMaDA reports strong benchmark performance while jointly training a diffusion model across these tasks. Its experiments use staged training, task-specific datasets, and task-specific evaluation metrics and baselines.

4.1 Experimental Setup

  • Training data include RefinedWeb and image-text datasets, Alpaca and LLaVA-1.5 instruction data, curated textual and multimodal reasoning data, and synthesized factual image-generation traces.UniGRPO uses the original mathematical and logical datasets used in Reasoning.Source: §4.1 · p. 9
  • Evaluation covers multimodal understanding, image generation, and text generation using task-specific benchmarks, metrics, and baselines.The protocols include POPE, MME, Flickr30k, VQAv2, GQA, and MMMU for understanding; CLIP Score, ImageReward, GenEval, and WISE for generation; and MMLU, GSM8K, and related benchmarks for text generation.Source: §4.1 · p. 9
  • MMaDA is initialized from LLaDA-8B-Instruct weights and Show-o image-tokenizer weights, then jointly trained through foundational, instruction-and-reasoning, and UniGRPO stages.The stages run for 200K plus 400K, 50,000, and 50,000 steps, respectively; training uses 64 A100 (80GB) GPUs, a global batch size of 1,280, and AdamW with an initial learning rate of 5e-5.Source: §4.1 · pp. 9–10

4.3 Text-to-Image Generation

  • MMaDA achieves the highest CLIP Score and ImageReward across generation-only and unified models, shows superior GenEval compositionality and object counting, and outperforms prior approaches on WISE Cultural.The paper attributes the CLIP Score and ImageReward results to UniGRPO rewards aligned with those metrics, and the WISE Cultural result to joint training on text-based reasoning.Source: Table 3 · p. 10, §4.3 · p. 10

4.2 Multimodal Understanding

  • MMaDA achieves comparable or superior results to dedicated understanding models on most benchmarks and consistently outperforms unified models on several benchmarks.The paper particularly attributes its understanding performance to Mixed Long-CoT Finetuning and UniGRPO; reasoning traces are evaluated using the final answer as the prediction.Source: §4.2 · p. 10

4.4 Textual Reasoning

  • MMaDA matches Qwen2-7B and LLaMA3-8B on MMLU and ARC-C and outperforms LLaDA-8B on math benchmarks despite limited task-specific tokens and open-source text data.The paper describes this as joint training of a unified diffusion model for text generation, multimodal reasoning, and image generation.Source: Table 4 · p. 12, §4.4 · p. 12

5 Observations, Analysis and Conclusion

MMaDA’s staged training improves reasoning and generation, while UniGRPO’s masking choices support steadier learning. Joint training benefits text, multimodal understanding, and image generation, and the model also supports efficient sampling and inpainting across modalities.

5.2 Design Choices of UniGRPO

  • On GSM8K, UniGRPO achieves higher training rewards than d1 by leaving questions unmasked and partially masking answers, retaining noise across denoising timesteps.Experiments use the same MMaDA pretrained checkpoint; the masking design is intended to better leverage diffusion-model learning across multiple denoising steps.Source: §5.2 · p. 14, Figure 3 · p. 14
  • Uniformly spaced answer-masking timesteps yield more stable rewards and faster convergence than fully random timestep selection, approximating Monte Carlo averaging in log-likelihood estimation.The procedure samples a starting timestep, then spaces the remaining denoising timesteps uniformly across the 1,000-step diffusion process.Source: §5.2 · p. 14, Figure 4 · p. 14

5.1 Ablation Study

  • Stage 2 Mixed Long-CoT fine-tuning strengthens mathematical and geometric reasoning, and Stage 3 UniGRPO brings performance to levels comparable to state-of-the-art methods across reasoning and image-generation tasks.After Stage 1, MMaDA still lags behind most baselines; Stage 3 improvements include mathematical reasoning, geometric problem-solving, CLIP Score, and ImageReward.Source: §5.1 · p. 14, Table 5 · p. 15

5.3 Synergy Across Various Tasks

  • During Stage 2 (120K–200K steps), metrics improve across text generation, multimodal understanding, and image generation, alongside increasingly coherent and aligned textual and visual outputs.The reported qualitative changes include more informative, logically structured text, more precise and grounded visual descriptions, and images better aligned with prompts.Source: §5.3 · p. 15, Figure 6 · p. 15, Figure 5 · p. 15

5.4 Sampling Efficiency

  • Image generation retains strong performance with 15 or 50 denoising steps, while text and multimodal tasks produce coherent outputs with one-quarter or one-half of the full steps.Generation starts from 1,024 masked tokens and allows up to 1,024 denoising steps; diffusion generates tokens in parallel within each step, reducing forward passes versus sequential autoregressive generation.Source: §5.4 · pp. 15–16, Table 6 · p. 16

5.5 Task Extension

  • Without additional fine-tuning, MMaDA supports inpainting in text, visual question answering, and images by framing each as masked-token prediction.The examples include predicting missing text spans, completing answers given an image and partial input, and filling in images from incomplete visual prompts.Source: §5.5 · p. 16

6 Related Work

Related work spans multimodal understanding, diffusion and autoregressive visual generation, and unified vision-language models; MMaDA uses diffusion across textual and visual content to address gaps in concurrent reasoning and post-training.

Multimodal Large Language Models for Multimodal Understanding

  • Most MLLMs use modality-specific encoders and autoregressive generation, but usually cannot perform textual and multimodal reasoning concurrently [51] [63] [64] [65] [67] [68] [69] [70].MMaDA develops diffusion foundation models to fill this gap.Source: §6 · p. 16

Diffusion Models and Autoregressive Models for Visual Generation

  • Diffusion models have succeeded in visual generation, while autoregressive methods such as LlamaGen [58] and VAR [100] still lag behind them.MMaDA models textual and visual content with diffusion and supports efficient AR or Semi-AR sampling.Source: §6 · p. 17

Unified Vision-Language Foundation Models

  • Unified multimodal models combine understanding and generation, but autoregressive approaches can struggle with visual generation and effective post-training remains underexplored.Some systems use a unified transformer, while Transfusion [21] and Show-o [20] combine autoregressive text modeling with diffusion for visual generation.Source: §6 · p. 17

7 Conclusion

MMaDA unifies textual reasoning, multimodal understanding, and generation in a single diffusion framework, with experiments showing performance comparable to or better than specialized models. The work identifies its 8B parameter size as a limitation and plans to use a larger model.

7 Conclusion

  • Experiments across diverse vision-language tasks show MMaDA is comparable to or better than specialized models.The authors cite these results as evidence of diffusion models’ potential as a foundation paradigm for multimodal intelligence.Source: §7 · p. 17
  • MMaDA integrates textual reasoning, multimodal understanding, and generation within a single probabilistic framework.The work describes itself, to the best of its knowledge, as the first systematic exploration of diffusion-based foundation-model design, and proposes novel post-training strategies.Source: §7 · p. 17
  • MMaDA’s current 8B-parameter size is a limitation, and the authors plan to use a larger model.Source: §7 · p. 17

Appendix … A.2 Proximal Policy Optimization and Group Relative Policy Optimization

The appendix reviews discrete diffusion and mask-token prediction, then summarizes PPO and GRPO as policy-optimization methods. It explains how token corruption and reconstruction work and how the methods constrain or compare policy updates.

A.1 Discrete Diffusion and Mask Token Prediction

  • Show-o [20] unifies discrete diffusion and mask-token prediction for joint optimization of noise corruption and semantic recovery.The appendix situates this approach among discrete-token generation methods including VQ-Diffusion [105], MaskGIT [106], and Muse [107].Source: §A.1 · p. 24

A.1.1 Forward and Reverse Diffusion Processes

  • The forward process corrupts tokens through a fixed Markov chain, while a learned reverse model reconstructs the original data by sequentially sampling earlier states.The appendix notes that corruption can replace tokens with uniform noise or a special [MASK] token.Source: §A.1.1 · p. 24

A.1.2 Transition Matrix Design and Masking Strategy

  • Uniform-noise transitions can cause abrupt semantic changes, so Show-o [20] uses a mask-and-replace strategy that signals corrupted positions for reconstruction.Ordinary tokens may be retained, uniformly diffused, or replaced by [MASK], while [MASK] remains unchanged; the transition probabilities are normalized by αt + Kβt + γt = 1.Source: §A.1.2 · pp. 24–25

A.1.3 Variational Objective and Loss Simplification

  • Show-o [20] simplifies training by predicting masked tokens in x0 from noised xt, avoiding explicit modeling of the full posterior and reducing computational complexity.The objective follows MaskGIT [106].Source: §A.1.3 · p. 25

A.2 Proximal Policy Optimization and Group Relative Policy Optimization

  • PPO [110] stabilizes policy optimization by clipping the importance-sampling ratio, and its advantage estimates are typically computed with Generalized Advantage Estimation [111].The clipping mechanism constrains updates to a proximal region around the previous policy.Source: §A.2 · p. 25
  • GRPO [27] removes the value-function requirement, normalizes rewards within groups of responses, and adds a KL penalty between current and reference policies.It averages per-token losses within each sequence before averaging across the group, which the appendix describes as enhancing stability in multi-response settings.Source: §A.2 · pp. 25–26

B Details of UniGRPO

UniGRPO adapts GRPO training for diffusion language models by keeping questions visible and varying answer masking across iterations, addressing limitations in prior log-probability estimation and diffusion-RL approaches.

LLaDA

  • LLaDA estimates log p(q, a) by averaging masked-region logits across 128 Monte Carlo mask ratios, but its computational overhead makes it impractical for on-policy RL.Each batch input combines the question and answer, with answer tokens masked according to a sampled ratio.Source: B Details of UniGRPO · p. 26

d1 (diff-GRPO)

  • d1 introduced the first GRPO-style RL algorithm for diffusion language models, using one forward pass with masked question tokens and a fully masked answer instead of Monte Carlo simulation.Its stochasticity comes from selecting different question tokens to mask across iterations, even when the mask ratio is unchanged.Source: B Details of UniGRPO · p. 26

Prior approach limitations

  • Unlike d1, which randomly masks questions and fully masks answers, UniGRPO aligns question inputs with inference and avoids restricting training to initial denoising.The paper argues question masking adds task-irrelevant stochasticity, while full answer masking may make the model behave like a single-step predictive AR model and underuse multi-step denoising.Source: B Details of UniGRPO · p. 26

UniGRPO modifications

  • UniGRPO keeps question tokens unmasked and varies answer-mask ratios across iterations, exposing the model to diverse denoising stages and supporting multi-step diffusion learning.The ratio is sampled uniformly anew or cycled through discrete steps, from nearly fully masked to almost fully denoised answers.Source: B Details of UniGRPO · pp. 26–27

Training procedure

  • Each UniGRPO iteration samples an answer-mask ratio, runs one forward pass with the question unmasked, computes masked-region token log-likelihoods, applies the GRPO objective, and updates the policy.Source: B Details of UniGRPO · p. 27

C Qualitative Comparisons (with Reasoning CoT)

The qualitative examples pair world-knowledge-aware image generation with textual and multimodal reasoning. They show MMaDA producing explicit reasoning and answers across these tasks, including cases where model responses differ.

Qualitative Comparison of World Knowledge-Aware Text-to-Image Generation

  • Across three prompts, MMaDA identifies a polar bear, the Eiffel Tower, and vodka, then generates corresponding image descriptions.The descriptions depict a polar bear on snow, the Eiffel Tower under a cloudy sky, and a vodka bottle beside a decanter and lime.Source: C Qualitative Comparisons (with… · pp. 28–29

Qualitative Comparison of Textual Reasoning (1)

  • For the fruit-balance problem, MMaDA sets up two weight equations and concludes that one pear balances 7 plums.Its derivation substitutes A = R − 6M into 3A + R = 10M to obtain R = 7M.Source: C Qualitative Comparisons (with… · p. 30

Qualitative Comparison of Textual Reasoning (2) and (3)

  • MMaDA’s worked examples give 60 positive divisors for n = 7^3 × 11^2 × 13^4 and identify 961 as the largest Hope Number within 1000.The divisor count uses (3 + 1)(2 + 1)(4 + 1); the Hope Number solution uses the fact that numbers with an odd divisor count are perfect squares.Source: C Qualitative Comparisons (with… · pp. 31–32

Qualitative Comparison of Multimodal Reasoning (2)

  • For the purple-cylinder question, MMaDA answers that two large objects share its material, while Janus Pro 7B’s response identifies one.Janus names the large green cylinder as the matching object.Source: C Qualitative Comparisons (with… · p. 34

Qualitative Comparison of Multimodal Reasoning (4)

D Qualitative Examples (without Reasoning CoT)

MMaDA can handle general, simple tasks without generating chain-of-thought, as illustrated by qualitative multimodal-understanding examples that describe varied scenes.

D Qualitative Examples (without Reasoning CoT)

Qualitative Example of Multimodal Understanding (1)

Qualitative Example of Multimodal Understanding (2)

Qualitative Example of Multimodal Understanding (3)

Qualitative Example of Multimodal Understanding (4)

Related papers

Loading 2505.15809v2…