PaperlayerView exact revision on arXiv

Source-linked AI summary

MMaDA: Multimodal Large Diffusion Language Models

Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, Mengdi Wang

arXiv:2505.15809v2cs.CV

TL;DR

Unified multimodal diffusion models have left post-training, especially in non-autoregressive settings, comparatively underexplored. MMaDA combines a shared modality-agnostic diffusion architecture with mixed long-CoT fine-tuning and UniGRPO, achieving superior and balanced performance across textual reasoning, multimodal understanding, and text-to-image generation.

  • Problem

    Existing unified multimodal foundation models predominantly emphasize architecture and pretraining, leaving post-training methods for non-autoregressive diffusion settings underexplored.

  • Method

    MMaDA combines a shared probabilistic, modality-agnostic diffusion architecture with mixed long-CoT fine-tuning and UniGRPO using diversified rewards.

  • Results

    MMaDA achieves superior and balanced performance across textual reasoning, multimodal understanding, and text-to-image generation.

  • Takeaways & Limitations

    MMaDA provides a unified framework bridging pretraining and post-training in multimodal diffusion models.

Abstract

from arXiv · show

We introduce MMaDA, a novel class of multimodal diffusion foundation models designed to achieve superior performance across diverse domains such as textual reasoning, multimodal understanding, and text-to-image generation. The approach is distinguished by three key innovations: (i) MMaDA adopts a unified diffusion architecture with a shared probabilistic formulation and a modality-agnostic design, eliminating the need for modality-specific components. This architecture ensures seamless integration and processing across different data types. (ii) We implement a mixed long chain-of-thought (CoT) fine-tuning strategy that curates a unified CoT format across modalities. By aligning reasoning processes between textual and visual domains, this strategy facilitates cold-start training for the final reinforcement learning (RL) stage, thereby enhancing the model's ability to handle complex tasks from the outset. (iii) We propose UniGRPO, a unified policy-gradient-based RL algorithm specifically tailored for diffusion foundation models. Utilizing diversified reward modeling, UniGRPO unifies post-training across both reasoning and generation tasks, ensuring consistent performance improvements. Experimental results demonstrate that MMaDA-8B exhibits strong generalization capabilities as a unified multimodal foundation model. It surpasses powerful models like LLaMA-3-7B and Qwen2-7B in textual reasoning, outperforms Show-o and SEED-X in multimodal understanding, and excels over SDXL and Janus in text-to-image generation. These achievements highlight MMaDA's effectiveness in bridging the gap between pretraining and post-training within unified diffusion architectures, providing a comprehensive framework for future research and development. We open-source our code and trained models at: https://github.com/Gen-Verse/MMaDA

1 Introduction

MMaDA addresses the limited exploration of post-training for unified multimodal diffusion models by combining a modality-agnostic diffusion architecture with mixed long-CoT fine-tuning and UniGRPO. It targets textual reasoning, multimodal understanding, and text-to-image generation within one framework.

  • Existing unified multimodal foundation models leave post-training methodologies, particularly for non-autoregressive settings, underexplored.
  • MMaDA uses a shared probabilistic formulation and modality-agnostic diffusion architecture without modality-specific components.The design is intended to extend diffusion models into generalist task solvers.
  • Mixed long-CoT fine-tuning aligns reasoning processes across textual and visual modalities and enables cold-start training.The unified CoT format supports learning intermediate reasoning before final output generation.
  • UniGRPO is a diffusion-centric reinforcement-learning algorithm using diversified reward modeling for reasoning and generation tasks.Its rewards can be tailored to varied task requirements under a unified formulation.
  • MMaDA achieves superior and balanced performance across textual reasoning, multimodal understanding, and text-to-image generation.The paper reports comparisons with autoregressive and diffusion-based baselines; qualitative examples include mathematical reasoning and multimodal question answering.

2 MMaDA: Multimodal Large Diffusion Language Models

MMaDA unifies text and image modeling with a shared diffusion objective, then extends post-training through mixed long-CoT finetuning and UniGRPO. These components support cross-modal reasoning, diversified rewards, and task-specific optimization within one framework.

  • 2.1 Pretraining with Unified Diffusion Architecture and Objective: MMaDA uses a modality-agnostic diffusion architecture that predicts masked text and image tokens under one probabilistic formulation.The model predicts all masked tokens simultaneously and computes cross-entropy only on masked tokens.
  • 2.2 Post-Training with Mixed Long-CoT Finetuning: Mixed long-CoT finetuning aligns reasoning trajectories across textual reasoning, multimodal reasoning, and text-to-image generation.The curated data uses a unified structure containing a reasoning process followed by a result.
  • 2.2 Post-Training with Mixed Long-CoT Finetuning: The unified CoT format bridges modality-specific outputs and supports knowledge transfer between tasks.The format is designed to accommodate heterogeneous text and image outputs while preserving step-by-step reasoning before the final result.
  • 2.3 Post-Training with UniGRPO: UniGRPO provides a diffusion-specific policy-gradient framework combining unified mathematical optimization with diversified task rewards.Its reward system can instantiate different rewards for reasoning, multimodal understanding, and world-knowledge-aware image generation.
  • 2.3 Post-Training with UniGRPO: UniGRPO addresses diffusion reinforcement-learning challenges through structured noising and efficient per-token log-likelihood approximation.The method exposes training to multiple denoising stages while avoiding the computational inefficiency of extensive mask-ratio sampling.

3 Flexible Sampling Strategies at Inference Time

MMaDA uses different inference schedules for text and images: semi-autoregressive denoising for text and parallel non-autoregressive sampling for image generation. These strategies control the denoising order and masking schedule for each modality.

  • Text Generation: Text generation partitions the output into blocks processed from left to right, while denoising masked positions within each block.Tokens are selected randomly or by confidence under a linear masking schedule.
  • Text Generation: 512 denoising steps are used for 1,024-token text sequences divided into blocks of 64 tokens.At each step, two lowest-confidence tokens in the current block are unmasked.
  • Text Generation: Semi-autoregressive denoising produces more intricate and detailed descriptions than fixed-length non-autoregressive generation in the qualitative comparison.The comparison uses sequence length 1,024, 512 steps, and block sizes of 64 versus 1,024.
  • Image Generation: Image generation treats the entire 1,024-token output as one block and applies low-confidence remasking with a cosine noise schedule.Evaluation uses 50 denoising timesteps, 512×512 resolution, and classifier-free guidance at scale 3.5.

4 Experiments

MMaDA is trained on diverse language, multimodal, instruction, and reasoning data and evaluated across understanding, image generation, and language benchmarks. The reported results show competitive or superior performance across these task families despite a unified objective.

  • 4.1 Experimental Setup: MMaDA combines foundational language and multimodal data with instruction-tuning and reasoning datasets across its training stages.The implementation uses RefinedWeb, open image-text datasets, Alpaca, LLaVA-1.5, and curated reasoning data.
  • 4.2 Multimodal Understanding: MMaDA achieves comparable or superior multimodal-understanding results against dedicated models across POPE, MME, Flickr30k, VQAv2, GQA, and MMMU.The model also consistently outperforms several unified baselines on multiple benchmarks.
  • 4.3 Image Generation: MMaDA achieves the highest CLIP Score and ImageReward among the reported generation-only and unified models.The authors attribute these results to UniGRPO rewards aligned with the evaluation metrics.
  • 4.3 Image Generation: MMaDA demonstrates compositionality, object counting, and world-knowledge-aware text-to-image generation capabilities on GenEval and WISE.The image-generation evaluation includes textual-alignment, human-preference, general, and world-knowledge-aware metrics.
  • 4.4 Textual Reasoning: MMaDA matches strong baselines on MMLU and ARC-C while consistently outperforming LLaDA-8B on mathematical benchmarks.The evaluated language tasks include MMLU, ARC-C, TruthfulQA, GSM8K, MATH, and GPQA.

5 Observations, Analysis and Conclusion

Ablations and analyses show that Mixed Long-CoT finetuning and UniGRPO improve reasoning and generation, while unified training produces gains across task categories. Diffusion sampling also offers efficiency and inpainting flexibility.

  • 5.1 Ablation Study: Mixed Long-CoT finetuning substantially improves mathematical and geometric reasoning after pretraining.The ablation reports that Stage 1 still trails most baselines, whereas Stage 2 strengthens reasoning capabilities.
  • 5.1 Ablation Study: UniGRPO further improves mathematical reasoning, geometric problem-solving, and image-generation metrics to state-of-the-art-comparable levels.The reported image-generation metrics include CLIP Score and ImageReward.
  • 5.2 Design Choices of UniGRPO: Uniformly spaced masking approximates Monte Carlo averaging in log-likelihood estimation, improving training stability and convergence speed.The strategy samples a starting timestep and spaces remaining denoising timesteps across the diffusion process.
  • 5.3 Synergy Across Modalities: All key metrics improve consistently during Stage 2 across text generation, multimodal understanding, and image generation.The authors describe this pattern as synergy across the three task categories, accompanied by increasingly complex and coherent outputs.
  • 5.4 Sampling Efficiency: Text and multimodal outputs can remain coherent with one-quarter or one-half of the full denoising steps.Image generation is also evaluated under varying denoising-step counts, highlighting diffusion models’ sampling-efficiency potential.
  • 5.5 Inpainting and Extrapolation: MMaDA supports text-span completion, visual-question-answer completion, and image inpainting without additional fine-tuning.These tasks are formulated as masked-token prediction problems within the diffusion objective.

6 Related Work

Unified vision-language research has pursued models that support both multimodal understanding and generation. Recent approaches use token-based unified transformers, diffusion, or hybrid objectives to bridge modalities.

  • Earlier multimodal systems combined language and diffusion models to process discrete text and continuous images separately.
  • Discrete diffusion methods model visual tokens through denoising processes, including approaches that use masked-token prediction.
  • Unified vision-language models aim to support both multimodal understanding and generation within one foundation model.
  • Token-based unified transformers represent multiple modalities as sequences and train them end-to-end with shared architectures.

7 Conclusion

MMaDA unifies textual reasoning, multimodal understanding, and generation under one diffusion foundation-model framework. The authors report broad competitiveness while identifying current model size as a limitation.

  • MMaDA integrates textual reasoning, multimodal understanding, and generation within a single probabilistic framework.
  • MMaDA is reported as comparable to or better than specialized models across diverse vision-language tasks.
  • The current 8B-parameter model size limits performance, motivating future use of larger models.

A.1 Discrete Diffusion and Mask Token Prediction

Discrete diffusion progressively corrupts token sequences and trains a reverse model to reconstruct them. Mask-based objectives and PPO/GRPO-style optimization provide complementary mechanisms for denoising and reinforcement learning.

  • Discrete diffusion: The forward process corrupts discrete tokens through a fixed Markov chain, while the learned reverse process reconstructs the original sequence.
  • Discrete diffusion: The direct transition q(x_t|x_0) is computed by multiplying transition matrices across diffusion steps.
  • Transition design: Uniform noise uses retention probability α_t and category-wise replacement probability β_t, but can cause abrupt semantic changes.
  • Mask token prediction: Mask-and-replace diffusion adds a [MASK] state so the reverse network can focus on corrupted regions.
  • Mask token prediction: Mask-token prediction simplifies training by reconstructing only corrupted regions instead of explicitly modeling the full posterior.
  • Policy optimization: PPO constrains policy updates with clipping, while GRPO uses group-relative advantages and an explicit KL penalty against a reference policy.

B Details of UniGRPO

UniGRPO modifies prior diffusion-RL masking schemes to better match question-answering conditions and expose training to multiple denoising stages. Its design contrasts with Monte Carlo estimation and fully masked-answer training.

  • Prior methods: LLaDA estimates answer log probabilities with Monte Carlo simulation over multiple randomly sampled mask ratios.
  • Prior methods: d1 reduces computation to one forward pass but randomly masks question tokens and fully masks answer tokens.
  • Limitations: Random question masking has unclear practical significance because questions are fully observed during typical training and inference.
  • Limitations: Fully masking answers trains primarily the initial denoising step and may underuse diffusion’s multi-step capabilities.
  • UniGRPO: UniGRPO keeps questions unmasked and varies answer-mask ratios across iterations to expose the model to diverse denoising stages.

C Qualitative Comparisons (with Reasoning CoT)

The qualitative comparisons present MMaDA responses for world knowledge, textual reasoning, multimodal reasoning, and image generation. The examples show correct answers in several tasks, while comparison models are marked as unsupported on some textual-reasoning questions.

  • World Knowledge-Aware Text-to-Image Generation (3): In another world-knowledge example, MMaDA identifies the Eiffel Tower and generates an image showing the landmark against a cloudy, muted sky.The textual answer names the Eiffel Tower, while the generated description specifies its hollow structure and surrounding sky.
  • World Knowledge-Aware Text-to-Image Generation (1): MMaDA identifies vodka from the prompt “A traditional strong spirit from Russia” and generates an image featuring a vodka bottle, decanter, and lime.The reasoning response gives vodka as the answer, followed by the depicted scene.
  • Textual Reasoning: Show-o, Emu3, and Janus Pro 7B are each marked as unsupported on the presented textual-reasoning questions.The unsupported-task labels appear for the fruit-balancing, divisor-count, and Hope Number comparisons.
  • Textual Reasoning (1): MMaDA answers the fruit-balancing problem correctly: one pear weighs as much as 7 plums.The solution derives R = 7M and verifies that an apple weighs one plum.
  • Textual Reasoning (2): For n = 7^3 × 11^2 × 13^4, MMaDA applies the divisor formula and obtains 60 positive divisors.The exponents 3, 2, and 4 produce (3 + 1)(2 + 1)(4 + 1) = 60.
  • Textual Reasoning (3): MMaDA concludes that the largest Hope Number within 1000 is 961, because numbers with an odd divisor count are perfect squares.It compares 31^2 = 961 with 32^2 = 1024.
  • Multimodal Reasoning (1): On the square-inscribed-in-a-circle problem, MMaDA produces both 90-degree and 45-degree answers across the displayed responses.The passages include a 90-degree answer and a separate derivation concluding 45 degrees for angle APB.

D Qualitative Examples (without Reasoning CoT)

MMaDA handles simple tasks without generating chain-of-thought, while qualitative examples show detailed descriptions of diverse visual scenes. These examples include dining areas, European streets, dolphins at sunset, and sunset landscapes.

  • MMaDA can efficiently handle general and simple tasks without generating CoT.
  • Another example describes dolphins swimming in calm ocean water during a colorful sunset.
  • A landscape example describes a sunset over a field of yellow flowers using broad, expressive brushstrokes.
  • A detailed description identifies a minimalist dining area with a dark table, blue velvet chairs, large windows, and a round mirror.
  • A detailed description characterizes a narrow European street with pastel buildings, ornate balconies, cobblestones, tram tracks, and a red tram.
Loading 2505.15809v2…