PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing

Dianyi Wang, Ruihang Li, Feng Han, Chaofan Ma, Wei Song, Siyuan Wang, Yibin Wang, Yi Xin, Hongjian Liu, Zhixiong Zhang, Shengyuan Ding, Tianhang Wang, Zhenglin Cheng, Tao Lin, Cheng Jin, Kaicheng Yu, Jingjing Chen, Wenjie Wang, Zhongyu Wei, Jiaqi Wang

arXiv:2602.12205v2cs.CVcs.AI

TL;DR

Unified multimodal generation and editing often rely on costly, large-scale models, motivating a more efficient alternative. DeepGen 1.0 combines a compact VLM-DiT architecture with SCB and progressive training, achieving competitive or superior results across diverse benchmarks while using about 50M samples.

  • Problem

    Existing high-performing unified models require massive parameter counts, training samples, and computational resources, while smaller unified models underperform across diverse generation and editing tasks.

  • Method

    DeepGen 1.0 uses a 5B VLM-DiT model with SCB hierarchical feature fusion and think tokens, trained through alignment pre-training, joint SFT, and MR-GRPO reinforcement learning.

  • Results

    DeepGen 1.0 achieves competitive or superior performance across generation, reasoning, and editing benchmarks, including 0.73 on WISE versus 0.57 for the 80B HunyuanImage 3.0 and 77.5 on UniREditBench versus 56.5 for 27B Qwen-Image-Edit.

  • Takeaways & Limitations

    The results support a compact unified model as an efficient alternative to massive parameter scaling, and the released framework and data components enable broader research access.

  • Takeaways & Limitations

    A KL penalty alone is insufficient to prevent capability degradation when reinforcement learning extends beyond approximately 1000 steps.

Abstract

from arXiv · show

Current unified multimodal models for image generation and editing typically rely on massive parameter scales (e.g., >10B), entailing prohibitive training costs and deployment footprints. In this work, we present DeepGen 1.0, a lightweight 5B unified model that achieves comprehensive capabilities competitive with or surpassing much larger counterparts. To overcome the limitations of compact models in semantic understanding and fine-grained control, we introduce Stacked Channel Bridging (SCB), a deep alignment framework that extracts hierarchical features from multiple VLM layers and fuses them with learnable 'think tokens' to provide the generative backbone with structured, reasoning-rich guidance. We further design a data-centric training strategy spanning three progressive stages: (1) Alignment Pre-training on large-scale image-text pairs and editing triplets to synchronize VLM and DiT representations, (2) Joint Supervised Fine-tuning on a high-quality mixture of generation, editing, and reasoning tasks to foster omni-capabilities, and (3) Reinforcement Learning with MR-GRPO, which leverages a mixture of reward functions and supervision signals, resulting in substantial gains in generation quality and alignment with human preferences, while maintaining stable training progress and avoiding visual artifacts. Despite being trained on only ~50M samples, DeepGen 1.0 achieves leading performance across diverse benchmarks, surpassing the 80B HunyuanImage by 28% on WISE and the 27B Qwen-Image-Edit by 37% on UniREditBench. By open-sourcing our training code, weights, and datasets, we provide an efficient, high-performance alternative to democratize unified multimodal research.

1 Introduction

DeepGen 1.0 addresses the cost and scale barriers of unified multimodal generation and editing with a compact 5B model, SCB alignment, and a three-stage data-centric training strategy. It reports competitive or leading performance across generation, reasoning, and editing benchmarks while using substantially fewer training samples than larger systems.

  • Motivation: Existing unified models often require billions of samples, massive computation, or separate generation and editing models, while smaller models underperform across diverse tasks.These constraints motivate a lightweight unified alternative.
  • Model and contributions: DeepGen 1.0 combines a 3B VLM and 2B DiT in a single 5B framework supporting generation, reasoning generation, text rendering, general editing, and reasoning editing.The model targets comprehensive capabilities within a compact parameter budget.
  • Results: 87.90 on DPG-Bench exceeds HunyuanImage 3.0’s 86.10, while 0.73 on WISE surpasses the 80B model’s 0.57 by 28%.On UniREditBench, DeepGen 1.0 reaches 77.5 versus 56.5 for the 27B Qwen-Image-Edit, a margin of over 37%.
  • Model and contributions: SCB fuses multi-layer VLM features with learnable think tokens to transfer semantic and reasoning-rich guidance to the DiT while preserving fine-grained visual details.The module uses hierarchical VLM representations rather than only final-layer or pooled features.
  • Training strategy: The training pipeline progresses from alignment pre-training on pairs and triplets, through joint supervised fine-tuning, to MR-GRPO reinforcement learning with mixed rewards and auxiliary supervision.The stages are designed to align VLM and DiT representations, build omni-capabilities, and optimize preferences without capability degradation.
  • Impact: The work releases model weights, training and evaluation code, and key data components to support broader exploration of efficient unified multimodal research.The release is presented as an efficient alternative to resource-intensive large models.

2 Model Architecture

DeepGen 1.0 uses a unified VLM-DiT architecture in which a VLM supplies multimodal understanding and a DiT performs high-fidelity generation. SCB combines hierarchical VLM representations and think-token information before passing aligned conditions to the generative backbone.

  • Architecture overview: The architecture pairs a pretrained 3B Qwen-2.5-VL understanding backbone with a 2B SD3.5-Medium DiT supporting joint generation and editing.A connector aligns the two modules.
  • Architecture overview: The VLM captures multimodal semantics and priors, while the DiT acts as a high-fidelity decoder guided by multimodal conditional inputs.This division assigns understanding to the VLM and image synthesis to the DiT.
  • Stacked Channel Bridging: SCB addresses information loss from single final-layer VLM features by aggregating representations from multiple layers while preserving fine-grained visual information.Prior unified models commonly rely on final- or penultimate-layer hidden states.
  • Stacked Channel Bridging: Learnable think tokens interact with textual and visual inputs across VLM layers, progressively summarizing representations to strengthen reasoning.They function as implicit chain-of-thought representations in the alignment process.
  • Stacked Channel Bridging: Six uniformly sampled hidden states span low-, mid-, and high-level VLM layers, capturing varying-granularity visual features, semantics, and think-token reasoning information.The selection balances representation coverage with computational efficiency.
  • Stacked Channel Bridging: Selected hidden states are concatenated along the channel dimension, projected with a lightweight two-layer MLP, and deeply fused by a Transformer-encoder connector.The resulting aligned features are prepared to match the DiT input width.

3 Training

DeepGen 1.0 is trained through staged VLM-DiT alignment, joint supervised optimization, and reinforcement learning. The RL design combines reward optimization with regularization and supervised guidance to improve quality while limiting capability degradation.

  • Stage 1: Alignment Pre-Training: Stage 1 trains only the connector and 128 learnable think tokens to align VLM representations with the DiT latent space.The stage uses general text-to-image generation and image-editing tasks.
  • Stage 2: Joint Supervised Fine-tuning: Stage 2 jointly trains the VLM and DiT to strengthen instruction following, image synthesis quality, visual fidelity, semantic alignment, and knowledge-aware reasoning.LoRA is applied to the VLM during joint optimization.
  • Stage 3: Reinforcement Learning: MR-GRPO extends GRPO to flow-matching models using on-policy sampling and a mixture of pointwise and pairwise reward models.The framework also incorporates noise-preserving stochastic sampling and an auxiliary supervised diffusion loss.
  • Stage 3: Reinforcement Learning: Three complementary reward functions jointly optimize visual quality, text-rendering accuracy, and semantic alignment.Advantages are normalized independently for each reward before aggregation, then normalized across the training batch.
  • Stage 3: Reinforcement Learning: KL regularization constrains policy drift, while auxiliary SFT guidance anchors generation quality to the supervised fine-tuning distribution.The total objective mixes GRPO and SFT losses, with λ as a small mixing coefficient.
  • Stage 3: Reinforcement Learning: ∼0.747 to ∼0.756 overall and ∼0.25 to ∼0.34 text scores improve steadily during 1,500 RL steps on UniGenBench.The curves indicate simultaneous gains in general generation quality and text-rendering fidelity.

4 Data

DeepGen 1.0 uses a broad mixture of real, synthetic, and curated open-source data. The training data covers general generation and editing, reasoning, text rendering, and application-oriented scenarios.

  • Data Composition: The training corpus combines real-world, synthetic, and curated open-source datasets across general generation, editing, reasoning, text rendering, and application-oriented tasks.The overall composition is illustrated in Figure 4.
  • General Generation: General-generation data includes large image-text pre-training corpora and curated instruction-following datasets.The listed sources include text-to-image-2M, LAION-Aesthetic-6M, Megalith-10M, RedCaps-5M, CC-12M, BLIP-3o, ShareGPT-4o-Image, Echo-4o-Image, and OpenGPT4o-Image.
  • General Editing: General-editing data uses image-instruction-image triplets from multiple public datasets together with 1.1M in-house editing samples.The cited public sources include NHR-Edit, GPT-Image-Edit, Pico-Banana, X2I2, and Uniworld-Edit.
  • Reasoning-based Generation and Editing: Reasoning datasets provide 150k generation samples and 100k editing samples spanning cultural, scientific, spatial, temporal, and logical reasoning.These datasets come from UniReason.
  • Text Rendering and Application-oriented Scenarios: Text-rendering data contains 500k synthesized samples with varied fonts, layouts, and color schemes derived from document- and infographic-focused prompts.Gemini 2.5 Pro composes rendering attributes, and Qwen-Image synthesizes the corresponding images.

5 Experiments

The experiments evaluate DeepGen 1.0 across general generation, reasoning generation, general editing, reasoning editing, and text rendering. The benchmark suite covers instruction following, semantic alignment, reasoning, editing consistency, output quality, and rendered-text accuracy.

  • General Generation: GenEval, DPG-Bench, and UniGenBench assess general text-to-image generation, including semantic alignment, long-prompt instruction following, and fine-grained capabilities.UniGenBench covers categories such as attribute binding, style control, and text rendering.
  • Reasoning Generation: WISE and T2I-CoREBench evaluate reasoning-based generation using world knowledge and eight reasoning categories.WISE covers cultural knowledge, natural science, and spatial-temporal understanding; T2I-CoREBench includes logical, behavioral, hypothetical, procedural, generalization, analogical, commonsense, and reconstructive reasoning.
  • General Editing: ImgEdit and GEdit-EN measure general editing through instruction following, editing consistency, and output quality.DeepGen 1.0 remains highly competitive on these benchmarks and ranks third under RL in the reported comparison.
  • Reasoning Editing: UniREditBench and RISE evaluate world-knowledge reasoning-based editing across real-world, game-world, temporal, causal, spatial, and logical scenarios.UniREditBench contains 2,700 samples, while RISE contains 327 samples.
  • Text Rendering: CVTG-2K evaluates English text rendering across street scenes, advertisements, and memes.Table 5 reports text-rendering evaluation on CVTG-2K.

5.2 Model Performance

DeepGen 1.0 delivers a strong performance–efficiency trade-off across generation, editing, reasoning, and text rendering while using only 5B parameters. It consistently matches or surpasses larger unified and specialized models across diverse benchmarks.

  • Overall performance: With only 5B parameters, DeepGen 1.0 ranks among the top three across all evaluated generation and editing settings while using one unified model.It avoids relying on separate specialized generation and editing models.
  • General generation: 0.87 on GenEval matches leading models, while 87.90 on DPGBench and 75.74 on UniGenBench rank DeepGen 1.0 highly among open-source systems.The model uses significantly fewer parameters than several competitors and no external LLM-based prompt rewriting.
  • General editing: DeepGen 1.0 remains competitive on ImgEdit and GEdit-EN, ranking third under RL and outperforming Mammoth2, BAGEL, and OmniGen2.It approaches larger edit-specialized models such as Qwen-Image-Edit and LongCat-Image-Edit.
  • Reasoning-based generation: 0.73 on WISE is the best result among open-source models, while 46.5 on T2I-CoREBench ranks DeepGen 1.0 among the top open-source models.The results span cultural, temporal, spatial, natural-scientific, logical, procedural, analogical, commonsense, and reconstructive reasoning.
  • Reasoning-based editing: 13.3 on RISE ranks first with SFT, while 77.5 on UniREditBench with SFT and 75.7 under RL outperform other open-source baselines.The UniREditBench result also exceeds the closed-source GPT-Image-1 overall score.
  • Text rendering: 0.6605 to 0.7533 improvement in Word Accuracy under RL enhances text correctness while preserving the highest open-source CLIPScore of 0.8278.The results indicate improved textual fidelity without compromising semantic alignment.

5.3 Ablation Study

Ablations show that SCB, learnable think tokens, VLM activation, and the MR-GRPO training safeguards each contribute to DeepGen 1.0’s performance or training stability. Removing these components degrades benchmark results, destabilizes optimization, or slows progress.

  • Architecture ablations: Removing SCB degrades DPGBench from 87.05 to 85.55, GEdit from 7.12 to 6.75, WISE from 0.72 to 0.70, and RISE from 13.3 to 12.6.The study attributes this to information loss from single-layer conditioning compared with aggregating multiple VLM layers.
  • Architecture ablations: Removing think tokens decreases WISE from 0.72 to 0.68 and RISE from 13.3 to 11.7, the largest regression on reasoning-intensive benchmarks.The paper characterizes think tokens as an implicit reasoning buffer that distills knowledge from VLM representations.
  • Architecture ablations: Disabling VLM activation reduces performance, including GenEval 0.85, GEdit 6.93, WISE 0.71, and RISE 12.9.The ablation indicates that modest VLM fine-tuning improves alignment with the DiT and downstream tasks.
  • RL training ablations: Removing the auxiliary SFT loss causes performance to degrade after approximately 300 steps and makes text-rendering improvement slower and more erratic.The full configuration uses auxiliary SFT loss to preserve generation quality during extended RL training.
  • RL training ablations: Removing KL regularization lowers UniGenBench from 75.69 to 75.07 and DPGBench from 87.75 to 87.32, with training curves indicating capability forgetting.The paper links the degradation to unconstrained policy updates after supervised fine-tuning.
  • RL training ablations: Joint reward normalization falls behind after approximately 600 steps and reduces text generation score from 35.06 to 32.18.Reward-wise normalization stabilizes multi-reward optimization by preventing high-variance rewards from dominating updates.

6 Conclusion

DeepGen 1.0 shows that a compact 5B unified model can integrate generation, reasoning, and editing without relying on massive parameter scaling. The authors present this efficiency as a route toward more accessible unified multimodal research.

  • Conclusion: DeepGen 1.0 integrates image generation and editing in a compact 5B-parameter framework while supporting comprehensive generation, reasoning, and editing capabilities.The conclusion reports performance that rivals substantially larger systems, particularly on reasoning-intensive and instruction-following tasks.
  • Conclusion: The authors connect the model’s reduced computational requirements and open-source release with broader access to unified multimodal research.They specifically frame consumer-grade hardware and democratized research as broader implications.

A Pre-Training & SFT Details

The appendix provides dataset-usage details and hyperparameter configurations for DeepGen 1.0’s pre-training and supervised fine-tuning stages.

  • Pre-Training & SFT Details: Tables 8 and 9 together provide dataset usage and hyperparameter details for the two training stages.These references serve as implementation details accompanying the main method description.
  • Pre-Training & SFT Details: Table 8 documents the data used in the pre-training and supervised fine-tuning stages.A dagger marks entries covering both Chinese and English prompts.
  • Pre-Training & SFT Details: Table 9 lists detailed hyperparameters and configurations for pre-training and supervised fine-tuning.The supplied passage identifies the table’s scope but does not enumerate individual settings.

B Reinforcement Learning Details

DeepGen’s reinforcement-learning pipeline combines noise-preserving stochastic sampling, multiple reward functions, and auxiliary supervised training to improve exploration while preserving generation capabilities. Prompt categories and reward weights are tailored toward either text-rendering accuracy or holistic image quality.

  • Sampling: Noise-preserving stochastic sampling keeps noise consistent with the flow-matching scheduler at every timestep, producing cleaner samples and more accurate reward signals.The standard Flow-SDE formulation can exceed the scheduler’s expected noise level, degrading sample quality and reward accuracy.
  • Reward Functions: Three complementary rewards assess pairwise preference, OCR-based text accuracy, and CLIP-based semantic consistency.The VLM-based reward evaluates alignment and visual quality through within-group comparisons, while OCR directly compares rendered text with the prompt target.
  • Reward Weights: Reward weighting prioritizes OCR for text-rendering prompts and the VLM-based preference reward for general text-to-image prompts.This assigns text-rendering optimization to direct text-accuracy feedback while using preference-based holistic assessment for general generation.
  • Training Data: Text-rendering prompts receive 3.0× sampling weight, whereas general text-to-image prompts receive 1.0× weight.The prompt mixture draws text-rendering data from UniGenBench and curated sources, and general prompts from UniGenBench, BLIP3-o, ShareGPT-4o, and CoREBench.
Loading 2602.12205v2…