Source-linked AI summary
HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer
Qi Cai, Jingwen Chen, Chengmin Gao, Zijian Gong, Yehao Li, Yingwei Pan, Yi Peng, Zhaofan Qiu, Kai Yu, Yiheng Zhang, Hao Ai, Siying Bai, Yang Chen, Zhihui Chen, Fengbin Gao, Ying Guo, Dong Li, Zhen Shen, Leilei Shi, Jing Wang, Siyu Wang, Yimeng Wang, Rui Zheng, Ting Yao, Tao Mei
TL;DR
Existing visual generators commonly separate VAEs and text encoders, motivating a more unified pixel-space architecture. HiDream-O1-Image maps pixels, text, and task conditions into one token space, achieving strong results across generation tasks at 8B parameters and scaling to 200B+.
Problem
Existing visual-generation pipelines use fragmented latent compression and disjoint text encoding, motivating structural unification of multimodal inputs.
Method
HiDream-O1-Image uses a pixel-level Unified Transformer that maps raw pixels, text tokens, and task conditions into one shared token space, aided by a reasoning-driven prompt agent.
Results
HiDream-O1-Image performs strongly across text-to-image generation, editing, and subject-driven personalization; its 8B model is competitive with larger systems, while the 200B+ version establishes state-of-the-art benchmarks.
Takeaways & Limitations
The results support natively unified architectures as a scalable framework for versatile multimodal image generation and editing.
Abstract
from arXiv · showhide
The evolution of visual generative models has long been constrained by fragmented architectures relying on disjoint text encoders and external VAEs. In this report, we present HiDream-O1-Image, a natively unified generative foundation model via pixel-space Diffusion Transformer, that pioneers a paradigm shift from modular architectures to an end-to-end in-context visual generation engine. By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs within an Unified Transformer (UiT) architecture. This native encoding paradigm eliminates the need for separate VAEs or disjoint pre-trained text encoders, allowing the model to treat diverse generation and editing tasks as a consistent in-context reasoning process. Extensive experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization. Notably, with only 8B parameters, HiDream-O1-Image (8B) achieves performance parity with or even surpasses established state-of-the-art models with significantly larger parameters (e.g., 27B Qwen-Image). Crucially, to validate the immense scalability of this paradigm, we successfully scale the architecture up to over 200B parameters. Experimental results demonstrate that this massive-scale version HiDream-O1-Image-Pro (200B+) unlocks unprecedented generative capabilities and superior performance, establishing new state-of-the-art benchmarks. Ultimately, HiDream-O1-Image highlights the immense potential of natively unified architectures and charts a highly scalable path toward next-generation multimodal AI.
1. Introduction
HiDream-O1-Image addresses fragmented visual-generation pipelines by unifying pixels, text, and task conditions in a shared token space. It extends this architecture with reasoning-driven prompting and supports diverse generation tasks and scales.
- Latent Diffusion Models rely on modular pipelines whose compression can lose high-frequency visual details and cap generation fidelity.
- Pixel-space Diffusion Transformers improve detail preservation but commonly retain disjoint text encoders, separating visual and textual encoding spaces.
- The work positions shared-token multimodal unification as a scalable route from specialized image generation toward generalist visual reasoning.
- HiDream-O1-Image maps raw pixels, discrete text tokens, and task-specific conditions into one shared token space for unified in-context visual reasoning.
- A Reasoning-Driven Prompt Agent refines complex user instructions before generation to improve instruction following for intricate visual tasks.
- The unified system supports text-to-image generation, instruction-based editing, subject-driven personalization, and outputs up to 2,048 × 2,048 resolution.
2. Data Curation and Prompt Construction
The data engine converts heterogeneous sources into filtered, deduplicated supervision for text-to-image, editing, personalization, and prompt construction. Prompt generation preserves visual, textual, structural, and transformation-specific information.
- Data Curation and Prompt Construction: The data engine converts heterogeneous sources into image-text pairs, editing triplets, and subject-reference samples through collection, deduplication, filtering, and prompt generation.
- 2.1. Source Data Collection: Source data covers standard image-text pairs, graphic layouts, long-text images, style-oriented samples, editing examples, and subject-personalization references.
- 2.2. Data Deduplication: Deduplication groups visual features into clusters and performs within-cluster similarity search using SSCD representations and GPU-accelerated Faiss.
- 2.2. Data Deduplication: Approximately 20% of initial candidates are removed during redundancy control while preserving the semantic coverage of collected data.
- 2.3. Data Quality and Safety Filtering: Quality and safety filtering removes inappropriate, unaesthetic, watermarked, or task-inconsistent samples while retaining useful high-resolution training data.
- 2.4. Prompt Construction: Qwen3-VL transforms metadata and visual-textual signals such as OCR, layout cues, and style labels into descriptive training prompts.
- 2.4. Prompt Construction: The resulting prompts and task conditions feed the Unified Transformer, which predicts clean image patches that are reassembled into target images.
- 2.4. Prompt Construction: Prompt construction preserves salient objects, attributes, spatial relations, scene context, styles, textual content, reading order, and layout structure.
3. Model Architecture: HiDream-O1-Image
HiDream-O1-Image uses a pixel-space Unified Transformer that natively processes text, visual conditions, and target images in one token space. Its prompt agent and hybrid attention support generalist multimodal generation and editing.
- The architecture is instantiated at 8B and 200B+ parameters to demonstrate efficient deployment and structural scalability.
- 3.1. Reasoning-Driven Prompt Agent: The Reasoning-Driven Prompt Agent explicitly reasons about layout, attributes, physical logic, and context before producing aligned prompts.
- 3.2. Unified Multimodal Tokenization: Unified multimodal tokenization encodes refined text, task-specific conditions, and target images into a shared token space.
- 3.2. Unified Multimodal Tokenization: Text uses native vocabulary tokens, conditions represent visual context, and generation images are partitioned into patches projected through learnable embeddings.
- Unified Transformer blocks jointly contextualize all modalities, while a linear head predicts clean image patches for reconstruction.
- The decoder-only Transformer backbone uses timestep tokens and learnable input and output patch embeddings for direct pixel-space diffusion.
- Hybrid unified attention applies causal masking to condition and text tokens but full attention to generation tokens for global context aggregation.
- Training couples flow matching for image prediction with LPIPS and perceptual DINO supervision to balance structural regression and perceptual alignment.
4. Model Training
Training progresses from foundational multimodal alignment to generalist in-context generation and high-resolution refinement, followed by supervised fine-tuning and RLHF. The reported setup spans multiple resolutions, tasks, and reward dimensions.
- Three-stage progressive training increases resolution from 512 × 512 to 1,024 × 1,024 and then 2,048 × 2,048 while preserving image aspect ratios.
- Stage I: Foundational Alignment: Stage I jointly trains text-to-image generation, language modeling, and multimodal understanding on image-text pairs and text-only corpora.
- Stage II: Generalist In-Context Learning: Stage II adds in-context generation and editing tasks, including image editing and subject-driven personalization, at 1,024 × 1,024 resolution.
- Stage III: High-Fidelity Refinement: Stage III uses an ultra-high-resolution subset to refine fine-grained details and perceptual quality above 2,048 × 2,048 resolution.
- Post-training combines supervised fine-tuning and RLHF to progressively refine reasoning capability and generative quality.
- Stage I: SFT: SFT uses several hundred thousand high-quality samples and reasoning trajectories to improve aesthetics, photorealism, and prompt reasoning.
- Stage II: RLHF: RLHF uses GRPO with rewards for OCR accuracy, aesthetics, instruction following, and reasoning quality to target photorealism and semantic consistency.
- HiDream-O1-Image debuts at #8 in the Artificial Analysis Text to Image Arena as an open-weights model.
5. Adversarial Diffusion Distillation for Fast Inference
HiDream-O1-Image-Dev accelerates inference by distilling the full model into a shorter sampling trajectory, while adversarial and auxiliary diffusion objectives preserve fidelity and stabilize training.
- Accelerated inference: 28-step HiDream-O1-Image-Dev reduces the full model’s roughly 50-step inference schedule through distillation.The student approximates the teacher’s generation behavior under fewer sampling steps.
- Distillation objective: DMD aligns the student’s trajectory distribution with the full HiDream-O1-Image model.This transfers the teacher’s main generative dynamics to the accelerated variant.
- Training stability: The standard diffusion loss provides auxiliary supervision that improves training stability and mitigates optimization oscillation.
- Perceptual fidelity: Adversarial learning uses a discriminator and frozen-teacher multilevel features to preserve perceptual fidelity and image sharpness.The discriminator compares real images with images reconstructed from the student’s pixel-space predictions.
6. Performance Comparisons for Text-to-Image Generation
HiDream-O1-Image performs strongly across general text-to-image synthesis, fine-grained text rendering, cinematic control, and multi-panel image generation. The 8B model compares favorably with larger systems, while the 200B+ model reaches state-of-the-art results in several evaluated settings.
- General text-to-image synthesis: 8B HiDream-O1-Image significantly surpasses similarly sized open-source counterparts, while the 200B+ model achieves state-of-the-art fidelity against leading closed-source models.The comparisons use GenEval, DPG, and HPSv3 benchmarks.
- High-fidelity text rendering: HiDream-O1-Image attains the highest scores across most CVTG-2K metrics for text rendering.
- High-fidelity text rendering: The 8B model performs comparably to 27B Qwen-Image on LongText-Bench, while the 200B+ model establishes a new state-of-the-art for ultra-long text rendering.
- High-fidelity text rendering: The end-to-end pixel-space framework bypasses text-to-vision translation bottlenecks and lossy VAE compression, supporting precise text-image alignment.The paper links this design to reduced structural distortions in visual text rendering.
- Diverse generation scenarios: HiDream-O1-Image handles cinematic shots, artistic styles, complex long text, and multi-panel images for storyboard production.
- Diverse generation scenarios: The model supports fine-grained manipulation across 15 cinematic shots and camera perspectives.These include shot scales, camera angles, and subject orientations.
- Diverse generation scenarios: Multi-panel generation can create coherent storyboards within a single inference pass.The paper positions this capability for cinematic pre-production and video-generation workflows.
7. Performance Comparisons for Image Editing
HiDream-O1-Image performs strongly on instruction-based image editing while preserving generation fidelity. The 8B model is comparable to substantially larger competitors, and the 200B+ model achieves state-of-the-art results on complex manipulations.
- Quantitative comparison: 8B HiDream-O1-Image yields comparable editing results to 16.8B FLUX.1 Kontext and 27B Qwen-Image-Edit.Evaluation covers GEdit and ImgEdit.
- Quantitative comparison: The 200B+ model establishes a new state-of-the-art for highly complex, fine-grained image manipulations.
- Editing quality: The model combines strong instruction following with high generation fidelity across GEdit and ImgEdit.
- Editing quality: HiDream-O1-Image captures nuanced editing intents, performs precise manipulations, and preserves unedited regions across both model scales.
8. Performance Comparisons for Subject-driven Personalization
HiDream-O1-Image is evaluated on multi-reference subject personalization using UniSubject, where it maintains strong performance across different numbers of reference subjects. The 8B model improves over listed baselines, and the 200B+ model extends multi-subject compositionality and identity preservation.
- Evaluation setup: UniSubject contains 300 test cases with 1.8K subjects, pairing one human subject with 1 to 10 reference objects and a compositional prompt.
- Subject personalization: For 4–8 subjects, the 8B model raises Q-O from 7.19 to 7.50, and for 9–11 subjects, from 6.73 to 7.48.
- Subject personalization: The 200B+ model further improves multi-subject compositionality and identity preservation in extreme scenarios.
- Mechanism: A shared token space bridges language and vision representations, forming cohesive fused subject representations and reducing interference as reference counts increase.
9. Conclusions
HiDream-O1-Image is presented as a natively unified pixel-level generative model that unifies raw pixels, text, and task conditions in one token space. Experiments across multiple benchmarks support its effectiveness across scales, including competitive 8B and state-of-the-art 200B+ variants.
- Conclusion: HiDream-O1-Image unifies raw pixels, text, and task conditions within a single shared token space.This replaces separate VAEs and segregated text encoders with a unified in-context generation process.
- Conclusion: The framework supports diverse synthesis tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization.The conclusion cites evaluation on GenEval, CVTG-2K, GEdit, and UniSubject.
- Conclusion: The 8B model achieves competitive or superior performance against significantly heavier latent-space DiTs across various generation tasks.The conclusion characterizes this as an efficiency advantage across multiple tasks rather than reporting one isolated benchmark value.
- Conclusion: The 200B+ model establishes new state-of-the-art records in visual synthesis.The reported scaling result is presented as evidence that the unified architecture remains effective at substantially larger scale.
A. Contributions and Acknowledgments
The acknowledgments section states that contributors are listed alphabetically by last name.
- A. Contributions and Acknowledgments: Contributors are listed alphabetically by last name.
- A. Contributions and Acknowledgments: The ordering criterion for contributors is their last name.
- A. Contributions and Acknowledgments: The contributor list uses alphabetical ordering.