Source-linked AI summary
X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Again
Zigang Geng, Yibing Wang, Yeyao Ma, Chen Li, Yongming Rao, Shuyang Gu, Zhao Zhong, Qinglin Lu, Han Hu, Xiaosong Zhang, Linus, Di Wang, Jie Jiang
TL;DR
Discrete autoregressive image generation has faced fidelity, artifact, and complex-instruction challenges, motivating movement toward less unified hybrid approaches. X-Omni uses reinforcement learning with a semantic tokenizer, unified autoregressive model, and diffusion decoder, achieving strong generation and understanding results while rendering long text and avoiding classifier-free guidance.
Problem
Discrete-token autoregressive image generation is limited by low fidelity, artifacts, and difficulty following complex instructions, while hybrid approaches complicate unified image-language modeling.
Method
X-Omni combines semantic image tokenization, a unified autoregressive model for text and images, an offline diffusion decoder, and reinforcement learning.
Results
X-Omni improves visual quality, instruction following, and long-text rendering, with strong text-to-image and image-understanding performance across reported benchmarks.
Takeaways & Limitations
The results support discrete autoregressive modeling as a unified route for image generation and understanding, with reinforcement learning improving generation quality and removing dependence on classifier-free guidance.
Abstract
from arXiv · showhide
Numerous efforts have been made to extend the ``next token prediction'' paradigm to visual contents, aiming to create a unified approach for both image generation and understanding. Nevertheless, attempts to generate images through autoregressive modeling with discrete tokens have been plagued by issues such as low visual fidelity, distorted outputs, and failure to adhere to complex instructions when rendering intricate details. These shortcomings are likely attributed to cumulative errors during autoregressive inference or information loss incurred during the discretization process. Probably due to this challenge, recent research has increasingly shifted toward jointly training image generation with diffusion objectives and language generation with autoregressive objectives, moving away from unified modeling approaches. In this work, we demonstrate that reinforcement learning can effectively mitigate artifacts and largely enhance the generation quality of a discrete autoregressive modeling method, thereby enabling seamless integration of image and language generation. Our framework comprises a semantic image tokenizer, a unified autoregressive model for both language and images, and an offline diffusion decoder for image generation, termed X-Omni. X-Omni achieves state-of-the-art performance in image generation tasks using a 7B language model, producing images with high aesthetic quality while exhibiting strong capabilities in following instructions and rendering long texts.
1 Introduction
X-Omni revisits discrete autoregressive image generation by using reinforcement learning to reduce cumulative errors while unifying text and image modeling. The resulting model improves visual quality, instruction following, and long-text rendering, with semantic tokenization also supporting image understanding.
- Motivation: Discrete autoregressive image generation has historically suffered low fidelity, partly because sequential token prediction accumulates errors.These limitations helped motivate a shift toward diffusion-based image generation, although heterogeneous architectures complicate unified vision-language modeling.
- Approach: X-Omni uses reinforcement learning with designed rewards to automatically refine generated tokens and align them more closely with the diffusion decoder.The approach targets cumulative error reduction while preserving a discrete autoregressive framework for images and text.
- Results: After reinforcement learning, image quality, instruction adherence, and long-text rendering improve over the lower-quality supervised-fine-tuned model outputs.The reported examples include accurate English and Chinese long-text rendering, alongside improved visual appearance and complex-instruction following.
- Unified capabilities: Semantic encoders such as SigLIP 2 produce discrete tokens that also lend themselves to image understanding tasks.This extends the role of the image tokenizer beyond generation.
- Unified capabilities: X-Omni combines isomorphic autoregressive modeling for images and text in one network, avoiding re-extraction of semantic embeddings from generated images during joint multi-turn use.The authors describe this as a more streamlined and efficient architecture than approaches using heterogeneous image encoders.
2 Related Work
Related work explores continuous and discrete visual tokens, hybrid autoregressive-diffusion designs, and reinforcement learning for generative models. X-Omni is positioned against limitations involving discretization, cross-modal mismatch, and the still-developing use of reinforcement learning beyond diffusion models.
- Continuous tokens versus discrete tokens: Continuous-token methods predict visual representations autoregressively or in parallel, but how reinforcement learning integrates with them remains unclear.This uncertainty may constrain their achievable performance limits.
- Continuous tokens versus discrete tokens: Discrete visual tokens support more complex distributions but introduce discretization losses that can constrain generated image detail.The related work discusses lossy quantization and semantic supervision as responses to this challenge.
- Hybrid AR and diffusion models: Hybrid autoregressive-diffusion systems commonly use either serial conditioning or parallel language-and-image generation architectures.These designs differ in how autoregressive outputs provide conditions to diffusion generation.
- Reinforcement learning for generative models: Most reinforcement-learning research for generative models has focused on diffusion models, using reward gradients, reward-weighted objectives, or language-modeling algorithms.Examples include DPO, PPO, and GRPO adaptations for diffusion models.
3 Method
X-Omni unifies image and text tokens in an autoregressive model, uses a semantic tokenizer and diffusion decoder, and trains the autoregressive component with reward-based optimization. Its rewards target aesthetic quality, prompt alignment, and text-rendering accuracy while bridging the autoregressive and diffusion token distributions.
- Overall Architecture: X-Omni integrates image and text tokens in one autoregressive architecture with a SigLIP-VQ tokenizer and diffusion decoder.The design supports unified tokenization and detokenization for multimodal inputs and outputs.
- Image Tokenization: The semantic image tokenizer converts images into discrete tokens using a SigLIP2 visual encoder, vector quantizer, and language-model alignment.The vector quantizer uses a 16,384-entry codebook with 2,048-dimensional embeddings, while the visual encoder and quantizer remain frozen during later training.
- Autoregressive Modeling: Visual and language tokens are concatenated into a unified multimodal sequence for next-token prediction, with supervision selected by task.Understanding tasks supervise language tokens, generation tasks supervise visual tokens, and resolution information is prefixed to visual tokens.
- Diffusion Decoder: The diffusion decoder reconstructs image pixels from discrete semantic tokens by mapping them into FLUX.1-dev feature channels and conditioning image reconstruction.The decoder is trained as an image-reconstruction component.
- Reinforcement Learning: Reinforcement learning bridges the distribution gap between semantic tokens used by the diffusion decoder and tokens generated by the autoregressive model.The process provides supervision throughout sampling to reduce error propagation and align the autoregressive output distribution with decoder expectations.
- GRPO Algorithm: GRPO evaluates groups of sampled trajectories after diffusion decoding, normalizes their rewards into advantages, and optimizes the policy without a separate critic network.The objective balances reward maximization with divergence from a stable reference policy.
- Reward Design: The reward system combines specialized signals for aesthetic quality, text-image alignment, and text-rendering accuracy.HPSv2 and Unified Reward assess human-aligned quality, Qwen2.5-VL-32B assesses semantic alignment, and GOT-OCR2.0 with PaddleOCR assesses rendered-text fidelity.
4 Experiments
X-Omni is evaluated across text rendering, text-to-image generation, image understanding, and classifier-free-guidance dependence after staged training with supervised fine-tuning and reinforcement learning. Results show strong instruction following, long-text rendering, image quality, and multimodal performance.
- Training Data: The training pipeline combines large-scale image-text pre-training, supervised fine-tuning, and on-policy GRPO reinforcement learning.The reinforcement-learning stage uses text prompts as inputs while the model generates images during training.
- Text Rendering: LongText-Bench contains 160 prompts across 8 scenarios to evaluate precise rendering of long Chinese and English text.It was introduced because OneIG-Bench prompts have limited text lengths.
- Text Rendering: X-Omni significantly outperforms recent unified models on English OneIG-Bench text rendering and leads Chinese LongText-Bench by a large margin.It is slightly behind GPT-4o on English LongText-Bench and comparable to Seedream 3.0 on Chinese OneIG-Bench.
- Text-to-Image Generation: X-Omni achieves state-of-the-art performance among recent unified models on DPG-Bench and comparable results on GenEval.These evaluations assess text-to-image generation and complex instruction following.
- Image Understanding: X-Omni matches Show-o2 across image-understanding benchmarks, surpasses earlier unified models, and significantly exceeds them and LLaVA-OneVision on OCRBench.The evaluation spans POPE, GQA, MMBench, SEEDBench-Img, DocVQA, and OCRBench.
- Findings: X-Omni maintains high image-generation quality without classifier-free guidance, whereas other autoregressive models degrade substantially when CFG is absent.The authors associate CFG independence with lower autoregressive inference cost and greater consistency between visual and language-token generation.
- Findings: Reinforcement learning surpasses supervised fine-tuning with best-of-N sampling by aligning the autoregressive and diffusion modules.The authors attribute this advantage to holistic optimization over local, spatially complex image features.
5 Conclusion
X-Omni uses reinforcement learning to train a unified autoregressive model for image generation and understanding, including long-text rendering and sampling without classifier-free guidance.
- X-Omni applies reinforcement learning to an omni autoregressive model for unified image generation and understanding.
- X-Omni is presented as the first unified model capable of rendering long text.
- The approach removes dependence on classifier-free guidance during autoregressive sampling.
A LongText-Bench Details
LongText-Bench evaluates long-text rendering in English and Chinese using curated prompts spanning eight text-rich scenarios, with Text Accuracy as its final metric.
- Benchmark Statistics: English prompts contain short texts of 10-30 words and long texts of 30-50 words, while Chinese long-text prompts typically exceed 60 characters.Chinese short prompts mostly contain 20 to 40 characters.
- Benchmark Statistics: The benchmark’s overall text length exceeds OneIG-Bench’s, emphasizing evaluation of long-text rendering capability.
- Prompt Construction: LongText-Bench contains 160 prompts across eight text-rich scenarios, balanced between short and long text contents through automatic generation and manual review.The scenarios include signboards, labeled objects, printed materials, web pages, slides, posters, captions, and dialogues.
- Evaluation Metric: LongText-Bench uses Qwen2.5-VL-7B for OCR and Text Accuracy because Edit Distance does not account for the relative order of long-text segments.Four images are generated for each prompt.