Source-linked AI summary
SenseNova-U1.5: Towards Native Unified Visual Intelligence
Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Bo Yang, Chen Feng, Chengguang Lv, Guangjia Liu, Guanlin Wang, Hanyu Zhang, Haojia Yu, Hongcan Xiao, Hongli Wang, Huan Wu, Huaping Zhong, Jian Fang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jing Zuo, Jingcheng Ni, Junxiang Xu, Linjun Dai, Mutian Xu, Peishen Yan, Penghao Wu, Ruijie Mao, Ruisi Wang, Shihao Bai, Shuang Yang, Shuya Yang, Shuyan Zheng, Silei Wu, Siying Li, Tao Chu, Tianbo Zhong, Tongxi Zhou, Weichao Luo, Weichen Fan, Wenhao Jia, Wenjie Gao, Xiangli Kong, Yan Li, Yang Yong, Zimo Wen, Zixuan Qian, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin
TL;DR
Most multimodal systems separate visual understanding from generation across encoder and VAE representations, limiting unified visual intelligence. SenseNova-U1.5 addresses this with an encoder-free, VAE-free native architecture and specialized-to-unified training, achieving strong understanding, generation, editing, and reasoning performance.
Problem
Most visual systems use separate vision encoders for understanding and VAEs for generation, placing semantic and pixel-level processing in distinct representational spaces.
Method
SenseNova-U1.5 combines spatially joint visual reconstruction, resolution-conditioned generation, progressive denoising training, and specialized reinforcement-learning experts consolidated through multi-expert on-policy distillation.
Results
SenseNova-U1.5 delivers strong performance across visual understanding, high-fidelity generation, bilingual text rendering, editing, multi-reference composition, and interleaved generation, with leading results on several evaluations.
Takeaways & Limitations
The results support native unified modelling as a path toward systems where perception, reasoning, generation, and editing operate within a shared visual substrate.
Abstract
from arXiv · showhide
We launch SenseNova-U1.5, an 8B-MoT native unified multimodal model that understands, reasons about, and generates visual content within an encoder-free and VAE-free architecture. We strengthen its visual interface through spatially coherent patch reconstruction and scale its training with carefully curated generation and editing data, improved task formulation, structural prompt enhancement, and native resolutions of up to 4K. For post-training, we optimize specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing, and consolidate their capabilities through multi-expert on-policy distillation. Across extensive evaluations, SenseNova-U1.5 largely advances image fidelity, text rendering, complex composition, multi-reference editing, and interleaved generation, while improving instruction following and preserving subject identity, geometry, and unmodified regions. Despite limited exposure to structured formats in its generation data, SenseNova-U1.5 generalizes effectively to long, complex, and structured visual instructions, further proving that multimodal understanding can transfer to visual planning and creation. Together, these findings position native unified modelling as a promising path towards systems that perceive, reason and create within a fully end-to-end framework. We will open-source training code, including supervised fine-tuning, reinforcement learning, and on-policy distillation.
1 Introduction
SenseNova-U1.5 advances native unified multimodal modelling with spatially joint visual reconstruction and specialized-to-unified post-training. The resulting encoder-free, VAE-free system supports understanding, reasoning, generation, editing, and structured visual instructions within one shared representation.
- The model addresses a divide in conventional systems, where pretrained vision encoders support perception while VAEs support generation in separate representational spaces.Native unified modelling instead learns directly from pixels and words, building on SenseNova-U1’s encoder-free and VAE-free architecture.
- Spatially joint reconstruction replaces independent patch prediction, enabling neighbouring regions to resolve colour, texture, and geometry together while supporting native generation up to 4K.Visual tokens are projected onto a two-dimensional feature field and reconstructed with spatial convolutions and Pixel Shuffle upsampling.
- Specialized reinforcement-learning experts for aesthetics, bilingual text, infographics, and editing are consolidated through multi-expert on-policy distillation into one unified policy.The strategy uses tailored data, rewards, sampling, and regularization before distilling along the student’s own generation trajectories.
- SenseNova-U1.5 extends encoder-free, VAE-free modelling to visual understanding, reasoning, generation, and editing through a shared visual representation.Its compact visual interface is intended to avoid separate perceptual and generative pathways and repeated conversion between encoder features and VAE latents.
- Despite limited reliance on fixed structured formats, SenseNova-U1.5 generalizes to long, compositional, and highly structured visual instructions.This extends the model beyond basic image synthesis toward visual planning and creation across complex instructions.
2 Related Works
Related work develops native multimodal models that reduce reliance on external encoders and latent bottlenecks, alongside reinforcement-learning and on-policy distillation methods for visual generation. These lines of research motivate unified systems that combine understanding, generation, and transferable capabilities.
- Native multimodal research combines encoder-free visual understanding with direct pixel modelling, while unified models pursue shared representations for understanding and generation.The literature includes discrete token-level autoregressive approaches and continuous end-to-end approaches without explicit tokenizers or latent bottlenecks.
- Visual-generation reinforcement learning builds on RLHF, PPO, GRPO, and DAPO, with online methods adapting reward-guided optimization to image-generation settings.These methods target efficiency, scalability, and online optimization without necessarily using explicit value models.
- Flow-matching generation requires stochastic exploration beyond deterministic ODE inference, motivating SDE-based rollouts and sampling methods that balance exploration with distributional fidelity.Related methods also address finite-step flow preservation and reward over-optimization.
- On-policy distillation addresses distribution mismatch by training students on their own trajectories with dense teacher supervision, and MOPD extends this to multiple domain experts.This provides a basis for consolidating independently optimized capabilities into a single model.
3 Methodology
SenseNova-U1.5 unifies multimodal understanding and pixel-space generation in an encoder-free, VAE-free architecture with spatially coherent visual reconstruction. Its training combines resolution-aware native generation, unified objectives, staged capability learning, and multi-expert post-training.
- Model architecture: Resolution-dependent noise conditioning extends native synthesis from 2048 × 2048 to a 4096 × 4096 reference resolution.A dedicated sinusoidal MLP combines normalized resolution noise with the diffusion timestep representation.
- Model architecture: Spatially coupled decoding replaces independent patch reconstruction, allowing neighboring regions to exchange information before pixels are finalized.The decoder restores two-dimensional token topology and progressively reconstructs images with spatial convolutions and Pixel Shuffle upsampling.
- Model architecture: The native Mixture-of-Transformers backbone interleaves clean image-text context and noise-conditioned visual states while using structured attention for causal text and bidirectional image interaction.Understanding and generation retain stream-specific parameters while communicating through shared attention.
- Training objectives: A unified objective combines autoregressive language modeling, pixel-space flow matching, and perceptual supervision to couple semantic understanding with visual formation.The three signals respectively support multimodal reasoning, direct RGB-space generation, and coherent global structure with faithful local textures.
- Training procedure: Training progresses from generation pre-training through unified mid-training and supervised fine-tuning to capability-specific reinforcement learning and multi-expert on-policy distillation.The post-training experts target aesthetics, text rendering, infographic generation, and image editing before their capabilities are consolidated.
4 Data Construction
SenseNova-U1.5 expands training data across diverse generation, editing, and interleaved multimodal trajectories, using curation and unified formatting to support complex visual capabilities.
- Data quality and alignment are improved through filtering, rebalancing, multi-granularity captions, bilingual coverage, and targeted synthesis for underrepresented capabilities.
- Approximately 59M additional text–image pairs from 78 sources broaden coverage across scenes, people, objects, text-rich imagery, infographics, and specialized domains.
- Approximately 38M editing examples span general, infographic, reference-conditioned, and spatially controlled editing scenarios.
- Editing data is distributed across general editing, spatially controlled infographic editing, and reference-conditioned subsets at approximately 43%, 42%, and 15%.
- Multimodal trajectories combine interleaved text and visual states to supervise local grounding, long-range coherence, temporal continuity, and reasoning-conditioned generation.
5 Experiments
SenseNova-U1.5 retains strong multimodal and language understanding while achieving leading results across generation, text rendering, editing, and interleaved-generation evaluations. Its gains extend to compositional reasoning, bilingual typography, preservation, and reasoning-intensive transformations.
- 5.1 General Understanding: SenseNova-U1.5 remains strong across multimodal benchmarks and comparable to or better than modular and encoder-free baselines on many tasks.It covers STEM reasoning, VQA, OCR, hallucination, and visual reasoning while preserving or improving performance over SenseNova-U1.
- 5.2 Image Generation: 60.22 and 60.13 on the English and Chinese Open-Image-Bench subsets establish the best overall performance among evaluated open-source models.These scores use prompt enhancement; without it, the model still improves over SenseNova-U1 and remains competitive with larger baselines.
- 5.2 Image Generation: SenseNova-U1.5 achieves leading or best open-source results across complex composition, infographic, business, and dense bilingual text-generation benchmarks.Reported gains include GenEval 0.92, stronger structured visual planning on IGenBench, improved BizGenEval performance, and leading results with reasoning or prompt enhancement.
- 5.2 Image Generation: 0.948 average on CVTG-2K is best among all evaluated models, with word accuracy above 0.95 in four- and five-region settings.The model improves normalized edit distance and high-region word accuracy over SenseNova-U1 while maintaining comparable CLIPScore.
- 5.3 Image Editing: The model delivers leading editing performance while preserving source content, identity, style, background, and semantic context across single- and multi-reference tasks.OmniRef-Bench shows stronger style and background preservation plus subject and pose consistency; ImgEdit, GEdit-Bench, and WeEdit report broad gains.
- 5.4 Interleaved Generation: Reasoning improves difficult visual transformations, while interleaved generation reaches state-of-the-art cognitive-faculty results and strong out-of-domain generalization.RISEBench reports especially strong causal, logical, and temporal edits with CoT, and VBVR-Pro-Bench exceeds leading proprietary models.
6 Conclusion
The conclusion presents SenseNova-U1.5 as evidence that native multimodal systems can learn understanding and generation on a shared computational substrate. It further argues that capability interaction—not only architectural simplification—enables reasoning, creation, and transformation to reinforce one another.
- 6 Conclusion: A compact native representation simultaneously supports strong understanding, high-fidelity generation, editing, interleaved generation, and reasoning-intensive visual tasks.The system moves beyond basic image synthesis while using a common substrate rather than separate perceptual and generative pipelines.
- 6 Conclusion: The paper identifies capability interaction as the central opportunity of multimodal unification, with reasoning improving creation and generation participating in visual problem solving.This frames seeing, reasoning, and generating as different expressions of shared visual intelligence.
7 Contributors
The contributor list organizes project roles from sponsor and advisors through senior leads, project leads, core contributors, and contributors. Individuals within each category are listed alphabetically by first name.
- Project Sponsor and Advisor: Dahua Lin is listed as Project Sponsor and Advisor.
- Senior Project Lead: Lei Yang, Lewei Lu, Quan Wang, Ruihao Gong, Wenxiu Sun, and Ziwei Liu are listed as Senior Project Leads.
- Project Lead: Haiwen Diao and Jiahao Wang are listed as Project Leads.
- Core Contributor: The Core Contributor category includes Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, and Zhongang Cai.