PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen-Image-VAE-2.0 Technical Report

Zekai Zhang, Deqing Li, Kuan Cao, Yujia Wu, Chenfei Wu, Yu Wu, Liang Peng, Hao Meng, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Shengming Yin, Tianhe Wu, Xiao Xu, Xiaoyue Chen, Yan Shu, Yanran Zhang, Yilei Chen, Yixian Xu, Yuxiang Chen, Zhendong Wang, Zihao Liu, Zikai Zhou, Yiliang Gu, Yi Wang, Xiaoxiao Xu, Lin Qu

arXiv:2605.13565v1cs.CV

TL;DR

High-compression VAEs must balance compression, reconstruction fidelity, and diffusability, especially for text-rich images. Qwen-Image-VAE-2.0 addresses this with architectural, data, and semantic-alignment advances, achieving state-of-the-art reconstruction and strong diffusion compatibility across high-compression settings.

  • Problem

    High compression can degrade fine-grained reconstruction, while larger latent spaces may hinder downstream diffusion convergence and generative performance.

  • Method

    Qwen-Image-VAE-2.0 combines Global Skip Connections, expanded latent channels, large-scale and synthetic document data, and semantic alignment for high-compression VAEs.

  • Results

    0.9617 NED: Qwen-Image-VAE-2.0-f16c128 surpasses all evaluated f8 VAEs, including FLUX.1-dev at 0.9546, while achieving state-of-the-art reconstruction across benchmarks.

  • Takeaways & Limitations

    The models demonstrate a supported path toward jointly achieving high compression, strong reconstruction fidelity, and downstream diffusability.

  • Takeaways & Limitations

    OmniDoc-TokenBench uses empirically chosen reference sizes for Chinese and English that keep reconstruction challenging while preserving high OCR recognition accuracy.

Abstract

from arXiv · show

We present Qwen-Image-VAE-2.0, a suite of high-compression Variational Autoencoders (VAEs) that achieve significant advances in both reconstruction fidelity and diffusability. To address the reconstruction bottlenecks of high compression, we adopt an improved architecture featuring Global Skip Connections (GSC) and expanded latent channels. Moreover, we scale training to billions of images and incorporate a synthetic rendering engine to improve performance in text-rich scenarios. To tackle the convergence challenges of high-dimensional latent space, we implement an enhanced semantic alignment strategy to make the latent space highly amenable to diffusion modeling. To optimize computational efficiency, we leverage an asymmetric and attention-free encoder-decoder backbone to minimize encoding overhead. We present a comprehensive evaluation of Qwen-Image-VAE-2.0 on public reconstruction benchmarks. To evaluate performance in text-rich scenarios, we propose OmniDoc-TokenBench, a new benchmark comprising a diverse collection of real-world documents coupled with specialized OCR-based evaluation metrics. Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction performance, demonstrating exceptional capabilities in both general domains and text-rich scenarios at high compression ratio. Furthermore, downstream DiT experiments reveal our models possess superior diffusability, significantly accelerating convergence compared to existing high-compression baselines. These establish Qwen-Image-VAE-2.0 as a leading model with high compression, superior reconstruction, and exceptional diffusability.

1 Introduction

Qwen-Image-VAE-2.0 introduces f16 and f32 high-compression VAEs to address the trade-off among compression, reconstruction fidelity, and diffusability. It combines architectural, data, and training advances with benchmarks and DiT experiments targeting general and text-rich image synthesis.

  • Introduction: Higher compression ratios degrade reconstruction fidelity, especially for text-rich images, while increasing latent channels can create over-complex, unstructured latent spaces.
  • Introduction: Qwen-Image-VAE-2.0 introduces f16 and f32 high-compression image VAEs designed to address these challenges through improved architecture, data engineering, and training strategy.
  • Introduction: Global Skip Connections preserve fine-grained detail, higher latent dimensionality alleviates information bottlenecks, and training scales to billions of images with specialized document data.
  • Introduction: Semantic alignment with DINOv2 intermediate features accelerates DiT convergence, while staged alignment balances strict semantic alignment with broader optimization.
  • Introduction: The system uses a lightweight asymmetric encoder and attention-free backbone for efficient diffusion training, and evaluation covers reconstruction fidelity and latent-space diffusability.
  • Introduction: Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction performance, maintains text legibility, and demonstrates excellent diffusability for large-channel VAEs.

2 Model

Qwen-Image-VAE-2.0 uses higher spatial compression ratios to reduce downstream DiT computation while addressing the resulting loss of latent information. Its design combines Global Skip Connections, an attention-free asymmetric backbone, and lightweight encoding with high-fidelity decoding.

  • Compression Strategy: Higher compression ratios f16 and f32 reduce DiT training costs relative to the conventional f8 paradigm, but constrain latent information and fine-grained structural detail.DiT sequence length is L = HW/f^2, with computation scaling as O(H^2W^2/f^4).
  • Global Skip Connection: Global Skip Connection (GSC) bypasses initial downsampling by folding pixel-level spatial information into channels, preserving high-frequency details in high-compression VAEs.The mechanism uses a space-to-channel operation followed by reshaping to feed information into the deeper latent space.
  • Global Skip Connection: An ablation on an f16c64 model trained from scratch found that GSC significantly accelerates convergence, leading to its integration across the Qwen-Image-VAE-2.0 series.The comparison included No Skip Connection (NSC), Local Skip Connection (LSC), and Global Skip Connection (GSC).
  • Attention-Free Backbone: The attention-free backbone replaces quadratic self-attention scaling O(N^2) with convolutional complexity O(N · k^2), reducing high-resolution throughput and activation-memory bottlenecks.Self-attention activation memory also scales as O(N^2), whereas convolution uses kernel size k.
  • Encoder-Decoder Asymmetry: An asymmetric encoder-decoder balances speed and quality through a lightweight encoder for faster latent extraction and a heavyweight decoder for high-fidelity reconstruction.This design reduces training latency for downstream DiT while preserving intricate image detail.

3 Data

Qwen-Image-VAE-2.0 scales training data to billions of images while filtering low-quality samples and emphasizing text-rich real-world imagery. A synthetic document-rendering pipeline further targets character-level reconstruction across English and Chinese, while addressing the domain gap between clean and complex-background text images.

  • Scaling Data to Billion Scale: The VAE training corpus scales to billions of images spanning diverse visual categories, resolutions, and aspect ratios.This scaling is intended to ensure robust generalization across diverse domains.
  • Scaling Data to Billion Scale: Clarity and blur filters prune samples with edge blur and compression artifacts, retaining high-fidelity supervision for high-frequency detail.The filters mitigate noise that impedes learning fine visual details.
  • Text-Rich Image Collection: An OCR filter prioritizes high-character-density samples, complemented by documents such as academic papers, slides, posters, and complex web pages.The specialized corpus targets reconstruction bottlenecks in text-rich scenarios.
  • Text-Rich Image Collection: A synthetic pipeline renders text documents in English and Chinese, accounting for their distinct stroke densities and complexities.Models trained on background-free synthetic data generalize poorly to real-world images where text overlays complex textures.

4 Training

Qwen-Image-VAE-2.0 uses a simplified training objective centered on reconstruction and semantic alignment, omitting KL and GAN losses. Semantic guidance and multi-stage curricula improve latent diffusability while progressively balancing resolution, text fidelity, and reconstruction quality.

  • Training Objective: The training objective combines pixel-level L1 reconstruction, LPIPS perceptual loss, and semantic alignment loss to preserve fidelity while making latents more generation-friendly.Semantic alignment matches latents to semantic counterparts extracted from pretrained encoders.
  • Training Objective: Removing KL regularization and adversarial training improves performance and stability by avoiding latent-capacity restrictions and unnecessary discriminator optimization.The paper reports that KL competes with semantic alignment, while GAN loss becomes unnecessary with sufficiently large data and iteration budgets.
  • Semantic Alignment: DINOv2-L is selected as the default semantic guidance because ablations find it more generation-friendly than DINOv3, MAE, and PE-Spatial.Middle encoder layers provide smoother spatial maps, whereas naively combining layers introduces noise.
  • Semantic Alignment: Semantic alignment combines marginal cosine similarity for directional agreement with marginal distance-matrix similarity for preserving relative spatial layout.The VAE latent is projected into the semantic feature dimensionality before alignment.
  • Multi-Stage Training: Multi-stage training progressively scales resolution to 2K, adds real-world and synthetic text-rich data, and relaxes strict alignment margins to balance semantics with pixel reconstruction.The curriculum also uses diverse aspect ratios and staged text data to improve geometric robustness and character precision.

5 OmniDoc-TokenBench

OmniDoc-TokenBench evaluates text-rich image reconstruction with a curated ∼3K-document benchmark spanning diverse document types and languages. It uses OCR-based Normalized Edit Distance to measure page-level readability without word-level annotations, alongside a quality-controlled construction pipeline.

  • Motivation: Standard photo benchmarks and pixel metrics can miss text illegibility caused by small stroke distortions.Natural-image datasets contain negligible textual content, while PSNR and SSIM may change little even when characters become unrecognizable.
  • Benchmark: OmniDoc-TokenBench contains ∼3K text-rich document images across nine categories, covering English and Chinese text.The categories include books, slides, textbooks, exams, academic papers, magazines, financial reports, newspapers, and notes.
  • Metric: Full-page OCR and Normalized Edit Distance directly measure reconstructed document readability without word-level bounding box annotations.OCR is applied to both original and reconstructed images, and the annotation-free design supports scaling to new document types.
  • Construction: The benchmark pipeline filters samples to 200–600 Chinese or 300–600 English recognized characters and balances language coverage through manual quality inspection.Samples are pruned when blurred, visually redundant, or dominated by blank regions.
  • Metric: NED compares OCR output from each original image with its VAE reconstruction using character-level Levenshtein distance, then averages across images.Using original-image OCR as the reference avoids penalizing reconstructions for systematic OCR errors present even on clean inputs.

6 Experiments · 6.1 Quantitative Results

Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction across general-domain, high-resolution, and text-rich benchmarks at high compression. Its latent spaces also show superior diffusability, enabling rapid DiT convergence and outperforming existing high-compression baselines in generation quality.

  • 6.1.1 Performance of Reconstruction: Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction fidelity within the f16 and f32 compression tiers on ImageNet and FFHQ.Evaluation uses PSNR and SSIM on 256p ImageNet and 1K FFHQ images.
  • 6.1.1 Performance of Reconstruction: At 4× compression, f32c192 performs comparably to established f8 VAEs such as Wan2.1.
  • 6.1.2 Performance of Text Rendering: Qwen-Image-VAE-2.0-f16c128 reaches SSIM 0.9706 and PSNR 30.45 dB, surpassing FLUX.1-dev’s f8 baseline of 0.9364 / 26.24 dB despite 2× higher spatial compression.
  • 6.1.2 Performance of Text Rendering: NED complements pixel metrics because character-level errors can reduce NED by 16.7% while causing less than 0.5 dB PSNR loss.Observed discrepancies between NED and SSIM or FID further support measuring text fidelity separately.
  • 6.1.2 Performance of Text Rendering: On OmniDoc-TokenBench, f16c64 achieves NED 0.9244 and f16c128 reaches 0.9617, exceeding evaluated f8 VAEs including FLUX.1-dev at 0.9546.The benchmark evaluates text-rich images using OCR-based NED alongside traditional pixel metrics.
  • 6.1.2 Performance of Text Rendering: At f32 compression, f32c192 achieves NED 0.8555 while competing models exhibit NED 0.07–0.57, an advantage attributed to diverse text-rich data and synthetic rendering pipelines.
  • 6.1.3 Performance of Diffusability: Downstream SiT experiments show that Qwen-Image-VAE-2.0 consistently outperforms existing high-compression baselines in generation quality without guidance.SiT-XL/2 is used for f8, SiT-XL/1 for f16 and f32, and results are reported after 80 epochs using unguided evaluation.
  • 6.1.3 Performance of Diffusability: Despite large latent dimensions, Qwen-Image-VAE-2.0 enables rapid DiT convergence, driven primarily by enhanced semantic alignment and staged alignment.This diffusability addresses the traditional trade-off among compression, reconstruction, and diffusion modeling.

6.2 Qualitative Results

Qualitative results show that Qwen-Image-VAE-2.0 preserves readable text and fine visual detail under extreme f16 and f32 compression. Its latent space also supports high-fidelity ImageNet generation and complex foundation-model text-to-image conditioning.

  • f16 qualitative reconstruction: At f16, Qwen-Image-VAE-2.0-f16c128 preserves crisp character boundaries, accurate spacing, and fine stroke detail despite severe baseline degradation.Weaker baselines exhibit character blurring, stroke merging, and inter-character ghosting, with Cosmos-0.1-CI16x16 showing near-complete text collapse.
  • f32 qualitative reconstruction: At f32, Qwen-Image-VAE-2.0-f32c192 retains distinguishable character forms and recognizable word boundaries while competing models produce fragmented noise.The visual result is consistent with substantially higher NED scores and supports the architecture’s effectiveness under extreme compression.
  • ImageNet generation: Across f16 and f32 compression ratios, SiT-XL ImageNet generations maintain high visual fidelity without structural degradation at 256 × 256 and 512 × 512 resolutions.Samples use a further-trained SiT-XL with classifier-free guidance, following the quantitative training protocol.
  • Foundation-model integration: Integration into Qwen-Image-2.0 demonstrates that the compressed latent space supports open-vocabulary conditioning, compositional constraints, precise text rendering, and refined photorealism.The result validates diffusability at foundation-model scale within a large-scale text-to-image pipeline.

7 Conclusion

Qwen-Image-VAE-2.0 introduces high-compression image VAEs designed to address bottlenecks in native high-resolution image synthesis. The approach targets the trade-off among compression ratio, reconstruction fidelity, and downstream diffusability through expanded latent channels and semantic alignment.

  • 7 Conclusion: Expanded latent channel dimensions compensate for spatial information loss from high compression, while semantic alignment improves latent-space diffusability.Together, these techniques provide a path toward balancing compression ratio, reconstruction fidelity, and downstream diffusability.

8 Authors

The paper lists its contributors and identifies equal-contribution and corresponding-author designations. It also notes that the VAE integrated into Qwen-Image-2.0 is an intermediate variant derived from this work’s methodology.

  • The paper credits 29 contributors, including Zekai Zhang, Deqing Li, Kuan Cao, and other listed authors.
  • Zekai Zhang, Deqing Li, and Kuan Cao are marked as equal contributors.
  • Chenfei Wu is identified as the corresponding author.
  • The VAE integrated into Qwen-Image-2.0 is described as an intermediate variant derived from the methodological framework established in this work.
Loading 2605.13565v1…