PaperlayerView exact revision on arXiv

Source-linked AI summary

Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models

Dong Chen, Fangyun Wei, Ziyu Wan, Dongdong Chen, Jiawei Zhang, Jinjing Zhao, Sirui Zhang, Yang Yue, Zhiyang Liang, Baining Guo, Chong Luo, Jianmin Bao, Ji Li, Lei Shi, Qinhong Yang, Xiuyu Wu, Xuelu Feng, Yan Lu, Yanchen Dong, Yitong Wang, Yunuo Chen

arXiv:2605.21573v1cs.CV

TL;DR

Foundational text-to-image models achieve strong synthesis and prompt understanding but require prohibitive training resources. Lens addresses this gap with a compact model and efficiency-focused training and post-training strategies, matching or surpassing larger state-of-the-art models while using about 19.3% of Z-Image’s training compute.

  • Problem

    Foundational text-to-image models deliver strong capabilities but typically require massive computational resources, creating substantial training costs.

  • Method

    Lens combines a compact 3.8B-parameter architecture with dense-caption, mixed-resolution training, selected VAE and language encoder designs, RL post-training, a reasoner, and distillation.

  • Results

    Lens matches or surpasses larger state-of-the-art models across multiple benchmarks while requiring approximately 19.3% of Z-Image’s training compute.

  • Takeaways & Limitations

    Lens demonstrates that a compact text-to-image model can achieve competitive generation quality at substantially lower training cost than larger state-of-the-art models.

  • Takeaways & Limitations

    English-only training limits multilingual prompt-following quality, and Lens still struggles with visual text rendering in some non-English languages.

Abstract

from arXiv · show

We introduce Lens, a 3.8B-parameter T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters across various benchmarks, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The training efficiency of Lens stems from two key strategies beyond its compact model size. First, we maximize data information density per training batch by (i) training on Lens-800M, a dataset of 800M densely captioned image-text pairs whose captions are generated by GPT-4.1 and contain approximately 109 words on average, providing richer semantic supervision than conventional short captions, and (ii) constructing each batch from images with multiple resolutions and diverse aspect ratios, thereby enlarging the effective visual coverage of each optimization step. Second, we improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. After pre-training, we apply RL with taxonomy-driven prompts (Lens-RL-8K) and structured reward rubrics to suppress artifacts and improve visual quality, a reasoner module with training-free system prompt search to better align user requests with the model, and distillation-based acceleration for 4-step inference. Through efficient training and systematic optimization, Lens generalizes to arbitrary aspect ratios from 1:2 to 2:1 and resolutions up to 1440^2, and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a 1024^2 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds.

1 Introduction

Lens is a 3.8B-parameter foundational T2I model designed to improve training efficiency through compact scale, higher information density per batch, and faster convergence. It achieves competitive or superior benchmark performance to larger models while using approximately 19.3% of Z-Image’s training compute, then applies RL, reasoning, and distillation for quality, alignment, and speed.

  • Model scale: Lens uses 3.8B parameters, versus 6B or more for several state-of-the-art open-source models, reducing per-step computational cost.Listed larger models include Z-Image (6B), LongCat-Image (6B), FLUX.2 (9B), Qwen-Image (20B), and Hunyuan-Image-3.0 (MoE, 80B).
  • Data information density: 109 words on average characterize Lens-800M’s GPT-4.1-generated dense captions, providing richer semantic supervision than conventional short captions.Dense captions encode objects, attributes, spatial relationships, actions, and background context.
  • Data information density: 512^2, 768^2, and 1024^2 resolutions combined with diverse aspect ratios increase visual coverage within each training batch.The approach supports learning visual content at different detail levels and reduces the need for costly high-resolution training.
  • Convergence speed: Semantic VAE selection and stronger language encoders accelerate convergence, while GPT-OSS enables multilingual inference generalization from English-only image-text training.Reported target languages include Chinese and French, reducing multilingual data and training-cost requirements.
  • Post-training and analysis: Post-training RL suppresses artifacts and improves composition, while a reasoner with training-free system-prompt search aligns requests and systematic ablations identify complementary efficiency strategies.RL data must cover the original training distribution sufficiently to avoid degradation on certain input types.

2 Method

Lens combines a diverse, heavily filtered 800M-image corpus with detailed caption supervision and mixed-resolution training to improve data efficiency and generalization. Its architecture uses a semantic VAE, GPT-OSS language encoder, and interchangeable Reasoner, followed by taxonomy-driven RL post-training.

  • Lens-800M: The corpus combines public real-world, public synthetic, private, and text-synthetic data to provide diverse visual and typographic coverage.Text-synthetic images use randomized backgrounds and augmentations in blur, color, font, scale, and rotation.
  • Lens-800M: 800M high-quality images remain after filtering corrupted files, low-resolution images, NSFW content, low-aesthetic samples, and watermarks.Images with area smaller than 384^2 are removed, and samples with Aesthetic Predictor v2.5 scores below 3 are discarded.
  • Captioning: Dense caption supervision improves each batch’s information density, data utilization, and training efficiency.Lens-800M captions are detailed, long-form English captions generated with GPT-4.1.
  • Architecture: FLUX.2’s semantic VAE achieves the best generation performance while accelerating convergence, so Lens adopts it for image-latent encoding.The semantic latent space supports text-image alignment and reduces optimization steps required for convergence.
  • Architecture: GPT-OSS is selected as Lens’s language encoder after performance-curve comparisons on GenEval using English-only Lens-130M training data.GPT-OSS is a 20B-parameter MoE language model with 3B activated parameters and 24 layers.
  • Post-training: Lens uses taxonomy-driven RL post-training with 8,406 prompts spanning broad T2I scenarios, because balanced coverage improves diverse-scenario generalization.The taxonomy includes Human, Object, Animal, Plant, Scene, Food, Event, Fictional World, Text, and UI and Graphic Design.

3 Comparison with State-of-the-art Models

The section compares Lens at 20-step inference and Lens-Turbo at 4-step inference with state-of-the-art models across four text-to-image benchmarks. These benchmarks assess overall generation, compositional alignment, long-text rendering, and complex multi-region visual text generation using task-specific metrics.

  • Main Results: OneIG (EN) evaluates general objects, portraits, anime/stylization, text rendering, knowledge/reasoning, subject alignment, text accuracy, reasoning, style, diversity, and overall performance.The benchmark contains 1,120 prompts.
  • Main Results: Table 2 compares Lens with 20-step inference and Lens-Turbo with 4-step inference against state-of-the-art models across four benchmarks.Reported metrics include overall scores for OneIG, GenEval, and LongText, plus average score, normalized edit distance (NED), and CLIP score for CVTG.
  • Main Results: GenEval measures object-centric compositional alignment across single-object generation, co-occurrence, counting, color, spatial relation, and attribute binding tasks.It uses 553 prompts and detector- and classifier-based verification.
  • Main Results: LongText (EN) tests faithful rendering of short and long text across signboards, labeled objects, printed materials, webpages, slides, posters, captions, and dialogues.The benchmark contains 160 prompts spanning eight text-rich scenarios, with text lengths of roughly 10–30 and 30–50 words.
  • Main Results: CVTG evaluates complex visual text generation with 2–5 text regions while varying content, position, length, color, font, and size.Its reported metrics are average score, normalized edit distance (NED), and CLIP score.

4 Visualization

Qualitative visualizations show Lens-Turbo performing across general image, portrait, multilingual text-rendering, and multilingual prompt-following scenarios. The examples use 1440^2-pixel images with varying aspect ratios and demonstrate high-quality generation, realistic portraits, multilingual text rendering, and multilingual instruction generalization.

  • Visualization: Lens-Turbo is visualized across general image generation, portrait generation, multilingual visual text generation, and multilingual prompt following.Figures 9–16 cover these scenarios, with multilingual prompt-following inputs written in different languages.
  • Visualization: 1440^2 pixels is the fixed image area used across examples, with varying aspect ratios.The visualizations span diverse generation scenarios while maintaining this image area.
  • Visualization: The examples demonstrate high-quality images, realistic portraits, multilingual text rendering, and generalization to multilingual user instructions.These qualitative results are presented for Lens-Turbo across the listed scenarios.

5 Conclusion

Lens is a 3.8B-parameter foundational text-to-image model designed for training-time efficiency. It combines denser data and mixed-resolution pre-training with architectural choices that accelerate convergence, then applies RL-based post-training to enhance generation quality.

  • 3.8B parameters: Lens is a foundational text-to-image model designed for training-time efficiency.
  • Dense captions and mixed-resolution/aspect-ratio pre-training on Lens-800M improve data information density.
  • Carefully selected VAE and language encoder designs accelerate convergence while supporting strong generation quality at substantially lower training cost.
  • RL-based post-training further enhances Lens after pre-training.

Contributor List (Alphabetical Order) · A Related Works · A.1 Foundational T2I Models

This block lists the Lens contributors, illustrates broad generation capabilities through image galleries, and situates the model within the evolution from latent diffusion to Transformer-based, rectified-flow, unified multimodal, and autoregressive systems.

  • Contributor List (Alphabetical Order): Core Contributors: Dongdong Chen, Jiawei Zhang, Jinjing Zhao, Sirui Zhang, Yang Yue, and Zhiyang Liang.The list also names Contributors including Baining Guo, Chong Luo, Jianmin Bao, Jiawei Bao, Ji Li, Lei Shi, Qinhong Yang, Xuelu Feng, Xiuyu Wu, Yan Lu, Yanchen Dong, Yitong Wang, Yunuo Chen.
  • Contributor List (Alphabetical Order): Lens generates diverse, high-resolution images spanning natural scenes, animals, architecture, objects, and imaginative worlds.The general galleries also demonstrate broad visual diversity, fine-grained details, and strong aesthetic quality across multiple domains.
  • Contributor List (Alphabetical Order): Lens produces portraits with expressive details, natural lighting, identity diversity, cinematic composition, and varied cultural and narrative settings.The portrait samples emphasize fine-grained facial details and rich contextual storytelling.
  • Contributor List (Alphabetical Order): Lens generates legible typography across posters, signs, product labels, stylized graphic designs, storefronts, murals, and environmental signage.These text-rich generations cover diverse visual contexts and label-like compositions.
  • Contributor List (Alphabetical Order): Lens follows multilingual prompts involving culturally representative cuisines, landmarks, traditional clothing, natural landscapes, and language-specific visual contexts.The examples span regional foods, iconic architecture, scenic destinations, cultural identity, and regional scenery.
  • A.1 Foundational T2I Models: T2I foundation models have progressed from latent diffusion models to diffusion Transformers, rectified-flow models, and proprietary multimodal image-generation systems.Early latent diffusion models such as Stable Diffusion established compressed-latent generation, reducing training and inference costs relative to pixel-space diffusion.
  • A.1 Foundational T2I Models: Unified multimodal and autoregressive generators integrate image synthesis with language modeling and visual understanding through frameworks such as Janus-Pro, Transfusion, and BAGEL.Janus-Pro uses a unified autoregressive framework, Transfusion combines next-token prediction with diffusion over continuous image representations, and BAGEL scales decoder-only multimodal pretraining on interleaved text, image, video, and web data.

A.2 Post-training for T2I Models · A.3 Distillation for T2I Models

Post-training improves T2I alignment and generation quality through supervised fine-tuning, preference optimization, and reinforcement learning, with reward design remaining critical. Distillation and related acceleration methods reduce the expensive iterative sampling burden by enabling one-step or few-step generation.

  • A.2 Post-training for T2I Models: Post-training targets alignment and generation quality beyond likelihood-based pre-training by optimizing human preferences, reward-model judgments, or task-specific criteria.The field has progressed from early supervised fine-tuning toward preference-based optimization and reinforcement learning.
  • A.2 Post-training for T2I Models: DPO-style methods train diffusion models on positive-negative image pairs or preference sets, offering relatively simple and stable preference optimization without explicit reward maximization during sampling.Representative approaches include Diffusion-DPO, D3PO, SPO, and related variants.
  • A.2 Post-training for T2I Models: Policy-gradient RL methods formulate image generation as sequential decision-making and directly optimize denoising or flow-matching trajectories.Flow-GRPO and related variants convert deterministic sampling into an equivalent stochastic formulation for continuous generative dynamics.
  • A.2 Post-training for T2I Models: Reward design must capture aesthetic quality, prompt faithfulness, object correctness, spatial and compositional consistency, text rendering, and safety constraints.Poor rewards can cause reward hacking, reduced diversity, or misalignment with human preferences.
  • A.3 Distillation for T2I Models: Iterative sampling is a central T2I bottleneck, motivating training-free acceleration methods that reduce function evaluations through improved inference solvers.Examples include DDIM, DPM-Solver, and UniPC, although quality can degrade when the sampling budget becomes small.
  • A.3 Distillation for T2I Models: Distillation compresses multi-step teacher models into one-step or few-step students by matching teacher trajectories or learning self-consistent mappings from noisy states to clean data.Progressive distillation, consistency models, and Latent Consistency Models apply these principles to general and latent-space T2I generation.
  • A.3 Distillation for T2I Models: Distribution-level and adversarial supervision further improve few-step distillation by enhancing perceptual quality, matching target data distributions, and reducing reliance on paired teacher trajectories.Adversarial Diffusion Distillation, Distribution Matching Distillation, and DMD2 represent these directions.

A.4 VAE

The section motivates generation-friendly VAEs by noting that reconstruction-focused latents may not be optimal for generative modeling. It reviews approaches that align tokenizer representations with generative objectives while improving semantic awareness, capacity, and scalability.

  • Motivation: Reconstruction-optimized latents may preserve pixel information without providing representations that are optimal for downstream generative modeling.VAEs bridge pixel space and latent representations in diffusion-based generation models, but reconstruction-oriented training can mismatch generative objectives.
  • Generation-friendly tokenizers: Recent tokenizers reduce this mismatch by aligning encoder representations with diffusion features or jointly optimizing reconstruction and generation objectives.REPA-E aligns encoder representations with diffusion Transformer features, while Unified Latents jointly trains reconstruction and generation objectives.
  • Capacity and semantic awareness: Other approaches improve tokenizer capacity, structure, or semantic awareness through visual understanding tasks and masked, multiscale, discrete, or sequential latent representations.Examples include VTP, MagViT-v2, VAR, and TiTok, which target richer semantic information, generative learnability, or scalability.

B More Results … C.1 Prompt-rubric Visualization

The paper reports Lens comparisons across multiple benchmarks and reasoner configurations, then visualizes taxonomy-driven prompt rubrics that check counts, placement, attributes, and structural coherence. The examples span playground, clock-tower, library, and panda scenes with detailed spatial and visual requirements.

  • B.1 Detailed Benchmark Results: Lens is compared with state-of-the-art models on OneIG (EN), GenEval, LongText (EN), and CVTG.These comparisons are presented in Tables 3, 4, and 5.
  • B.2 Various Reasoners: Lens variants are evaluated with no reasoner, GPT-5.5, GPT-OSS-20BA3B, and Qwen3 reasoners sized 0.6B, 1.7B, and 4B.The reasoner comparison appears in Table 6.
  • C.1 Prompt-rubric Visualization: Prompt-rubric examples define checks for exact object counts, spatial placement, actions, sign text, visual attributes, and overall structural integrity.The playground example verifies one bright red kickball, five empty swings, one child, left-foreground placement, sign text, clothing, shadows, and physical coherence.
  • C.1 Prompt-rubric Visualization: The playground rubrics verify the child’s left-foreground position, the swings’ relationship to the child, and the specified playground sign text.Additional checks cover a bright-red kickball, blue sneakers, dusty knees, and crisp midday shadows.
  • C Visualization: The clock-tower example checks centered low-angle composition, one tower, warm ochre sandstone, green-patinated copper, an ivory face with black numerals, and golden-hour lighting.It also evaluates gray cobblestones, red banners, and soft background bokeh.
  • C.1 Prompt-rubric Visualization: The library example specifies a central aisle between four long oak tables leading toward a distant circulation desk, with warm lamps and wood paneling.Its rubrics also check the dusty atmosphere, table count and appearance, desk presence, and aisle geometry.
  • C.1 Prompt-rubric Visualization: The panda example evaluates right-foreground shoulder placement, center-left placard placement, dense black-and-white fur, and dewy blurred bamboo.The prompt includes a brushed-steel placard reading “GIANT PANDA — Ailuropoda melanoleuca” under morning zoo light.

C.2 Lens-800M Training Data Visualization

Figure 17 visualizes examples from Lens-800M, a training set composed of densely captioned image-text pairs.

  • C.2 Lens-800M Training Data Visualization: Figure 17 shows several examples from the Lens-800M training set, with each sample consisting of a densely captioned image-text pair.

D Implementation Details · D.1 RL Details · D.2 Distillation Details

The implementation details describe DiffusionNFT-based reinforcement learning with bounded rewards, mixed-resolution LoRA finetuning, and stabilization choices, followed by DMD-based distillation into a 4-step generator. RL uses Lens-RL-8K for 180 steps, while distillation combines score matching, adversarial supervision, and teacher-guided updates.

  • D.1 RL Details: DiffusionNFT optimizes the flow-matching objective directly in the forward diffusion process, using rewards to distinguish desirable from undesirable generation directions.High-reward samples emphasize positive velocities, whereas low-reward samples emphasize negative velocities to move away from undesirable trajectories.
  • D.1 RL Details: Rewards are normalized into a bounded optimality probability r ∈[0, 1], stabilizing the balance between positive-direction learning and negative-direction avoidance.The normalization factor Z_c is typically set to the global standard deviation of rewards.
  • D.1 RL Details: 180 RL steps on Lens-RL-8K use 64 NVIDIA A100 80GB GPUs, mixed-resolution buckets, and LoRA with rank r=64 and scaling factor α=128.The nine buckets span aspect ratios from 736×1472 through 1472×736 at a fixed base area of 1024^2.
  • D.1 RL Details: RL applies a KL penalty β_KL=1 × 10^-4 and AdamW with β_1=0.9, β_2=0.999, ϵ=10^-8, weight decay 1 × 10^-4, and learning rate 3 × 10^-4.The implementation also sets β=1 and uses η_i = min(0.001, 0.5) for stabilization.
  • D.2 Distillation Details: DMD distills the multi-step model by matching the student distribution to the target data distribution through an approximate score-based gradient.A frozen teacher score pulls samples toward the target distribution, while a fake score model counteracts student over-concentration.
  • D.2 Distillation Details: Distillation uses a 100K image-caption subset balanced across major generation scenarios, initializes the student and fake score model from the RL checkpoint, and keeps the teacher frozen.The objective combines a CFG-augmented term L_CA with a distribution-matching term L_DM, plus an adversarial branch using frozen-teacher features.
  • D.2 Distillation Details: 4-step distillation uses four fake-score and discriminator updates per student update, IDA with µ = 0.03, guidance scale 5.0, and up to 1K global steps on 8 NVIDIA A100 80GB GPUs.The student and fake score model use learning rate 5 × 10^-7, while the discriminator uses 1 × 10^-4.

E Prompt … E.8 Prompt for LongText and CVTG Benchmarks

The LongTextBench and CVTG prompt rewrites prioritize accurate rendering of required visible text while preserving the original scene, layout, category, and visual meaning. They also structure text-heavy scenes and interfaces for readability, using concise single-paragraph outputs.

  • E.8 Prompt for LongText and CVTG Benchmarks: The reasoner rewrites LongTextBench and CVTG inputs into improved image-generation prompts focused on readable text while preserving scene and layout intent.These benchmarks cover signs, labels, printed documents, webpages, and app screens.
  • E.8 Prompt for LongText and CVTG Benchmarks: Required visible text must be reproduced exactly, without omission, translation, paraphrase, correction, reordering, or normalization.The instructions preserve capitalization, punctuation, numbers, dates, URLs, currency symbols, units, and spacing as much as possible.
  • E.8 Prompt for LongText and CVTG Benchmarks: Long menus, maps, letters, and documents should use wide text blocks with large line spacing to improve readability.Decorative borders, illustrations, shadows, folds, and background objects should remain away from required text.
  • E.8 Prompt for LongText and CVTG Benchmarks: Webpages and app screens should use clean front-facing layouts with readable navigation, hero text, cards, buttons, tabs, and interface labels.Non-required interface text should be minimal or abstract, with simple sections and sufficient spacing for required text.
  • E.8 Prompt for LongText and CVTG Benchmarks: The rewritten output must be exactly one coherent paragraph that starts with the image description and contains no headings, lists, markup, explanations, questions, or alternatives.The prompt should remain concise and text-focused rather than discussing the instructions.
  • E.8 Prompt for LongText and CVTG Benchmarks: The recommended prompt length is 80 to 160 words for short examples and 140 to 260 words for long or text-heavy examples.The output must not mention the rewriting instructions.

F Broader Impacts and Limitations

Lens aims to reduce the computational cost of foundational text-to-image training, broadening access to visual content creation while requiring safeguards against misleading, biased, or harmful outputs. Its main limitations are weaker non-English prompt performance and remaining needs for broader data coverage, stronger refinement, and more robust safety mechanisms.

  • Broader impacts: Lens reduces training costs for foundational T2I models, lowering barriers to research and development in visual content creation.The paper identifies image generation, design assistance, education, and creative applications as areas that could benefit.
  • Safety risks and safeguards: Lens may be misused to create misleading, biased, or harmful visual content, motivating a reasoner that rejects inappropriate requests before generation.Responsible deployment should also include content moderation, provenance tracking, and misuse detection.
  • Limitations: Lens is trained primarily on English text–image pairs, and its generation quality and prompt-following accuracy may be lower for Chinese, French, and other non-English prompts.Some multilingual generalization emerges from the language encoder and pre-training, but it is incomplete.
  • Future work: Future work should expand multilingual and text-rich data, improve long-tail coverage, strengthen post-training or refinement, and develop more robust safety mechanisms.These improvements target responsible real-world deployment as well as model capability.
Loading 2605.21573v1…