PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen-Image-PE: Learning Visual Design for Text-to-Image Prompt Enhancement

Zekai Zhang, Yanran Zhang, Shengming Yin, Yan Shu, Zhening Liu, Bingxuan Zhao, Chenyang Zhu, Donghao Zhou, Jiahao Li, Jiahao Wang, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Qihang Ai, Rui Song, Shuo Cao, Xiao Xu, Xiaoyue Chen, Yuxiang Chen, Zhendong Wang, Zhouyang Li, Zihao Liu, Zikai Zhou, Bo Zheng, Huishuai Zhang, Dongyan Zhao, Chenfei Wu

arXiv:2610.33472v1cs.CV

TL;DR

Prompt enhancement often starts from brief requests that leave visual-design choices unspecified, and existing approaches emphasize intent or overall appeal more than fine-grained design. Qwen-Image-PE learns design through caption supervision and groupwise visual comparison, improving generation performance and complementary design evaluations.

  • Problem

    Existing prompt enhancers preserve intent or improve overall appeal, but place less explicit emphasis on fine-grained design dimensions such as lighting, composition, layout, and detail.

    Source: §1 · p. 2

  • Method

    Qwen-Image-PE uses supervised design imitation from dense captions of well-designed images, followed by reinforcement learning that jointly compares candidate images for each request.

    Source: §1 · p. 2

  • Results

    Qwen-Image-PE-397B improves Overall over original requests by 7.81 points, and evaluations also show gains in visual design, aesthetics, image quality, and human preference.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Qwen-Image-PE scales with training data, model size, and information density, and transfers across four additional T2I backbones, with sufficient information density a prerequisite for effective enhancement.

    Source: §1 · p. 2

Abstract

from arXiv · show

Learning visual design is essential for prompt enhancement (PE) to turn underspecified user requests into coherent visual specifications for text-to-image (T2I) generation. Existing methods preserve user intent and improve overall visual appeal, with less explicit emphasis on fine-grained design dimensions. To address this gap, we introduce Qwen-Image-PE, which learns visual design through two-stage training. Supervised fine-tuning with Caption-Based Reverse Distillation (CRD) uses dense captions of well-designed images as design supervision. Reinforcement learning with Groupwise Design Reward (GDR) jointly compares all candidate images for the same request to refine lighting, composition, layout, and detail. On Qwen-Image-Bench with Qwen-Image-2.1, Qwen-Image-PE-397B improves over original requests by 7.81 points, while Qwen-Image-PE-9B outperforms all evaluated PE baselines. Further evaluations show gains in visual design, aesthetics, image quality, and human preference. We find that PE scales along three axes: training data, model size, and information density, with sufficient information density as a prerequisite for effective enhancement. Moreover, we demonstrate that Qwen-Image-PE generalizes to four additional T2I backbones, even surpassing native prompt enhancers on HunyuanImage-2.1 and FLUX 3 Image Klein. We release Qwen-Image-PE-9B at https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I.

1 Introduction

Brief user requests leave visual-design choices unspecified, while existing prompt enhancers focus on intent or overall appeal rather than fine-grained dimensions. Qwen-Image-PE addresses this gap with two-stage design learning and reports gains across benchmark and complementary evaluations.

1 Introduction

  • User requests are often brief and underspecify visual choices, while existing enhancers emphasize intent preservation or overall appeal more than individual design dimensions.Source: §1 · p. 2
  • Dense captions make design choices in well-designed images explicit, and jointly comparing candidates for the same request provides a common basis for more informative relative-quality feedback.Source: §1 · p. 2
  • The method uses dense captions of well-designed images to supervise design imitation, then groupwise visual comparison to refine lighting, composition, layout, and detail.The groupwise reward scores all candidate images for the same request together.Source: §1 · p. 2, Figure 2 · p. 3
  • Qwen-Image-PE-397B improves Overall over original requests by 7.81 points, while Qwen-Image-PE-9B outperforms all evaluated prompt enhancement baselines in Overall.Complementary evaluations show gains in visual design, aesthetics, image quality, and human preference.Source: §1 · p. 2
  • Qwen-Image-PE performance scales with training data, model size, and information density, with sufficient information density a prerequisite for effective enhancement in the evaluated setting.Source: §1 · p. 2
  • Qwen-Image-PE transfers across four additional T2I backbones without their training captions and outperforms native prompt enhancers on HunyuanImage-2.1 and FLUX 3 Image Klein.Source: §1 · p. 2

2 Method

Qwen-Image-PE enhances requests while preserving explicit requirements and specifying visual design choices. It learns these choices from image-grounded captions, then refines them with groupwise visual comparisons.

2.1 Formulation of Prompt Enhancement

  • The enhancer generates a rationale, enhanced prompt, and aspect ratio that condition a frozen T2I generator to synthesize an image.The design objective covers lighting, composition, layout, and detail while preserving the user's explicit requirements.Source: §2.1 · p. 3

2.2 Supervised Fine-Tuning with Caption-Based Reverse Distillation

  • Caption-Based Reverse Distillation (CRD) trains the enhancer to predict dense captions of selected, well-designed images from user requests.Request–image pairs are selected for design quality and request consistency, and rationales connect requirements to captioned design choices.Source: §2.2 · pp. 3–5

2.3 Reinforcement Learning with Groupwise Design Reward

  • The reinforcement-learning stage samples candidate prompts and images for design-related requests, then jointly scores each candidate group across design dimensions.The total reward combines dimension-specific design rewards with an image–request alignment reward; GRPO updates the SFT policy with group-normalized advantages and KL regularization.Source: §2.3 · p. 5
  • GDR uses DK judge calls per group, versus DG for pointwise scoring and D(G 2) for exhaustive pairwise scoring, with K ≪G.All candidates for a request are judged together under the same design rubric; scores are averaged across shuffled presentations to reduce sensitivity to order.Source: §2.3 · p. 5

3 Experiments

Experiments evaluate Qwen-Image-PE’s generation quality, visual design, and human preference, then test its two training stages. Results support caption-based supervision and groupwise feedback, with gains across design dimensions and in human evaluations.

3.2 Main Results

  • Qwen-Image-PE-397B leads on eight of nine visual-design and aesthetics metrics, while Qwen-Image-PE-9B beats trained PE baselines on all metrics.The evaluation covers fine-grained design, aesthetics, and image quality.Source: Table 2 · p. 7, §3.2 · p. 7

3.1 Experimental Setup

  • The evaluation assesses generation performance, visual design, aesthetics, image quality, and human preference using benchmark, model-based, and human measures.Evaluation requests do not overlap with SFT or RL data, and evaluation metrics are distinct from training rewards.Source: §3.1 · p. 7

3.2 Main Results

  • Qualitative examples show more expressive lighting, cohesive compositions, organized layouts, and richer details than original requests or baseline enhancers.Examples include directional lighting emphasizing a bed’s form and aligned headings establishing reading order.Source: Figure 5 · p. 8, §3.2 · p. 8
  • The 397B model achieves positive net win rates against every baseline in a 500-case expert GSB evaluation.Net win rates range from 13.60% over FLUX 3 Image Klein PE to 35.48% over Original Request.Source: §3.2 · p. 8, Figure 4 · p. 8

3.3.1 Effectiveness of Caption-Based Reverse Distillation

  • Caption-Based Reverse Distillation raises Overall from 59.86 to 61.52 (+1.66) and improves all five dimensions over Forward-Based Supervision.The largest gains are Aesthetics (+3.00) and Creative Generation (+2.42).Source: §3.3.1 · p. 9

3.3.2 Effectiveness of Groupwise Design Reward

  • Groupwise Design Reward reaches 62.17 Overall, exceeding Pointwise Design Reward by 0.81 points and the SFT initialization by 0.65 points.It improves four of five dimensions over pointwise scoring; its reward ordering agrees with human preferences 59.1% of the time versus 50.9% for pointwise scoring.Source: Table 3 · p. 9, §3.3.2 · p. 10
  • During RL training, Lighting rises from 72.14 to 77.64 with the most consistent improvement, while Layout improves only slightly with substantial fluctuations.Benchmark evaluation uses pointwise scoring by a separate evaluator, not the groupwise training rewards.Source: Figure 6 · p. 9, §3.3.2 · p. 10

4 Scaling Properties of Qwen-Image-PE

The scaling study varies training-data size, enhancer model size, and caption information density while evaluating SFT models on Qwen-Image-Bench. Performance increases along all three axes, though gains from data growth diminish over the tested range.

Scaling Training Data

  • Increasing training data from 12.5K to 100K examples raises Overall from 55.87 to 57.61, a 1.74-point gain with diminishing increments.The successive gains are 0.81, 0.63, and 0.30 points.Source: §4 · p. 11

Scaling Model Size

  • With 50K examples fixed, Overall increases monotonically from 49.56 for Qwen3.5-2B to 59.86 for Qwen3.5-397B-A17B, a 10.30-point gain.All five benchmark dimensions improve between the smallest and largest models.Source: §4 · p. 11

Scaling Information Density

  • Across information-density levels L1–L4, Overall rises from 44.65 to 57.31; L1 and L2 trail Original Request (54.36), while L3 and L4 exceed it.The total improvement is 12.66 points, and the passage identifies sufficient information density as a prerequisite for effective enhancement in this setting.Source: §4 · p. 11

5 Generalization Behavior of Qwen-Image-PE

Qwen-Image-PE transfers beyond its Qwen-Image-2.1 training backbone to four additional T2I backbones. Both variants outperform original requests across them, and surpass native enhancers on two.

Consistent improvements across backbones

  • Both PE variants outperform original requests on all four additional backbones, with gains across all five benchmark dimensions.Overall improves by 2.53–6.74 points for Qwen-Image-PE-9B and 3.86–8.03 points for Qwen-Image-PE-397B.Source: §5 · p. 11, Table 5 · p. 12

Transfer beyond native prompt enhancement

  • Both variants surpass native prompt enhancers in Overall on HunyuanImage-2.1 and FLUX 3 Image Klein.Qwen-Image-PE-397B exceeds the respective native enhancers by 3.07 and 2.74 points, without access to the target generators’ training captions.Source: §5 · p. 11, Table 5 · p. 12

6 Related Work

Prior T2I systems support detailed visual control, but user requests often leave presentation choices unspecified. Related PE work emphasizes intent preservation and overall appeal; Qwen-Image-PE targets fine-grained design within individual images.

Text-to-Image Generation

  • T2I systems increasingly support detailed control over lighting, composition, layout, and detail, while user requests often leave these choices unspecified.The cited work frames prompt enhancement as specifying and coordinating those choices into visual designs (Wu et al., 2025a; Zhao et al., 2026; ByteDance Seed, 2026; Black Forest Labs, 2025; OpenAI, 2026b; Google DeepMind, 2026).Source: §6 · p. 12

Prompt Enhancement for Text-to-Image Generation

  • Existing PE methods preserve intent and clarify requests, while other approaches optimize visual appeal through aesthetic or preference rewards.ERNIE-Image-PE also uses structured descriptions of layout, composition, textual content, and scene structure (ERNIE Team, Baidu, 2026).Source: §6 · p. 12
  • Qwen-Image-PE targets lighting, composition, layout, and detail in individual images, unlike WanPE’s caption-based cinematic planning across shots and time (Zhu et al., 2026).The paper also studies PE scaling and generalization across T2I backbones.Source: §6 · p. 12

7 Conclusion

Qwen-Image-PE learns fine-grained visual design through imitation and comparative refinement. Experiments report improvements across generation performance, visual design, and human preference, alongside evidence of scaling and transfer across T2I backbones.

7 Conclusion

  • Experiments show improvements in overall generation performance, visual design, and human preference.Source: §7 · pp. 12–13
  • Qwen-Image-PE learns fine-grained visual design through imitation and comparative refinement.Caption-Based Reverse Distillation provides design supervision, while Groupwise Design Reward refines choices through groupwise visual comparison.Source: §7 · p. 12
  • Sufficient information density is a prerequisite for effective prompt enhancement, according to the scaling experiments.Source: §7 · p. 13
  • Both PE variants improve Overall over original requests on four additional T2I backbones without access to their training captions.The authors interpret this as support for transfer of learned visual design principles.Source: §7 · p. 13

Related papers

Loading 2610.33472v1…