Source-linked AI summary
Scaling Properties of Text Conditioning in Visual Generation
Zilong Chen, Chaorui Deng, Kunchang Li, Hongyi Yuan, Haoqi Fan
TL;DR
Text-to-image models lack well-measured scaling laws for caption informativeness beyond natural-language length. This paper measures structured-language scaling, develops structured prompts and a trained prompter, and obtains broad gains across compositional, reasoning, and world-knowledge evaluations.
Problem
Text-to-image models receive weaker conditioning supervision for visual content that captions omit or describe ambiguously, while caption-length scaling provides limited evidence.
Source: §1 · p. 3, §3.1 · p. 6Method
The paper calibrates GPG and ED against diffusion loss, constructs image-grounded structured prompts, and trains a prompter through distillation and verifier-gated refinement.
Source: §1 · p. 4, §3.3 · p. 10Results
The resulting system leads evaluated open-weight models on all but one reported metric and matches or surpasses the strongest closed systems on most.
Source: §1 · p. 4Takeaways & Limitations
Caption information content, organized through structured prompts and translated by a capable prompter, is a measurable axis for scaling text conditioning in visual generation.
Source: §5 · pp. 18–19The fitted scaling relations depend on image-conditioned judges or extractors and are backbone-specific, while structured prompting adds prompter latency.
Source: §3.2 · p. 9, §5 · p. 19
Abstract
from arXiv · showhide
We study empirical scaling properties for text conditioning in visual generation. Such properties have rarely been measured because diffusion loss does not scale with the number of tokens in natural-language prompts. Surprisingly, we find that the converged diffusion loss scales with the amount of structured language in the prompt. To quantify structured language, we adapt two complementary measures: a white-box likelihood metric (GPG) and a black-box attribute metric (ED). Across controlled training runs, the converged diffusion loss decreases approximately linearly with GPG and follows a power law with ED. Guided by these scaling properties, we improve \emph{diffusability} by constructing structured prompts with semantic and geometric annotations derived from images, and improve \emph{promptability} by training a prompter through supervised fine-tuning, cold-start, and verifier-gated on-policy distillation. The resulting system outperforms all evaluated open-weight models on nearly every compositional, reasoning, and world-knowledge benchmark, while matching or surpassing the strongest closed-weight models on most evaluations.
1 INTRODUCTION
The paper argues that visual generation scales with image-grounded information in text conditioning rather than prose length alone. It develops structured prompts and a trained prompter to improve diffusability, promptability, and end-to-end generation quality.
- Longer natural-language captions fail to scale because verbosity adds little image-grounded information, whereas structured captions continue improving reconstruction and predict converged diffusion loss.Existing systems peak early or decline with longer prose, while structured prompts expose scene context, element properties, geometry, and relations through typed fields.Source: §1 · p. 3
- GPG and ED quantify caption informativeness, with converged diffusion loss fitting a linear function of GPG and following a power-law trend in ED.These relations are measured under fixed image data, architecture, initialization, and compute, enabling caption conditions to be ranked before retraining.Source: §1 · p. 4
- Structured prompts raise diffusability by organizing image-grounded annotations into named fields and lowering diffusion loss without architectural changes.An image-to-SP pipeline combines a VLM with frozen experts for human pose, depth, and segmentation to produce full-schema supervision.Source: §1 · p. 4
- Promptability improves through SFT, cold-start distillation, verifier-gated on-policy self-distillation, and agentic refine–render–judge inference-time refinement.The prompter infers unspecified visual details while preserving user requests, and refinement revises the structured fields responsible for failed visual decisions.Source: §1 · p. 4
- The resulting system leads every evaluated open-weight model on all but one reported metric and matches or surpasses the strongest closed systems on most evaluations.The largest gains occur on composition- and reasoning-heavy benchmarks, extending to prompt fidelity, visual coherence, and compositional detail.Source: §1 · p. 4
2 RELATED WORK
Prior work improves text-to-image generation through model scaling, richer captions, structured controls, and inference-time prompting. This paper instead measures caption information directly and studies how structured representations affect both diffuser training and prompter behavior under controlled comparisons.
Text-to-image scaling, long captions, and structured control
- Caption information, measured by GPG or ED, predicts converged diffusion loss more directly than prompt length when model, data, and compute are fixed.This contrasts with existing compute–loss scaling laws and aligns with benchmarks showing that longer prompts can reduce compositional accuracy or graph-structured alignment.Source: §2 · p. 4
- Unlike layout-conditioned or attention-intervention methods that steer a fixed diffuser, this approach encodes boxes, depth, and relations in ordinary text fields and evaluates them on shared GPG/ED axes.The comparison targets caption information rather than layout control.Source: §2 · p. 5
Recaptioning and caption quality
- Structured representations have improved annotation recall, prompt following, and reconstruction, but this work calibrates image-grounded information across representation and field choices against matched-budget diffusion loss.It also separates prompt production from diffusion training by varying prompter scale, reasoning, and training while holding the schema and trained diffuser fixed.Source: §2 · p. 5
LLM prompters, RFT, and inference-time methods
- Inference-time prompt enhancers improve prompts for fixed generators, whereas this work studies structured-caption diffusability and prompter promptability with fixed schema and backbone, including verifier-gated self-distillation.Its verifier filters prompter rollouts while RFT keeps the SP-trained diffuser fixed; GPG and ED complement alignment metrics by scoring caption informativeness relative to paired images.Source: §2 · pp. 5–6
3 METHOD
The method replaces lengthened free-form captions with structured prompts that expose image-grounded semantics and geometry, then improves diffusability through structured supervision and promptability through prompter scaling and training. Caption informativeness predicts converged diffusion loss, while the resulting two-factor interface supports stronger image generation.
3.1 FROM NATURAL-LANGUAGE CAPTIONS TO STRUCTURED PROMPTS
- Structured prompts organize global context, per-element properties and geometry, and cross-element relations, improving reconstruction as field groups increase while longer natural-language captions remain essentially flat.The representation is a typed JSON caption that assigns visual variables to named fields.Source: §3.1 · p. 6
3.2 CAPTION INFORMATIVENESS PREDICTS TRAINING LOSS
- Across 15 controlled configurations, converged diffusion MSE is approximately linear in GPG and follows a power law in ED, with residual standard deviations of approximately 6×10^-4 and 7.8×10^-4.The relations hold within the calibrated architecture, training recipe, and measured ranges; GPG and ED agree on configuration ranking with Spearman ρGPG,ED = 0.96.Source: Figure 7 · p. 8, §3.2 · p. 9
- The calibration predicts held-out spatial and field variants with mean absolute MSE errors of 5.0×10^-4 using GPG and 8.1×10^-4 using ED.Fits trained on NL and nested-SP families predict six excluded variants, separating loss gaps larger than the fit error.Source: §3.2 · p. 9
3.3 RAISING DIFFUSABILITY: CONSTRUCTING STRUCTURED SUPERVISION
- An image-to-SP pipeline combines VLM semantics with specialist pose, depth, mask, and occlusion evidence, then reconciles them into a coherent full-schema annotation and deterministic field ladder.The ladder derives L5–L9 from L10 by masking predefined field groups, isolating exposed information and organization from re-annotation quality.Source: §3.3 · pp. 10–11
- GenEval2 GM and order-swapped GSB against L5 improve monotonically with schema richness, showing that higher diffusability benefits generated images as well as training loss.Global scene context is the largest individual field-wise contributor, followed by bounding-box conditioning.Source: §3.3 · p. 11, Table 1 · p. 11
3.4 RAISING PROMPTABILITY: SCALING AND TRAINING THE LLM PROMPTER
- Prompter quality scales with model size and reasoning: thinking-mode GenEval++ rises from 46.4% at 0.8B to 86.8% at 397B, while chain-of-thought generally adds gains.At 0.8B, chain-of-thought can underperform because reasoning enters repetitive loops before producing valid JSON.Source: §3.4 · p. 12, Figure 9 · p. 12
- The prompter is trained with SFT, cold-start distillation, and verifier-gated on-policy self-distillation, transferring image-grounded preferences while keeping the inference prompter image-free.A conservative verifier accepts only prompt-faithful, structurally coherent, and visually acceptable renders; OPSD then transfers token-level teacher preferences on accepted trajectories.Source: §3.4 · pp. 12–13, Figure 10 · p. 13
4 EXPERIMENTS
Experiments show that the complete structured-prompt system generalizes across alignment, dense prompt following, world knowledge, and reasoning benchmarks, while prompter training and inference-time refinement provide complementary gains. Training improves the starting point, refinement mainly repairs structural errors, and improvements saturate after a short horizon.
4.1 END-TO-END PERFORMANCE AND MATCHED CONTROL
- Ours leads or ties nearly every evaluated open-weight metric and matches or surpasses closed-source systems on most benchmarks spanning composition, dense prompt following, world knowledge, and reasoning.Table 2 evaluates single-shot inference across complementary capabilities, while Figures 2 and 11 show improved spatial layouts, object counts, and attribute binding.Source: §4.1 · pp. 14–15, Table 2 · p. 15
- The same architecture and data with a free-form NL interface reach only 56.2 GenEval2 GM and 76.1 CoReBench, versus 72.5 and 85.2 for the structured-prompt system.The official Qwen-Image prompt enhancer scores 52.8 and 74.7, showing that retraining alone does not reproduce structured-prompt gains.Source: §4.1 · p. 15
4.2 ISOLATING PROMPTABILITY
- Training the prompter makes the strongest single-turn backend on all four metrics, while the full pipeline raises structure to 7.600 and GSB to 42.0%.SFT provides the largest single-stage structure gain, Cold-start further improves structure and GSB, and verifier-gated OPSD gives the strongest endpoint; DPG-Bench changes by only 1.29 points.Source: §4.2 · pp. 15–16, Table 3 · p. 16, Table 4 · p. 16
4.3 INFERENCE-TIME SCALING OF PROMPTABILITY
- At eight refinement rounds, the trained prompter reaches 54.7% GSB versus 44.7% for the strongest coding agent, while averaging 2.31 rounds to PASS versus 3.41 for Base.Training also raises single-shot structure and GSB from Base’s 6.107 and 24.0% to 7.600 and 42.0%, showing that refinement complements rather than replaces training.Source: §4.3 · p. 18
- Increasing the trained prompter’s budget from four to eight rounds raises structure from 8.213 to 8.260 and GSB from 54.0% to 54.7%, indicating rapid saturation.Average rounds increase only from 2.04 to 2.31, so longer prompt-side reasoning yields little additional benefit under the tested setup.Source: §4.3 · p. 18
5 CONCLUSION … A.1 DEFAULT SCORING PROTOCOL
The paper identifies caption information content as a scalable axis for text-to-image learning and frames captioning as a Diffusability×Promptability interface. Structured prompts and trained prompters improve generation, while inference-time refinement helps with diminishing returns and introduces measurement and deployment limitations.
5 CONCLUSION
- GPG and ED predict converged diffusion loss, showing that caption information content—not caption length—is a measurable scaling axis for text conditioning.These calibrated empirical relations enable comparisons among caption representations and motivate the Diffusability×Promptability view of the caption interface.Source: §5 · p. 18
Appendix
- The inference-time scaling protocol varies Tmax to allocate more refine–render–judge rounds under fixed schema, backbone, judge, and prompter weights.Final outputs use offline GPT-5.4 evaluation and order-swapped GSB preference against the zero-shot Qwen3.5-397B-A17B Base prompter over 150 prompts.Source: Table 5 · p. 19
5 CONCLUSION
- Structured prompts and trained prompters improve visual coherence, prompt fidelity, and compositional generation without changing the diffusion architecture.Structured prompts expose image-grounded variables in addressable fields, while prompter training translates user requests into detailed representations; matched natural-language retraining rules out training quantity alone as the explanation.Source: §5 · p. 19
- Refine–render–judge inference loops provide further gains, but rapidly diminishing returns make prompt-side inference compute a complement to training rather than a substitute.The trained prompter internalizes much of the correction, reducing the marginal value of additional refinement rounds.Source: §5 · p. 19, Table 5 · p. 19
- Both GPG and ED rely on image-conditioned measurement, so their fitted relations may shift with the chosen vision-language judge or attribute extractor.Agreement between the metrics lowers but does not eliminate the risk that their shared trend reflects a particular scoring interface.Source: §5 · p. 19
- The hand-designed schema remains limited, with automatic schema discovery and extension to video and three-dimensional generation left open.The structured-prompt path also adds prompter latency, requiring deployment tradeoffs between generation gains and inference cost.Source: §5 · p. 19
A.1 DEFAULT SCORING PROTOCOL
- Headline GPG scores use a frozen Qwen3.5-397B-A17B VLM judge and a fixed preprocessing, canonicalization, template, and content-mask recipe across caption conditions.The resulting GPG–loss relation calibrates this specific scoring recipe rather than independently validating the scoring choices; new caption families require the same recipe or recalibration.Source: §A.1 · p. 25
A.2 GPG ROBUSTNESS ACROSS JUDGES
GPG’s linear scaling relation is robust across compatible vision-language judges, with stable configuration rankings rather than dependence on judge size. However, robustness requires matching the judge to the caption’s spatial coordinate representation.
A.2 GPG ROBUSTNESS ACROSS JUDGES
- GPG depends on the complete judge interface, so headline comparisons hold the VLM, tokenizer, template, image processor, and coordinate convention fixed.The default Qwen3.5-397B-A17B judge uses a bbox convention directly compatible with the caption’s 0–999 coordinates, avoiding coordinate conversion.Source: §A.2 · p. 25
- Without a bbox-format adapter, Qwen2.5-VL-7B inverts the ordering (r = +0.23), whereas the native-coordinate adapter restores a strong negative fit (r = −0.97).This exception shows that GPG remains judge-dependent but its configuration ordering is stable once the judge can parse the caption’s spatial representation.Source: §A.2 · p. 26, Table 6 · p. 26
- Across six additional compatible judges, GPG shows strong negative fits (r = −0.91 to −0.99) and stable per-cell rankings (Spearman ρ≥0.86).Fully evaluated Qwen-family judges achieve R2 = 0.93–0.97; the 122B judge reaches 0.99 on six cells, which is not directly comparable to full-cell fits.Source: §A.2 · p. 25, Table 6 · p. 26, Figure 16 · p. 26
A.3 FULL 15-SETTING DATA
Table 7 provides the complete 15-setting GPG, ED, and converged diffusion-loss data underlying Eqs. (3) and (4), measured under shared data and compute conditions.
- All settings use the same 30,000 paired image UIDs for both caption-side measurements.Source: §A.3 · p. 26
- GPG uses Qwen3.5-397B-A17B with content masking and canonicalized JSON, ED follows its extraction-and-matching protocol, and MSE uses a unified 2.84×1010 cumulative-image-token budget.Source: §A.3 · p. 26
- Table 7 gives the full 15-setting scaling-property dataset, sorted by GPG, with the full-schema ablation baseline counted once as Structured L10.The full-schema baseline coincides with Structured L10.Source: Table 7 · p. 27
A.4 MONOTONICITY ANALYSIS
Across 15 settings, MSE decreases overall as GPG increases, despite small local reversals that are treated descriptively rather than assigned specific causes.
A.4 MONOTONICITY ANALYSIS
- Across 15 settings, MSE decreases overall when sorted by GPG, with local reversals totaling approximately 0.0016 MSE units.Approximately 0.0012 comes from structured L5 relative to nearby NL settings; remaining violations are |∆MSE|≤2×10−4, comparable to trailing-window read-out variation.Source: §A.4 · p. 27
A.5 FIT SENSITIVITY ACROSS SETTINGS AND TRAINING BUDGETS … B.3 SHARED-UID RESAMPLING SENSITIVITY
Across controlled settings, the scaling relations remain strong under resampling and matched budget refits, while ED’s empirical calibration is most predictive under precision-weighted F0.5. The validation details also define ED’s image-grounded extraction procedure and quantify uncertainty without interpreting these sensitivities as population confidence intervals.
A.5 FIT SENSITIVITY ACROSS SETTINGS AND TRAINING BUDGETS
- The sensitivity analyses are operational checks, not population confidence intervals, independent training replications, seed-level optimization uncertainty, or a decomposition of residuals into fit and run variability.Within-trajectory block-mean standard deviations span 1.0–5.0 × 10−4, with median 3.5×10−4, versus residuals of approximately 6×10−4 for GPG and 7.8 × 10−4 for ED.Source: §A.5 · pp. 27–28
- Mean±SD fit magnitudes across B =2,000 resamples are |r| = 0.984 ± 0.008 for GPG and |r| = 0.973 ± 0.012 for ED.The resampling evaluates sensitivity to the 15 deliberately designed sweep settings.Source: Figure 17 · p. 27, §A.5 · p. 27
- Across six matched budget cuts, refitting the same trajectories preserves the linear GPG relation and ED power law, with slopes and exponents steepening mildly during training.The refits use twelve settings and trailing-window training MSE read-outs, with corrected token axes for two restarted runs.Source: §A.5 · pp. 27–28, Figure 18 · p. 28
A.6 MUTUAL-INFORMATION MOTIVATION AND THE EMPIRICAL STATUS OF THE RELATIONS
- ED agrees closely with GPG ordering (ρSpearman =0.96) and follows a negative power-law trend, but both relations are empirical calibrations rather than information-theoretic theorems.GPG is judge-dependent, while converged flow-matching velocity MSE is not measured in nats and residuals do not directly quantify unused information.Source: §A.6 · p. 28
B.1 IMAGE-GROUNDED SOURCE EXTRACTION
- ED is built from image-only OARG extraction averaging 102 tuples per image, caption-side extraction, and symmetric paraphrase-tolerant matching on 30,000 paired images.Pair scores use attribute precision and recall, F0.5, and a two-sided 10% trimmed mean; cached sources, tuples, and match masks are reused.Source: §B.1 · pp. 28–29
B.2 WHY PRECISION-WEIGHTED F0.5?
- Unsupported caption attributes create contradictory supervision, whereas omissions reduce conditioning bandwidth, motivating the fixed precision-oriented F0.5 convention.The weighting was selected during controlled BAGEL metric development and then frozen at β = 0.5 for subsequent analyses.Source: §B.2 · p. 29
- R2 decreases from 0.943 for F0.5 to 0.313 for F2, while variant MAE more than doubles, although the ED–loss relation remains negative and the nested SP ladder monotonic.The precision-oriented F0.5 score is therefore markedly more predictive, despite the relation’s direction being robust to the weighting choice.Source: §B.2 · p. 29
B.3 SHARED-UID RESAMPLING SENSITIVITY
- Shared-UID bootstrap resampling yields Spearman ρ = −0.979 ± 0.010 between ED and converged MSE, with central 95% range [−0.993, −0.957].The ranges condition on cached proposer, extractor, and matcher outputs, excluding API/model stochasticity and training-run uncertainty.Source: §B.3 · p. 29
B.4 ROBUSTNESS ACROSS SOURCE EXTRACTORS
ED’s relationship with converged MSE is robust to the image-source extractor: extractors preserve rankings and the negative power-law relationship, but shift absolute ED scales. Accordingly, ED is used as a relative ruler with one fixed extractor throughout the paper.
Robustness across source extractors
- Absolute ED agreement is low across extractors, with ICC 0.11 for absolute agreement versus 0.64 for consistency, so comparisons use rankings and slopes only.The paper fixes Gemini 3 Pro as the extractor and never compares absolute ED values between backends.Source: §B.4 · p. 30
- ρ(ED, MSE) remains −0.86 to −0.90 across extractors, while Table 9 reports stable cell rankings and tuple agreement despite backend changes.Cross-backend cell-rank ρ is 0.80–0.92 and pairwise per-tuple Cohen κ is 0.75–0.77; the power-law slope retains its sign and rough magnitude.Source: §B.4 · p. 29, Table 9 · p. 30
- ED values in Table 7 support the power-law fit, with Grid-k denoting verbal locations on a k×k grid and ablations masking the named L10 field.The captioned settings report ED alongside GPG and converged MSE.Source: §B.4 · p. 30
C STRUCTURED-PROMPT SCHEMA AND IMPLEMENTATION DETAILS
The appendix specifies the structured-prompt schema and the full implementation pipeline, from image-derived annotation and diffusion training to prompter optimization, inference, and dependencies. It also defines the controls and mechanisms needed to reproduce the system, including extensible fields, frozen annotation models, verifier-gated distillation, and structured agentic refinement.
C.2 ANNOTATION PIPELINE
- The annotation pipeline combines global and per-element VLM descriptions with pose, depth, masks, occlusion, and relation evidence before assembling a well-formed structured record and deterministic degradations.Seed-VL creates the scene skeleton; Sapiens overlays 133 keypoints, DepthAnything V2 quantizes depth from 0 to 255, and SAM 2.1 supplies masks and occlusion cues.Source: §C.2 · pp. 30–31
C.1 STRUCTURED-PROMPT SCHEMA
- The L10 schema requires intent, macro scene, and elements, while optional fields encode relationships and global controls; ratio is an inference-only canvas control.Dynamic per-element keys allow new attributes and actions without changing the schema, and the wrapper removes ratio before diffusion conditioning.Source: §C.1 · p. 30, Table 10 · p. 31
C.3 DIFFUSION-BACKBONE TRAINING
- The diffusion implementation trains BAGEL per scaling cell and Qwen-Image once on the structured-prompt corpus, using frozen KL-VAEs, packed sequences, and rectified-flow matching.BAGEL supports the scaling-property fit, whereas Qwen-Image remains fixed across prompter ablations and main results; both use multi-aspect resolutions and block-diagonal attention.Source: §C.3 · p. 32, Table 12 · p. 33
C.5 PROMPTER TRAINING
- The prompter is a rank-128 LoRA on frozen Qwen3.5-397B-A17B trained serially with SFT, Cold-start, and RFT under shared LoRA topology.SFT uses approximately 333k token-balanced examples and teaches plausible structured completions rather than a deterministic prompt-to-layout mapping.Source: §C.5 · p. 33, §C.5.1 · p. 34
C.5.1 TRAINING STAGES
- Cold-start filters image-conditioned teacher traces with strict alignment and reasoning-quality gates, retaining 50,182 deduplicated examples from 172,208 candidates for approximately eight epochs.The gate accepts 58,907 candidates, while unsupported imagined specifics account for 71.9% of rejected traces and are the sole rejection reason in 40.7%.Source: §C.5.1 · pp. 34–35
C.5.2 OPSD: ON-POLICY SELF-DISTILLATION
- OPSD distills an image-conditioned frozen teacher into the image-free student on on-policy rollouts, applying the objective only to verifier-accepted samples scoring at least 6/10 on structure, alignment, and aesthetics.The verifier selects high-confidence rollouts, while the teacher supplies token-level targets; the full method uses neither an auxiliary token cross-entropy term nor a weighted combination of signals.Source: §C.5.2 · p. 35
C.6 INFERENCE AND THE AGENTIC LOOP
- At inference, greedy single-shot prompter decoding emits reasoning plus JSON, after which the wrapper strips reasoning, uses ratio for canvas selection, and passes the remaining record to diffusion.Qwen-Image renders with 50 Euler denoising steps and guidance scale 4.0; the agentic loop instead iteratively edits named fields after structured critiques, stopping on PASS or after eight rounds.Source: §C.6 · p. 36
C.7 EXTERNAL DEPENDENCIES AND LICENSES
- Table 14 records the external code and model dependencies together with available upstream license information, explicitly marking repositories whose licensing remains unavailable or requires confirmation.Unverified terms are not treated as verified licenses.Source: §C.7 · p. 36
C.4 SCHEMA FIELD ABLATION
A controlled schema-field ablation identifies scene context as the dominant contributor to diffusion learning, while faithful expert-derived annotations remain important for correct conditioning even when their marginal loss contribution is small.
C.4 SCHEMA FIELD ABLATION
- Removing scene context reduces GPG by 42 and raises MSE by 35.3 × 10−4, making it the most influential field-group ablation.Removing bounding boxes causes the second-largest MSE increase, 14.4 × 10−4; removing depth, relationships, or atmosphere/lighting changes MSE by at most 1.5 × 10−4.Source: §C.4 · p. 33
- The ablation ranks marginal loss contributions rather than annotation accuracy, so faithful expert-derived fields remain necessary for correct generation and editing.The comparison uses six retrained BAGEL runs at a common budget of 2.84 × 10^10 cumulative image tokens.Source: §C.4 · p. 33
D ADDITIONAL BENCHMARK RESULTS · D.1 DPG-BENCH PER-CATEGORY BREAKDOWN · D.2 GENEVAL PER-SKILL BREAKDOWN
This appendix documents evaluation protocols and score provenance, then provides category-level DPG-Bench and skill-level GenEval breakdowns for the reported benchmark results. The evaluations distinguish published baseline scores from matched re-evaluations of selected models.
D ADDITIONAL BENCHMARK RESULTS
- The appendix specifies evaluation protocols, score provenance, matched settings, and single-shot inference for the reported benchmark results.Published baselines follow leaderboards or original papers unless otherwise noted; matched NL and Ours use Appendix C.2 settings.Source: D ADDITIONAL BENCHMARK RESULTS · p. 36
- Nano Banana’s DPG-Bench score is sourced from He et al. (2026), while WISE combines legacy-protocol evaluations with specified re-evaluations.WISE uses GPT-4o-2024-05-13 under the legacy WiScore protocol; remaining scores come from published evaluations (Niu et al., 2025; Guo et al., 2025).Source: D ADDITIONAL BENCHMARK RESULTS · p. 36
- The protocol marks Qwen-Image∗’s official prompt enhancer and † re-evaluations on Qwen-Image-2512, clarifying which scores are directly comparable.Nano Banana, Qwen-Image∗, matched NL, and Ours are evaluated under the specified WISE protocol.Source: D ADDITIONAL BENCHMARK RESULTS · p. 36
D.1 DPG-BENCH PER-CATEGORY BREAKDOWN
- DPG-Bench is broken down by global scene description, entity presence, attribute binding, inter-entity relations, and other dense-prompt aspects, followed by an overall score.Table 15 reports published category breakdowns for baselines and matched per-category evaluations for Qwen-Image∗, matched NL, and Ours.Source: §D.1 · p. 36, Table 15 · p. 38
- The appendix also lists external code and model dependencies by pipeline role, including available upstream license information and explicitly unresolved entries.These dependencies are documented separately from the DPG-Bench category scores.Source: Table 14 · p. 37
D.2 GENEVAL PER-SKILL BREAKDOWN
- GenEval is reported by skill for single-object and two-object presence, counting, colors, position, and color-attribute binding, with overall accuracy last.Table 16 shows published per-skill baseline breakdowns and the evaluation for Ours; Qwen-Image∗ uses its official prompt enhancement.Source: §D.2 · p. 39, Table 16 · p. 39
D.3 WISE PER-CATEGORY BREAKDOWN
This section details legacy-protocol WISE per-category evaluation across six domains and explains how official overall WiScores are weighted and reported. It also distinguishes reproduced evaluations from transcribed published scores.
D.3 WISE PER-CATEGORY BREAKDOWN
- Legacy WISE scores consistency, realism, and aesthetic quality across cultural, temporal, spatial, biology, physics, and chemistry domains.The evaluator is GPT-4o-2024-05-13.Source: §D.3 · p. 40
- Models are grouped as dedicated T2I diffusion models, unified multimodal LLMs, proprietary systems, or open systems with prompt rewriting.Table 17 reports per-category WiScores under this legacy protocol.Source: §D.3 · p. 40, Table 17 · p. 40
- Official overall WiScore uses domain weights of 40%, 16.7%, 13.3%, 10%, 10%, and 10%, rather than the unweighted category mean.The four self-evaluated rows reproduce from their category entries within displayed precision, whereas published overalls are transcribed and may not be recoverable from rounded values.Source: §D.3 · p. 40
D.4 T2I-COREBENCH PER-CATEGORY BREAKDOWN … G ADDITIONAL QUALITATIVE EXAMPLES
This section details category-level benchmark analyses, audit criteria, and evaluation protocols for structured reasoning, image quality, factual errors, and prompter behavior. It also documents the prompts and filtering rules used to reproduce training, evaluation, editing, and qualitative examples.
D.4 T2I-COREBENCH PER-CATEGORY BREAKDOWN
- Ours has its largest T2I-CoReBench advantages in multi-attribute and multi-relation composition and across all eight reasoning categories, while Nano Banana leads multi-instance composition and GPT-Image-1 leads text rendering.Table 18 reports matched evaluation for Qwen-Image∗, matched NL, and Ours under the Gemini 2.5 Flash harness; composition and reasoning averages are unweighted means.Source: Table 18 · p. 41, §D.4 · p. 41
E.1 COLD-START TEACHER SYSTEM PROMPT
- The Cold-start teacher receives an image and user prompt, then emits a reasoned visual blueprint in an <analysis> block followed by a single structured JSON object.The prompter learns this derivation style from the caption alone, enabling inference without the image.Source: §E.1 · p. 42
- The teacher prompt enforces staged, reason-first construction covering knowledge, reasoning, aspect ratio, composition, layout, bboxes, depth, and spatial relationships.It separates factual knowledge from image-specific design and preserves uncertainty for under-specified attributes through hedged visual choices.Source: §E.1 · pp. 42–43
E.2 AESTHETIC JUDGE SYSTEM PROMPT
- The aesthetic judge evaluates visual quality separately from prompt alignment and structural accuracy, checks technical and compositional defects, and passes images with score >= 6.Its rubric covers exposure, saturation, banding, blur, artifacts, composition, lighting, color harmony, overall impression, and aspect-ratio suitability.Source: §E.2 · pp. 43–44
E.3 COLD-START FILTERING SYSTEM PROMPTS
- Validator v4 uses five independent Gemini calls to assess prompt–image alignment, introduction quality, imagined-detail justification, reverse rationalization, and reasoning-stage boundaries.The resulting accept/reject statistics use a strict gate over these five axis-specific outputs.Source: §E.3 · p. 44
E.3.1 PROMPT–IMAGE ALIGNMENT
- The alignment filter labels image–prompt pairs as aligned, partial, or mismatch based on named subjects, actions, settings, entities, genres, and elements, while ignoring unspecified style, quality, and non-dominant extras.It is designed to remove mismatched pairs that would teach the model wrong associations downstream.Source: §E.3.1 · pp. 44–45
E.3.2 INTRODUCTION QUALITY
- The introduction-quality filter does not penalize the teacher for adding detailed visual content beyond the user prompt, because enriching short descriptions into visual blueprints is the task’s purpose.The reference image is an intended guiding signal, and added clothing, colors, props, lighting, backgrounds, and accessories are acceptable.Source: §E.3.2 · p. 45
- Introduction quality requires visual additions to be framed as creative choices, common-sense knowledge, contextual implications, or a coherent build narrative rather than bald observational assertions.A coherent narrative can count as framing even without an explicit reason for every item; severity ranges from none to minor or major depending on how many details are unmotivated.Source: §E.3.2 · p. 45
E.3.3 JUSTIFICATION OF IMAGINED DETAILS
- Specific imagined attributes must include reasons tied to the prompt, canonical knowledge, creative motivation, or named context, because unsupported specifics teach hallucination without justification.Prompt-stated decisions, general knowledge, bounding-box coordinates, and tool-derived facts are excluded from this audit; severity depends on the number of naked specifics.Source: §E.3.3 · p. 46
E.3.4 REVERSE RATIONALIZATION
- Reverse rationalization occurs when a concrete decision precedes and merely back-fills its justification instead of genuinely deriving from acknowledged prompt evidence, omissions, and reasoning.Prompt-named decisions and concise summaries backed by real upstream reasoning do not count; severity increases when decisions-first reasoning dominates.Source: §E.3.4 · p. 47
E.3.5 REASONING-STAGE BOUNDARIES
- The reasoning trace assigns distinct purposes to stages, including Stage A for knowledge and common sense and Stage B for mapping user words to visual decisions.Stage-boundary auditing checks whether content stays within the purpose of its designated stage rather than crossing into another stage’s reasoning role.Source: §E.3.5 · p. 48
- The reasoning-stage boundary check distinguishes image-specific decisions, world knowledge, composition, and spatial layout, flagging only clear misplacements as minor or major violations.Stage D handles element-by-element imagining, while Stage E assigns spatial constraints to already described elements; brief bridging overlap is acceptable.Source: §E.3.5 · p. 48
E.4 VLM-AS-JUDGE SYSTEM PROMPTS
- The offline judge uses independent GPT-5.4 Structure and Alignment calls, each scoring one axis from 0–10 and reporting mean scores over 150 prompts.The Structure judge assesses anatomy, counts, layout, and physical plausibility; the Alignment judge assesses whether the image satisfies the prompt.Source: §E.4 · p. 48
E.4.1 STRUCTURE
- The Structure judge applies zero tolerance to visible defects, inspecting humans, animals, objects, composition, and text, with score >= 6 required to pass.Any clearly visible structural defect at normal viewing distance forces a score of at most 5 and a failed result.Source: §E.4.1 · pp. 49–50
E.4.2 ALIGNMENT
- The Alignment judge checks counts, required subjects, relationships, colors, style, contradictions, text, actions, and mood, while missing or contradicting any primary subject causes automatic failure.Extra elements are acceptable only when they do not violate, contradict, or omit explicit prompt requirements.Source: §E.4.2 · p. 50
E.5 GSB PAIRWISE PREFERENCE PROTOCOL
- GSB compares each candidate with a fixed same-prompt reference in both image orders, recording Good, Bad, or Same only after restoring candidate identity and requiring order agreement.Tables 3–5 use the zero-shot, single-shot Qwen3.5-397B-A17B Base prompter as reference and evaluate one image per prompt over N = 150; Same contributes zero to net preference.Source: §E.5 · pp. 50–51
- The GSB protocol treats serious violations of widely recognized appearances for famous people, landmarks, or other subjects as factual errors.Source: §E.5 · p. 52
F PROMPTS FOR MAIN-PAPER FIGURES
- The appendix records retained qualitative-example prompts in figure order, marks unavailable originals instead of reconstructing them, and includes a 14-output gallery from the final SP/L10 + RFT-prompter Qwen-Image system.Editing examples begin from base prompts and apply targeted structured-prompt edits, usually changing one field; object moves may update both objects’ position fields.Source: F PROMPTS FOR MAIN-PAPER FIGURES · pp. 52–53
- The teaser compares natural-language and structured prompts at L5/L6/L8/L10 while preserving 39 entities and 11 relationships, with reconstruction similarity remaining nearly flat as captions lengthen.The BAGEL scaling-property sweep uses the L6/L8/L10 natural-language configurations, while L5 provides the sparser visual endpoint.Source: F PROMPTS FOR MAIN-PAPER FIGURES · p. 53
F.9 FIGURES 23–24 — SOTA LLM AS PROMPTER
- The different-LLM-as-prompter comparison uses shared user prompts, including a rabbit racing a turtle and an inventory of one glass, two wine bottles, and three beer cans.Source: §F.9 · p. 55
G.1 PROMPTER COMPARISON: PROMPTER SCALE × TRAINING · G.2 GENERAL-PURPOSE LLMS AS PROMPTERS
The appendix compares prompter scale and training under fixed rendering conditions, then evaluates general-purpose LLMs against the trained Qwen3.5-397B-A17B prompter using the same schema and Qwen-Image backbone.
G.1 PROMPTER COMPARISON: PROMPTER SCALE × TRAINING
- Four variants isolate prompter scale and training effects: Qwen3.5-35B-A3B and Qwen3.5-397B-A17B, each before and after the full pipeline.All variants use the same user prompts, Qwen-Image backbone, and decoding settings; only the prompter changes.Source: §G.1 · p. 55, Figure 20 · p. 56, Figure 21 · p. 57
- Reading within each scale isolates training, while reading across scales isolates prompter size.Rows correspond to shared user prompts, with full prompt text provided in Appendix F.8.Source: §G.1 · p. 55
- Figures 20–22 present the complete four-column comparison across the shared prompt set, with aspect ratio predicted by the prompter.The figures retain the same prompt order across the three panels.Source: Figure 20 · p. 56, Figure 21 · p. 57, Figure 22 · p. 58
G.2 GENERAL-PURPOSE LLMS AS PROMPTERS
- Five prompters fill the same schema: GPT-5.5, Gemini 3 Pro, GLM-5.2, Claude Opus 4.8, and the trained Qwen3.5-397B-A17B prompter.The four general-purpose LLMs operate in single-turn schema-filling mode.Source: §G.2 · p. 59
- Figures 23–24 compare general-purpose LLMs and the trained prompter while rendering with the same Qwen-Image backbone.Rows share the same user prompts, and Table 3 reports quantitative per-backend scores.Source: Figure 23 · p. 60, Figure 24 · p. 61
- The general-purpose-LLM comparison uses a shared five-column layout across two continued figure panels.The panels preserve the same user-prompt rows and rendering setup.Source: Figure 23 · p. 60, Figure 24 · p. 61
Related papers
This paper cites
- Score-Based Generative Modeling through Stochastic Differential Equations2020
- Scaling Laws for Neural Language Models2020
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding2022
- Training Compute-Optimal Large Language Models2022
- SAM 2: Segment Anything in Images and Videos2024
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning2021