Source-linked AI summary
SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-to-image Generation
Yufei Jiao, Jinfa Huang, Zeyue Xue, Tianyu Xie, Yuexiao Ma, Rongrong Ji, Xiawu Zheng, Jiebo Luo
TL;DR
Prompt-rewriting RL for text-to-image generation uses image-level feedback, although compositional edits form semantic spans that do not necessarily align with subword tokens. SCOPE pools policy shifts within those spans to weight token updates while retaining one advantage per completion. Across three frozen generators, it reports average relative gains over vanilla GRPO of 9.48% on GenEval and 7.62% on T2I-CompBench.
Problem
Image-level rewards provide no direct attribution to spans or tokens, while compositional edits form coherent semantic spans that may cross subword boundaries.
Source: §1 · p. 1Method
SCOPE jointly generates a rewritten prompt and semantic partition, pooling policy log-probability shifts within each span to reweight token losses under one completion advantage.
Source: §1 · p. 2Results
SCOPE achieves average relative gains over vanilla GRPO of 9.48% on GenEval and 7.62% on T2I-CompBench across Qwen-Image, SD3, and FLUX.1-dev.
Source: §5 · p. 9Takeaways & Limitations
Ablations favor both pooling policy shifts within semantic spans and semantic boundaries over token-level weighting or random and fixed-window boundaries.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Reinforcement learning (RL) enables language models to improve text-to-image (T2I) generation by rewriting prompts for a frozen image generator. Existing methods typically apply a single signal derived from the image to individual subword tokens, although objects, counts, attributes, and relations form coherent semantic spans. We propose SCOPE, an objective based on Group Relative Policy Optimization (GRPO) that aligns update weighting with this structure. The rewriter jointly generates an enhanced prompt and its semantic partition. SCOPE pools shifts in policy log probabilities within each span to derive a shared coefficient for its token losses while retaining a single advantage for each completion. It requires neither separate span rewards nor additional image renders. Across Qwen-Image, SD3, and FLUX.1-dev, SCOPE achieves average relative gains over vanilla GRPO of 9.48% on GeneVal and 7.62% on T2I-CompBench. Ablation studies show that both semantic boundaries and pooling within spans contribute to these gains, supporting semantic structure as a practical inductive bias for prompt engineering with RL.
1 Introduction
Prompt rewriting can improve text-to-image composition while keeping the generator fixed, but image-level feedback is poorly matched to semantic edits that cross subword boundaries. SCOPE pools policy shifts within semantic spans to weight token updates, and outperforms vanilla GRPO across three backbones and two benchmarks.
1 Introduction
- Prompt adaptation offers a cheaper alternative to retraining or adapting the visual decoder by improving the language interface while keeping the image generator fixed.Small wording changes can affect object counts, color binding, and spatial relations.Source: §1 · p. 1
- Compositional edits form coherent phrases that can span subword tokens, but image rewards evaluate the composition holistically without attributing credit to spans or tokens.Uniform token updates can fragment these edits across tokenizer boundaries, while dense per-span rewards require costly additional rendering.Source: §1 · pp. 1–2
- SCOPE jointly outputs a rewritten prompt and semantic partition, then pools policy log-probability shifts within each span to reweight token losses using one completion advantage.It uses the existing rendering and reward pipeline without separate span rewards, additional image renders, or a learned critic.Source: §1 · p. 2, Figure 1 · p. 2
- SCOPE consistently outperforms vanilla GRPO on GenEval [13] and T2I-CompBench across Qwen-Image, SD3, and FLUX.1-dev.Ablations favor pooling shifts within semantic spans and semantic boundaries over token-level weighting, random boundaries, or fixed windows.Source: §1 · p. 2
2 Related Work
Related work covers finer-grained credit assignment for sequence generation, prompt optimization for text-to-image systems, and reinforcement learning directly on visual generators. SCOPE instead optimizes the prompt-rewriting policy while keeping the renderer frozen.
Credit Assignment for Sequence Generation
- Sequence-generation methods explore eligibility traces, counterfactual evaluation, tree-based estimates, segment-level advantages, and policy-divergence weighting [9] [14] [22] [25] [33] [35] [37] [44] [48] [51].SCOPE retains one completion advantage and modulates token contributions with a shared coefficient for each semantic span.Source: §2 · p. 3
Prompt Optimization for T2I Generation
- Prompt-optimization work includes search and iterative refinement, learned prompt adaptation, Promptist’s supervised initialization and image-reward RL [15], and GRPO-based RePrompt [50] [55] [61].SCOPE addresses the optimization objective by incorporating semantic structure into token weighting while retaining a shared advantage for the complete rewrite.Source: §2 · p. 3
3 Methodology
SCOPE embeds semantic-span weighting in a GRPO prompt-rewriting pipeline while keeping image feedback completion-level. It uses pooled policy shifts to redistribute token losses across spans.
3.1 Grouped Rewards and Advantages
- Grouped rollouts append the original prompt as a reward reference and share rendering seeds; rewards combine image alignment with structured-output validity.When the original-prompt reward is below τ, the advantage measures improvement over that prompt; otherwise, sampled rewrites are standardized within the group.Source: §3.1 · pp. 3–4
3.2 Policy Optimization with GRPO
- GRPO uses a frozen old-policy ratio and applies each completion's shared advantage through clipped losses at valid token positions.Because ratios vary by token, positions describing one object or relation still contribute separately under the baseline objective.Source: §3.2 · pp. 4–5
3.3 Semantic Span Credit Modulation
- Generated semantic spans align to disjoint answer-token sets, whose mean log-ratio supplies a signal without direct scaling by token count.Source: §3.3 · p. 5
- Exponentiated span means become clipped, normalized, detached weights that redistribute shared advantage without changing fixed-ratio clipping decisions.Normalization keeps the average answer-content coefficient at one; coefficient computation adds one πold forward but no extra image renders.Source: §3.3 · pp. 5–6
- Table 1 reports that SCOPE has the highest overall GenEval score on each frozen backbone.Source: Table 1 · p. 6
4 Experiments
Experiments compare SCOPE with GRPO and other baselines on compositional prompt benchmarks using three frozen T2I backbones. Results, ablations, and qualitative examples examine performance and the role of semantic-span credit weighting.
4.1 Experimental Setup
- Evaluation uses GenEval and T2I-CompBench, with disjoint native evaluation sets; training uses Qwen3-8B as rewriter and frozen Qwen-Image, SD3, and FLUX.1-dev generators.GRPO with uniform token weights is the primary baseline and shares SCOPE’s reward pipeline and training setup.Source: §4.1 · p. 6
4.2 Quantitative Analysis
- 6.7%, 8.0%, and 8.2% are SCOPE’s average T2I-CompBench gains over GRPO on Qwen-Image, SD3, and FLUX.1-dev, respectively.SCOPE exceeds GRPO in all six reported categories on each backbone.Source: Table 2 · p. 7, §4.2 · p. 7
- 0.91 to 0.95 (+4.4%), 0.74 to 0.84 (+13.5%), and 0.76 to 0.84 (+10.5%) are SCOPE’s GenEval overall scores versus GRPO on Qwen-Image, SD3, and FLUX.1-dev.SCOPE improves on GRPO in 17 of 18 category benchmarks across the three backbones.Source: §4.2 · p. 7
4.3 Ablation and Mechanistic Analysis
- Unpooled token-level shifts degrade performance below GRPO, while SCOPE scores 0.95, 0.84, and 0.84 versus token-local control scores of 0.90, 0.75, and 0.76.The figure caption reports that SCOPE consistently outperforms uniform GRPO across backbones.Source: §4.3 · pp. 7–8, Figure 3 · p. 8
- On Qwen-Image, semantic boundaries score 0.95 versus 0.86 for random partitions and 0.87 for fixed 3-token windows.The reported comparison favors linguistic boundary alignment over arbitrary groupings.Source: §4.3 · p. 8
- On FLUX.1-dev, ∆bg scores 0.827, z-norm 0.832, and the hybrid rule 0.841, all above vanilla GRPO’s 0.760; SCOPE’s smoothed Qwen-Image training trajectory also stays above GRPO’s.The hybrid rule anchors updates to the original prompt when initial quality is low, then switches to group discrimination.Source: §4.3 · p. 8
4.4 Qualitative Analysis
- Across four prompts, SCOPE’s rewrites make layout, count, and counterfactual color constraints explicit; its Qwen-Image output matches gpt-image 2 on layout and counting and renders a white orange.The gpt-image 2 output falls back to an orange-colored prior for the counterfactual attribute.Source: §4.4 · p. 9
5 Conclusion
SCOPE applies semantic-span credit optimization to prompt engineering for frozen T2I generators. Across three backbones, it improves over vanilla GRPO on both reported benchmarks.
- SCOPE pools policy log-probability shifts within coherent semantic spans to modulate a shared completion advantage for prompt rewriting with frozen T2I generators.The method requires no separate span rewards, additional image renders, or external parsers.Source: §5 · p. 9
- 9.48% on GenEval and 7.62% on T2I-CompBench are SCOPE’s average relative gains over vanilla GRPO across Qwen-Image, SD3, and FLUX.1-dev.Source: §5 · p. 9
- SCOPE presents structure-aligned credit assignment as a plug-and-play approach for prompt engineering with frozen visual generators.Source: §5 · p. 9
A.1 Implementation overview
The implementation distributes rewriter training, rendering, and reward evaluation across workers while keeping the T2I backbones frozen.
- Distributed policy training optimizes the Qwen3-8B rewriter, while separate reward and rendering workers serve frozen T2I backbones.Source: §A.1 · p. 13
- Qwen3-8B generates semantic spans during rollout without a separate span model or parser service.Source: §A.1 · p. 13
- Worker layout and hardware placement do not change the objective or evaluation protocol.Source: §A.1 · p. 13
A.2 Hyperparameters
Table 3 covers optimization settings for prompt rewriting, span weighting, and normalization; reproducing the tables requires the reported configuration and protocol.
- Table 3 lists the optimization configuration for prompt rewriting, span weighting, and normalization.Source: Table 3 · p. 13
- Local launcher defaults can differ from the configuration used for the reported runs.Source: §A.2 · p. 13
- Exact reproduction of the tables requires the reported configuration together with the same data and sampling protocol.Source: §A.2 · p. 13
A.3 Rendering configuration
Table 4 specifies the fixed rendering configuration used for training rewards and benchmark evaluation.
A.3 Rendering configuration
- Table 4 specifies the fixed rendering configuration used for both training rewards and benchmark evaluation.The supplied passages do not provide the configuration values.Source: §A.3 · p. 13, Table 4 · p. 13
A.4 Span Extraction and Fallbacks
Span boundaries are mapped onto rollout tokens, with uniform GRPO weighting used when partitions fail validation and a special handling for one benign EOS mismatch.
A.4 Span Extraction and Fallbacks
- The rewrite and its semantic boundaries are generated in one completion, then character offsets align extracted spans with rollout tokens.The instruction preserves the request while clarifying subjects, counts, colors, attributes, and spatial relations.Source: §A.4 · p. 13
- Malformed wrappers, empty or missing spans, separator errors, truncation, incomplete coverage, alignment failures, and token-count mismatches trigger uniform GRPO token weights.A valid partition requires nonempty spans, full coverage of valid answer tokens, and consistent offsets and token counts.Source: §A.4 · p. 13
- One extra trailing placeholder is treated as a benign EOS mismatch: the aligned prefix is retained and the extra token receives zero weight.Span statistics are computed from the aligned prefix.Source: §A.4 · p. 13
- Table 5 reports fallback counts and rates across benchmarks and frozen image backbones.The supplied passages identify the table's scope but do not provide its values.Source: §A.4 · p. 13, Table 5 · p. 14
A.5 Z-Image Rewriter
The appendix provides Z-Image rewriter score tables and implementation details covering compute, span weighting, reward handling, reproduction, training, and qualitative comparisons.
A.5 Z-Image Rewriter
- Tables 6 and 7 report Z-Image rewriter scores by frozen backbone for GenEval and T2I-CompBench, respectively, as subsets of the main tables.The accompanying passage says the tables use the evaluation protocol in Section 4 and the same rendering configuration as the corresponding main-table rows.Source: Table 6 · p. 14, Table 7 · p. 14, §A.5 · p. 14
A.6 Compute
- Policy optimization and auxiliary jobs ran on NVIDIA H200 GPUs; exact node counts depend on deployment notes, and Table 4 does not prescribe cluster configuration.Table 4 specifies sampling for each backbone.Source: §A.6 · p. 14
A.7 Environment variables and span weighting
- Span weighting reuses current- and old-policy log probabilities from the clipped surrogate, with sensitivity β distinct from the KL coefficient βKL.Reward-service endpoints, timeouts, generation count, and reward and advantage weights are exposed as environment settings.Source: §A.7 · p. 14
A.8 Rewrite instruction
- The trainer removes span delimiters and concatenates span text into a natural-language prompt before rendering, while the format reward evaluates the structured output.The system message requests an answer containing spans separated by |.Source: §A.8 · p. 14
A.9 Reward notes
- Training groups contain eight prompt members sharing one random diffusion seed, while groups and evaluation use fresh seeds across prompts and outer loops.The hybrid advantage rule and shared seeds require no separate reward model or estimation module.Source: §A.9 · p. 14
A.10 RePrompt: Qwen-Image reproduction
- The Qwen-Image RePrompt† entries are reproductions of Wu et al.'s [55] recipe, while SD3 and FLUX.1-dev entries reproduce scores reported by Wu et al. [55].The Qwen-Image runs use the same reward clients, rendering seeds, and benchmark harness as the GRPO and SCOPE runs.Source: §A.10 · pp. 14–15
A.11 Training procedure
- The training procedure keeps the renderer and evaluator frozen, initializes the old policy from the current policy on the first step, and allows an optional KL penalty against a separate frozen reference.The procedure optimizes Eqs. 1–12; span fallbacks are described in Section A.4.Source: §A.11 · p. 15
A.12 Additional Qualitative Comparisons
- Figure 5 presents additional comparisons using Figure 4's five columns, matched seeds within each row, and rows grouped by prompt type and evaluation setting.The dashed frame encloses rewrite snippets and SCOPE outputs.Source: §A.12 · p. 16, Figure 5 · p. 16
Related papers
This paper cites
- Classifier-Free Diffusion Guidance2022
- Hierarchical Text-Conditional Image Generation with CLIP Latents2022
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding2022
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow2022
- Elucidating the Design Space of Diffusion-Based Generative Models2022
- GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment2023