Source-linked AI summary
Iterative Refinement Improves Compositional Image Generation
Shantanu Jaiswal, Mihir Prabhudesai, Nikash Bhardwaj, Zheyang Qin, Amir Zadeh, Chuan Li, Katerina Fragkiadaki, Deepak Pathak
TL;DR
Text-to-image models struggle with complex prompts, and parallel sampling cannot revise earlier outputs. This paper iteratively refines images using vision-language-model feedback, achieving state-of-the-art performance across compositional benchmarks and improving ConceptMix all-correct rate by 16.9% at concept binding=7 versus compute-matched parallel sampling.
Problem
Text-to-image models lack structured reasoning traces and rely on one-shot generation, while parallel sampling cannot revise earlier outputs and struggles with complex compositional prompts.
Source: §1 · p. 2Method
The framework iteratively refines generated images using a vision-language-model critic to propose corrections, an editor to apply them, and a verifier to assess prompt alignment.
Source: §1 · p. 2Results
Takeaways & Limitations
Sequential corrections let the model handle subsets of bindings at each step and compound previously resolved components over time.
Source: §1 · p. 2The verifier is a lightweight VLM for automatic test-time guidance, not an oracle or benchmark evaluator.
Source: §3 · p. 4
Abstract
from arXiv · showhide
Text-to-image (T2I) models have achieved remarkable progress, yet they continue to struggle with complex prompts that require simultaneously handling multiple objects, relations, and attributes. Existing inference-time strategies, such as parallel sampling with verifiers or simply increasing denoising steps, can improve prompt alignment but remain inadequate for richly compositional settings where many constraints must be satisfied. Inspired by the success of chain-of-thought reasoning in large language models, we propose an iterative test-time strategy in which a T2I model progressively refines its generations across multiple steps, guided by feedback from a vision-language model as the critic in the loop. Our approach is simple, requires no external tools or priors, and can be flexibly applied to a wide range of image generators and vision-language models. Empirically, we demonstrate consistent gains on image generation across benchmarks: a 16.9% improvement in all-correct rate on ConceptMix (k=7), a 13.8% improvement on T2I-CompBench (3D-Spatial category) and a 12.5% improvement on Visual Jenga scene decomposition compared to compute-matched parallel sampling. Beyond quantitative gains, iterative refinement produces more faithful generations by decomposing complex prompts into sequential corrections, with human evaluators preferring our method 58.7% of the time over 41.3% for the parallel baseline. Together, these findings highlight iterative self-correction as a broadly applicable principle for compositional image generation. Results and visualizations are available at https://iterative-img-gen.github.io/
1. Introduction
The introduction frames complex compositional prompts as a challenge for one-shot text-to-image generation and proposes iterative refinement guided by a vision-language critic. It reports gains over compute-matched parallel sampling and argues that self-correction can transfer to generative vision models.
1. Introduction
- T2I training data consists of image-caption pairs without structured reasoning traces, unlike LLM text pretraining, and the paper links this difference to one-shot generation strategies.The introduction says these models do not inherently develop self-correction or iterative refinement.Source: §1 · p. 2
- The proposed pipeline pairs a text-to-image generator with a VLM critic, an image editor, and a verifier to iteratively improve outputs against the prompt.The critic proposes corrections by comparing the generated image with the target prompt; the editor applies suggested edits.Source: §1 · p. 2
- Parallel sampling selects among independent images but cannot revise earlier outputs, so pass@k may remain near zero when one pass cannot resolve dozens of concept bindings.Iterative refinement reuses intermediate generations and handles subsets of bindings across steps.Source: §1 · p. 2
- 16.9% higher all-correct rate on ConceptMix (concept binding=7) and 13.8% gain on T2I-Bench 3D Spatial came against compute-matched parallel sampling.The introduction also reports that parallel sampling failed on the illustrated prompt after four passes, while iterative refinement succeeded at the same compute.Source: §1 · p. 2
- The method uses VLM critic feedback with standard image editing rather than heavy auxiliary tool stacks or model-specific engineering pipelines.The introduction contrasts this with tool-based approaches whose components can lag behind foundation models and accumulate errors.Source: §1 · pp. 2–3
- The introduction presents self-correction as an inductive principle for generative vision models and points to systems that critique and improve their outputs.It describes this as a possible step toward a more unified view of reasoning across modalities.Source: §1 · p. 3
2. Related Work
Prior work improves text-to-image alignment through inference-time guidance, iterative feedback, and compositional methods, but often relies on specialized tools, priors, or training objectives. This paper applies a chain-of-thought-like feedback loop using a VLM critic for targeted refinement.
2. Related Work
- Classifier-free guidance, parallel sampling, and grounding-based methods can improve prompt fidelity, but often do not scale to richly compositional prompts.Prior iterative approaches include SDEdit, InstructPix2Pix, and IterComp, while human-preference-guided evaluation highlights adaptive inference-time guidance.Source: §2 · p. 3
- Related compositional methods use tool-calling, regional generation priors, or reinforcement learning, whereas this method is training-free and uses a VLM critic with generation and editing.The paper reports stronger performance benefits across different T2I model families.Source: §2 · p. 3
- The method adapts iterative reasoning and self-refinement ideas to image generation: the critic evaluates candidates and issues targeted refinement prompts.The passage draws on chain-of-thought work [27] [30] [36] and iterative self-refinement [21].Source: §2 · p. 3
3. Method
The method iteratively refines generated images using critic feedback, with a verifier assessing prompt alignment. It distributes a fixed inference budget across refinement rounds and parallel streams.
3. Method
- Each stream starts with a generator-produced image, then a verifier scores alignment and a critic proposes an action and sub-prompt for the next update.The editor applies the sub-prompt to the selected image.Source: §3 · p. 3
- The critic can stop, backtrack to an earlier image, restart from the original prompt, or continue editing the current best candidate.Each action determines how the proposed refinement sub-prompt is applied.Source: §3 · p. 3
- The fixed budget B is allocated as T refinement rounds across M parallel streams, exposing a depth–breadth trade-off.Each budget unit corresponds to one call to the generator or image editor.Source: §3 · p. 3
- Refinement ends when every stream emits STOP or the budget is exhausted.The verifier is a lightweight VLM for test-time guidance, not an oracle or benchmark evaluator.Source: §3 · pp. 3–4
- Sequential refinement decomposes compositional prompts into steps, while the verifier checks alignment and the critic supplies targeted corrections.This process is described as analogous to chain-of-thought reasoning in language models.Source: §3 · pp. 3–4
4. Experiments
Experiments compare iterative refinement and mixed iterative-parallel sampling with compute-matched parallel baselines across compositional image-generation benchmarks. Results show gains on complex compositions and spatial reasoning, human preference for refined images, improved Visual Jenga solve rates, and benefits from allocating most compute to iteration.
4. Experiments
- Experiments use Qwen-Image, Gemini 2.5 Flash Image, and GPT-Image across ConceptMix, T2I-CompBench, and TIIF-Bench, with Visual Jenga as an additional evaluation.A multimodal language model evaluates prompt-image consistency; the in-loop critic and verifier are weaker than the final benchmark evaluator.Source: §4 · p. 4, §4.1 · pp. 4–5
4.1. Compositional Image Generation
- At ConceptMix k=5/6/7, Qwen-Image improves 18.8%/21.6%/16.9%; on T2I-CompBench, gains are 7.1% Spatial, 13.8% 3D-spatial, and 6.3% Numeracy.Both iterative and iterative-plus-parallel variants outperform parallel sampling, with the largest ConceptMix gains on complex compositions.Source: Table 1 · p. 5, §4.1 · p. 5
- On TIIF-Bench, Qwen-Iter+Par achieves state-of-the-art results among open-source methods, improving over Qwen-Parallel by 5.0% on Basic Reasoning, 2.7% on advanced Relation+Reasoning, and 4.0% on text rendering.The reported results cover short descriptions; full long-description results are in the supplement.Source: §4.1 · p. 6, Table 2 · p. 6
4.2. Qualitative analysis and human evaluation
- Across ConceptMix and T2I-CompBench, human raters preferred the method 58.7% of the time versus 41.3% for parallel sampling.Examples show Continue, Restart, and Backtrack actions correcting object placement, style, and pose errors over refinement steps.Source: §4.2 · pp. 6–7, Figure 6 · p. 7
4.3. Visual Jenga Scene Decomposition
- On 56 Visual Jenga scenes, iterative feedback raised full decomposition solve rate from 64.29% with parallel sampling to 76.79%.A proposer suggests an object-removal phrase, an editor generates the next scene, and a critic checks for removal errors and other violations.Source: §4.3 · pp. 7–8, Table 3 · p. 8
4.4. Ablations
- At B=16, I=8, P=2 achieved 69.6% on ConceptMix and 92.6% on T2I-CompBench, versus 69.2% and 92.1% for I=16, P=1.Across larger budgets, iterative allocation outperformed parallel allocation; the authors suggest diminishing returns from additional refinements may explain the benefit of a small parallel component.Source: Table 4 · p. 8, §4.4 · p. 9, Figure 9 · p. 9
- Gemini-Pro as critic improved performance by about 4% over Gemini-2.5-Flash, while Qwen3-VL-32B reduced it by about 3%; the full action space performed best.Both Fresh Start and Backtrack contributed to the full action-space result.Source: §4.4 · p. 9, Table 5 · p. 9, Table 6 · p. 9
5. Conclusion
The paper presents iterative refinement as a broadly applicable inference-time strategy for improving compositional image generation, reporting strong performance across benchmarks and tasks.
- The method combines image generation and editing with an in-loop VLM critic, and outperforms traditional inference-time scaling on compositional benchmarks and Visual Jenga scene decomposition.The paper also studies qualitative examples, human evaluation, compute allocation, critic choice, and action-space ablations.Source: §5 · p. 9
Appendix
A qualitative example shows the model correcting a chicken that initially appears glass-shaped over successive steps.
- The chicken example illustrates correction of a glass-shaped appearance across refinement steps.Source: Figure 10 · p. 12
6. Further qualitative examples.
The qualitative examples include comparisons across model families and illustrate failure modes associated with the VLM verifier and image editor.
- Interactive examples compare iterative refinement with the best of 16 parallel samples across Qwen-Image, GPT, and NanoBanana.Source: §6 · p. 12
7. Further experiment details
Additional experiments examine results across models and prompt complexity, human evaluation, compute allocation, implementation choices, and illustrative failure cases.
7. Further experiment details
- Illustrated failure cases include a verifier missing that a giraffe is not heart-shaped and an editor failing to place a carrot in front of a pine tree.Source: Figure 11 · p. 12, Figure 12 · p. 12
- The experiments used Qwen-Image, GPT-Image-One, and NanoBanana, with Gemini-2.5-Flash-Lite as the primary in-loop critic and verifier; the authors say they will open-source the codebase.Comparative baselines included IterComp, RPG, and GenArtist.Source: §7 · pp. 12–14
- The editing prompt permits CONTINUE, BACKTRACK, FRESH START, or STOP, and asks the model to check prior edits, missing elements, and available image space within a four-step limit.The example includes verifier scores for prompt elements and a cumulative mean binary score of 0.857.Source: §7 · p. 13
7.1. Human evaluation details
- The human-evaluation interfaces present paired images for preference judgments and yes/no questions for assessing prompt concepts, attributes, and relations.Source: Figure 13 · p. 14, Figure 14 · p. 14
- The human study used 150 ConceptMix images from k=5–7, generated with a 16-step budget, and collected responses from 75 participants.Participants answered prompt-specific yes/no questions and indicated which image they preferred.Source: §7.1 · p. 14
- In direct human comparisons, 8 iterative steps yielded higher perfect agreement than one iterative step (37.3% vs. 33.3%); preference for the method was strongest on k=6 prompts (66.7%).Net agreement was 86.1% vs. 84.8%; Table 8 compares one and eight iterative steps.Source: §7.1 · p. 14, Table 8 · p. 15
7.2. Additional experiment specifications
- Iterative refinement consistently outperforms parallel-only sampling across three models and ConceptMix prompt complexities, with Qwen-Image gains exceeding 15% in solve rate at k=5–7.Figure 15 presents the ConceptMix k=1–7 comparison.Source: §7.2 · p. 14, Figure 15 · p. 15
- With budget B = 16, an 8-iterative-to-2-parallel allocation generally outperformed purely parallel or purely iterative strategies on T2I-Bench, using 30 prompts per category.Figure 16 reports the compute-allocation comparison.Source: Figure 16 · p. 15, §7.2 · p. 15