Source-linked AI summary
InterleaveThinker: Reinforcing Agentic Interleaved Generation
Dian Zheng, Harry Lee, Manyuan Zhang, Kaituo Feng, Zoey Guo, Ray Zhang, Hongsheng Li
TL;DR
Existing image generators are built for single-image outputs, limiting interleaved text-image generation for sequential applications. InterleaveThinker adds Planner, Generator, and Critic agents with step-wise correction, achieving strong benchmark performance and improving reasoning-based generation.
Problem
Existing image generators cannot natively produce coherent multi-step text-image sequences, despite demand for interleaved generation in visual narratives, guidance, and embodied manipulation.
Method
InterleaveThinker uses a closed-loop Planner-Generator-Critic pipeline with step-wise instruction planning, output evaluation, and prompt refinement for regeneration.
Results
InterleaveThinker significantly outperforms open-source UMMs, matches Nano Banana on UEval, and improves WISE from 0.47 to 0.73 and RISE from 13.3 to 28.9.
Takeaways & Limitations
The framework equips off-the-shelf image generators with strong interleaved generation capabilities while boosting complex reasoning performance.
Takeaways & Limitations
The framework remains constrained by the base model’s generative prior and cannot generate concepts absent from its training corpus.
Abstract
from arXiv · showhide
Recent image generators have demonstrated impressive photorealism and instruction-following capabilities in single-image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text-image sequence), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open-source Unified Multimodal Models (UMMs) exhibit limited performance in this regard. In this paper, we introduce InterleaveThinker, the first multi-agent pipeline designed to endow any existing image generator with interleaved generation capabilities. Specifically, we employ a planner agent to organize the image-text input sequence, instructing the image generator on the required execution at each step. Subsequently, we introduce a critic agent to evaluate the generator's outputs, identify samples that deviate from the planned instructions, and refine the instructions for regeneration. To implement this pipeline, we construct the Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k to perform a format cold-start. Then we develop Interleave-Critic-RL-13k to reinforce the step-wise instruction correction capability within a generation trajectory using GRPO. Since a single interleaved generation trajectory may involve over 25 generator calls, optimizing the entire trajectory is computationally impractical. Therefore, we propose accuracy reward and step-wise reward, allowing single-step RL to effectively guide the entire generation trajectory. The results show that InterleaveThinker improves performance across various image generators. On interleaved generation benchmarks, it achieves performance comparable to Nano Banana and GPT-5. Surprisingly, it also significantly enhances the base model on reasoning-based benchmarks; for example, on 4-step FLUX.2-klein, we observe substantial gains on WISE and RISE.
1 Introduction
InterleaveThinker is introduced as a multi-agent framework that equips fixed image generators with interleaved generation capabilities while addressing visual over-reliance and step-wise error accumulation. Its data pipeline spans diverse scenarios, and experiments show broad gains, including substantial improvements on reasoning benchmarks.
- Motivation: Interleave generation requires coherent multi-step text-and-image sequences, but conventional image generators are fundamentally designed for single-image generation and editing.The paper identifies interleaved generation as increasingly important for real-world applications.
- Motivation: Unified Multimodal Models naturally support interleaved outputs but suffer from visual over-reliance and error accumulation in long-horizon tasks.Their step-by-step generation conditions later outputs on preceding images, creating critical reliability problems.
- Method: InterleaveThinker uses a Planner-Gen-Critic workflow to correct intermediate errors and reduce visual over-reliance when extending fixed image generators.The framework is presented as the first multi-agent approach designed to provide strong interleaved generation capabilities to any fixed image generator.
- Method: The training pipeline produces dedicated interleaved-generation data across embodied manipulation, art, storytelling, workflows, science, and other scenarios.The paper names Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k among the resulting datasets.
- Results: 0.73 WISE and 28.9 RISE are achieved after improvements from 0.47 and 13.3, respectively, demonstrating stronger reasoning-based generation.Across multiple off-the-shelf image generators, the pipeline shows consistent gains and reaches performance comparable to Nano Banana and GPT-5 on interleaved-generation benchmarks.
2 Related Works
Related work spans unified image generators, unified multimodal models (UMMs), and agentic reinforcement learning. Unified image generators combine strong text-to-image and image-editing capabilities, while UMMs natively support interleaved generation but struggle with long-horizon tasks.
- Unified Image Generation: Diffusion and autoregressive advances have improved image-generation photorealism and instruction following, enabling unified models that support both text-to-image generation and image editing.These unified image generators preserve strong text-to-image capabilities while adding robust editing functionality.
- Unified Multimodal Models: UMMs model text and visual tokens in a unified framework, natively supporting interleaved generation but struggling with long-horizon tasks.The passage identifies visual over-reliance on immediately preceding visual states as one fundamental issue.
- Agentic Reinforcement Learning: Agentic reinforcement learning trains language and vision-language models for multi-agent, multi-step reasoning and long-horizon tool interaction, and is being adapted for visual generation.Existing visual-generation efforts include search agents for knowledge-intensive image generation and multi-turn refinement for image generation and editing.
3 InterleaveThinker
InterleaveThinker is a universal multi-agent framework that equips frozen image generators with robust interleaved generation through planned, iterative execution and self-correction. It combines dedicated data construction with SFT format cold-start and GRPO-based Critic reinforcement, addressing long-horizon trajectory costs.
- Framework: InterleaveThinker decomposes interleaved generation into a closed-loop Planner–Generator–Critic pipeline with step-wise execution and self-correction.The framework is designed for existing frozen image generators that support image generation and editing.
- Planner: The Planner converts input sequence S into N step instructions, model-friendly prompts, and auxiliary text for task-specific generation.For each step i, it produces ui, pi, and ai.
- Closed-loop execution: At each step, the Generator uses the refined prompt and previous image, while the Critic evaluates alignment, provides feedback, and refines the prompt.The initial step uses no prior visual context; the generation-evaluation loop repeats until a True judgment or Tmax is reached.
- Closed-loop execution: The pipeline finalizes each accepted image and auxiliary text, appends them to the output sequence, and advances to the next step.This step-wise progression supports interleaved image-text generation across the full input sequence.
- Training: The training pipeline uses SFT for multi-agent format cold-start, then GRPO-based RL to reinforce the Critic’s correction capabilities.Planner-SFT-80k initializes planning, while Critic-SFT teaches visual deviation detection and refined-prompt formulation.
- Training: A single interleaved task may require over 25 generator calls, making end-to-end standard RL computationally prohibitive and difficult for credit assignment.This motivates efficient Critic reinforcement strategies focused on the long-horizon trajectory challenge.
4 Experiments
Experiments show that InterleaveThinker performs strongly on interleaved-generation benchmarks and improves reasoning-based image generation and editing across base models. Ablations attribute these gains to the multi-agent workflow, dual rewards, filtered critic data, and iterative refinement.
- Benchmark Results: InterleaveThinker significantly outperforms existing open-source UMMs on UEval and achieves performance comparable to Nano Banana.Integrating Qwen-Image-Edit yields further gains, indicating model-agnostic and generalizable behavior.
- Benchmark Results: On CoMM, InterleaveThinker surpasses all existing methods with 4-step FLUX.2-klein, while Qwen-Image-Edit-2511 further pushes benchmark performance.CoMM evaluates interleaved input-output performance through task 3.
- Reasoning-Based Evaluation: InterleaveThinker significantly improves base models on both WISE image generation and RISE reasoning-based image editing, despite no explicit training on reasoning-based image generation.These results support the benefit of the plan-generate-critic framework beyond interleaved generation.
- Effectiveness of Multi-Agent Workflow: 33.5 to 58.5: introducing Planner-SFT produces a massive surge in the UEval Text score over the raw FLUX.2-klein-9B generator.The raw generator fails entirely at interleaved generation, while the ablation compares zero-shot and trained multi-agent configurations.
- Reward Ablation: Combining Accuracy Reward and Step-wise Reward yields the best result, whereas removing either reward decreases average performance.Removing Step-wise Reward impairs refined-prompt optimization; removing Accuracy Reward causes inaccurate score evaluation.
- Workflow and Refinement Ablations: Increasing Tmax consistently improves performance over the single-pass baseline, demonstrating the effectiveness of closed-loop refinement.Additional ablations show that combining planning and evaluation in one model severely degrades performance, while unfiltered critic data causes trivial constant predictions and performance drops.
5 Conclusion and Limitations
InterleaveThinker addresses long-horizon interleaved-generation failures with a decoupled Planner–Critic framework and dual-reward single-step RL. It enhances off-the-shelf image generators while remaining constrained by the base generator’s learned generative prior.
- Contributions: InterleaveThinker decouples planning and visual evaluation through a Planner that predicts global instructions upfront and a Critic that performs step-wise evaluation.This design targets visual over-reliance and step-wise error accumulation in existing multimodal models.
- Contributions: A dual-reward strategy enables efficient single-step RL on the Critic to guide the entire generation sequence.The strategy is introduced to reinforce Critic-based instruction correction without optimizing the whole trajectory.
- Results: Extensive experiments show that InterleaveThinker equips off-the-shelf image generators with strong interleaved-generation capabilities, matching proprietary models and boosting complex reasoning performance.The reported improvements span both interleaved generation and complex reasoning performance.
- Limitations: The framework’s capacity remains constrained by the base model’s generative prior, preventing it from generating concepts absent from the base generator’s training corpus.The paper illustrates this limitation with a bad case in Fig 8 in the Appendix.
A System Prompt
The system prompt defines multimodal planning as a continuous sequence in which each visual step builds on or edits the previous one, with modality-specific routing and strict output formatting. A paired evaluator checks intent matching and visual quality, then refines prompts when outputs fail.
- Planner Pure-Text System Prompt: The pure-text planner requires every execution-plan step to represent an image generation or editing action and converts every step instruction into an optimized prompt.It explicitly excludes separate steps used only for captions, summaries, or other text.
- Planner Interleaved System Prompt: The planner treats multimodal requests as continuous step-by-step processes, organizing visual actions so each step builds upon or edits the previous one.It is framed as a Multimodal Sequence Planner and Orchestrator.
- Planner Interleaved System Prompt: Task identification routes complete image-to-text responses into auxiliary_text, while sequence continuation generates both text instructions and image-editing prompts.The routing distinguishes general text response or problem-solving tasks from sequential editing tasks.
- Generation/Edit Evaluation and Prompt Refinement System: The evaluator returns True or False after comparing before and after images for intent matching and visual quality, and rewrites the prompt when either criterion fails.The comparison uses a blank canvas for initial generation and checks the resulting change against the intended instruction.
- Generation/Edit Evaluation and Prompt Refinement System: Prompt refinement diagnoses gaps between the original instruction, rewritten prompt, and generated result, then adds concrete visual details or resolves contradictions as needed.For initial generation, refinement can specify subject, environment, lighting, camera angle, composition, and art style.
B Bad Cases
FLUX.2-klein remains unable to handle concepts unknown to its frozen image generator, and may exhibit color shifts in such out-of-domain cases.
- B Bad Cases: Unknown concepts remain uncorrectable when the frozen image generator lacks them.The framework could not fix these cases.
- B Bad Cases: Color shifts can occur in these out-of-domain failures.This artifact is reported for concepts the generator does not know.
- B Bad Cases: The reported color-shift failure does not occur in in-domain situations.The passage contrasts these failures with in-domain behavior.