Source-linked AI summary
GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning
Kaixun Jiang, Yuzheng Wang, Junjie Zhou, Pandeng Li, Zhihang Liu, Chen-Wei Xie, Zhaoyu Chen, Yun Zheng, Wenqiang Zhang
TL;DR
Existing multimodal generation systems face training-cost and understanding-generation trade-offs, while modular pipelines lack autonomous adaptation. GenAgent decouples understanding from generation through iterative tool interaction and trains this behavior with supervised fine-tuning and agentic reinforcement learning. It improves FLUX.1-dev performance on GenEval++ and WISE and exhibits cross-tool generalization, test-time scaling, and task-adaptive reasoning.
Problem
Existing integrated models face understanding-generation trade-offs and expensive training, while modular systems use static workflows that cannot autonomously adapt.
Method
GenAgent uses a multimodal model for understanding and external image generators as tools, trained with SFT followed by RL using pointwise and pairwise rewards plus trajectory resampling.
Results
+23.6% on GenEval++ and +14% on WISE are achieved over FLUX.1-dev, alongside cross-tool generalization, test-time scaling, and task-adaptive reasoning.
Takeaways & Limitations
GenAgent provides a flexible alternative to unified architectures while enhancing traditional diffusion-model generation through adaptive multi-turn interactions.
Abstract
from arXiv · showhide
We introduce GenAgent, unifying visual understanding and generation through an agentic multimodal model. Unlike unified models that face expensive training costs and understanding-generation trade-offs, GenAgent decouples these capabilities through an agentic framework: understanding is handled by the multimodal model itself, while generation is achieved by treating image generation models as invokable tools. Crucially, unlike existing modular systems constrained by static pipelines, this design enables autonomous multi-turn interactions where the agent generates multimodal chains-of-thought encompassing reasoning, tool invocation, judgment, and reflection to iteratively refine outputs. We employ a two-stage training strategy: first, cold-start with supervised fine-tuning on high-quality tool invocation and reflection data to bootstrap agent behaviors; second, end-to-end agentic reinforcement learning combining pointwise rewards (final image quality) and pairwise rewards (reflection accuracy), with trajectory resampling for enhanced multi-turn exploration. GenAgent significantly boosts base generator(FLUX.1-dev) performance on GenEval++ (+23.6\%) and WISE (+14\%). Beyond performance gains, our framework demonstrates three key properties: 1) cross-tool generalization to generators with varying capabilities, 2) test-time scaling with consistent improvements across interaction rounds, and 3) task-adaptive reasoning that automatically adjusts to different tasks. Our code will be available at \href{https://github.com/deep-kaixun/GenAgent}{this url}.
1. Introduction
GenAgent addresses the trade-offs and rigidity of existing multimodal generation systems with an agentic framework for iterative reasoning, tool use, and reflection. It combines two-stage training with strong benchmark gains and adaptive behavior across tools and interaction rounds.
- Motivation: Integrated models face understanding-generation trade-offs and substantial training costs, while modular systems are limited by static workflows and weak adaptation.These limitations motivate a framework that separates reasoning from rendering while enabling autonomous decisions.
- Framework: GenAgent enables autonomous multi-turn reasoning by planning actions, invoking image generators, evaluating outputs, and reflecting on whether to continue.Its multimodal reasoning traces combine reasoning, tool invocation, judgment, and reflection.
- Training: The two-stage training framework uses hint-guided SFT for tool invocation and reflection, followed by agentic RL with outcome and process rewards plus trajectory resampling.Pointwise rewards assess final image quality, while pairwise rewards encourage accurate reflections across consecutive rounds.
- Properties: The framework exhibits cross-tool generalization, consistent test-time scaling across interaction rounds, and task-adaptive reasoning.These properties are reported as emergent behaviors of the agentic framework.
2. Related Work
Prior work studies integrated and modular approaches to multimodal generation, while agentic multimodal reasoning has mainly advanced visual understanding. GenAgent extends dynamic tool-based reasoning to multimodal generation.
- Integrated approaches: Integrated approaches combine stronger understanding modules or unified architectures but face understanding-generation trade-offs and substantial training demands.These systems target complex multimodal reasoning through tightly coupled designs.
- Agentic reasoning: Agentic multimodal reasoning has used external tools to refine visual understanding, ranging from single-function tools to Python-based image manipulation.The cited progression concerns reasoning support for visual understanding rather than multimodal generation.
- GenAgent: GenAgent applies iterative thinking, generation, judgment, and reflection to multimodal generation through dynamic interaction with an image-generation tool.Its pipeline repeatedly invokes the tool until requirements are met or the maximum interaction rounds are reached.
3. Method
GenAgent uses an iterative multimodal agent to invoke image-generation tools, evaluate outputs, and refine them across interaction rounds. Its training combines cold-start supervised fine-tuning with agentic reinforcement learning using outcome and process rewards.
- Overall Pipeline: GenAgent iteratively invokes an image-generation tool, evaluates each image against user requirements, and either terminates or produces a refined prompt for another round.A trajectory continues until the output satisfies the query or reaches the maximum interaction rounds.
- Overall Pipeline: Each trajectory records the query, reasoning traces, prompts, generated images, judgments, and termination action, with the final generated image serving as output.The actual number of rounds n satisfies n <= nmax.
- Supervised Fine-Tuning: Cold-start training uses curated multi-turn tool-invocation and reflection data, yielding 32K high-quality samples after filtering and balanced sampling.The pipeline constructs two-round trajectories and filters logical inconsistencies before sampling.
- Supervised Fine-Tuning: SFT reduces tool-invocation errors, produces longer and more detailed refined prompts, and improves reflection quality.The authors describe these changes as addressing the base model’s diagnosed limitations.
- Agentic Reinforcement Learning: Agentic reinforcement learning uses GRPO over sampled trajectories while masking environment observations and optimizing tokens generated by the policy.The advantage is computed from trajectory rewards, and image observations are excluded from the policy loss.
- Agentic Reinforcement Learning: The hybrid reward combines pointwise final-image quality assessment with pairwise rewards that encourage successive images to improve, while interaction-round resampling supports varied trajectory lengths.The pointwise evaluator verifies all specified image conditions, and pairwise judgments compare consecutive images with shuffled positions to reduce positional bias.
4. Experiments
GenAgent is evaluated across instruction-following, knowledge-grounded reasoning, and creative generation, with ablations examining training stages, tools, interaction rounds, and reasoning behavior. Results show gains from agentic training and iterative interaction, while performance eventually meets the underlying generator’s capability ceiling.
- Evaluation Setup: GenAgent is evaluated on GenEval++, WISE, and Imagine, covering instruction-following, knowledge-grounded reasoning, and creative generation.The experiments compare diffusion, unified, and decoupled approaches, using FLUX.1-dev as the default image-generation tool.
- Main Results: With Qwen-Image, GenAgent reaches 0.725 on GenEval++, 0.72 on WISE, and 7.794 on Imagine, approaching GPT-4o on these tasks.The corresponding GPT-4o values are 0.739, 0.80, and 8.560.
- Main Results: GenAgent outperforms existing decoupled approaches, with FLUX-based results of 0.561 on GenEval++ versus ReflectionFlow’s 0.361.PromptEnhancer reaches 0.56 on WISE, 0.382 on GenEval++, and 6.281 on Imagine, while GenAgent’s single-rewrite variant still outperforms it across benchmarks.
- Training Ablation: SFT improves GenEval++ and Imagine but yields limited WISE gains, whereas RL improves all metrics and reflection rewards produce the strongest overall model.The ablation attributes stronger downstream performance to more effective reflection.
- Interaction Ablation: The first interaction round provides substantial gains, later rounds add more modest improvements, and two rounds offer the best balance between performance and efficiency.Further refinement is constrained by the underlying image generator’s limitations in complex semantics and fine-grained attributes.
- Case Studies: Case studies show fact-checking and correction, targeted editing, creative scene construction, visual mathematical reasoning, and an over-reflection failure mode.For near-correct outputs, GenAgent edits missing details rather than regenerating the image; ambiguous prompts can cause cycles through valid interpretations.
5. Conlcusion
GenAgent unifies visual understanding and generation through active tool invocation, using a two-stage training pipeline for iterative refinement. The framework demonstrates task-adaptive reasoning and cross-tool generalization while enhancing diffusion-model generation performance.
- GenAgent decouples understanding and generation, handling understanding within the multimodal model and generation through external models as tools.
- Its two-stage pipeline combines supervised fine-tuning on tool invocation and reflection data with reinforcement learning using pointwise and pairwise rewards.The rewards support iterative refinement and complex multi-turn interactions.
- The framework exhibits task-adaptive reasoning, cross-tool generalization, and reinforcement-learning-enabled multi-turn interactions.
- GenAgent significantly enhances traditional diffusion-model generation performance, achieving results comparable to powerful unified models while maintaining greater flexibility and lower training costs.