GEMS: Agent-Native Multimodal Generation with Memory and Skills

• General-purpose multimodal models perform strongly on straightforward prompts but struggle with intricate instructions and specialized downstream applications.Source: §1 · p. 2
• Existing iterative and multi-agent systems can provide insufficient guidance or excessive redundancy by relying on single-step updates or accumulated historical context.Source: §1 · p. 2
• GEMS combines Agent Loop, persistent hierarchically compressed Agent Memory, and extensible on-demand Agent Skill to refine outputs and provide domain-specific expertise.Source: §1 · p. 2
• 9 diverse tasks validate GEMS across five mainstream benchmarks and four specialized downstream tasks, with generalizability tested across multiple generative backends.Source: §1 · p. 2
• 14.22 average gain on mainstream benchmarks and 14.03 on downstream tasks are achieved with Z-Image-Turbo, while the 6B model surpasses Nano Banana 2 on GenEval2.Source: §1 · p. 2
• The architecture augments the user prompt with skills, iteratively refines it in the Agent Loop, and uses Agent Memory to manage historical context.Source: Figure 2 · p. 3

• Inference-time scaling methods include prompt rewriting, random search, Chain-of-Thought reasoning, and iterative optimization for multimodal generation.Source: §2.1 · p. 3
• Agent systems extend language-model reasoning and execution through structured planning, interaction, self-correcting loops, and specialized roles communicating in multi-agent frameworks.Source: §2.2 · p. 3

• GEMS comprises Agent Loop, Agent Memory, and Agent Skill, which collaborate on complex instruction following and specialized downstream tasks.Source: §3 · p. 3

Agent Loop

• Agent Loop coordinates Planner, Decomposer, Generator, Verifier, and Refiner modules.Source: §3.1 · p. 4
• The Planner retrieves triggered domain skills, synthesizes an enhanced prompt, and sends the original prompt to the Decomposer for evaluation planning.Source: §3.1 · p. 4
• The Decomposer partitions the original prompt into atomic visual requirements represented as binary semantic or structural probes.Source: §3.1 · p. 4
• At each iteration, the model-agnostic Generator synthesizes an image from the current optimized prompt.Source: §3.1 · p. 4
• The Verifier uses an MLLM to assess each image against atomic criteria and produce binary feedback for refinement or termination.Source: §3.1 · p. 4
• When criteria remain unsatisfied, the Refiner combines the current prompt, image, verification feedback, internal reasoning, and prior memory to produce the next prompt.Source: §3.1 · p. 4

Agent Memory

• Agent Memory stores the optimization trajectory while hierarchical compression distills verbose reasoning traces into concise experiences and retains factual states with strategic reflections.Source: §3.2 · pp. 4–5

Main Results

• 14.22 average normalized-score increase on mainstream tasks with Z-Image-Turbo demonstrates consistent gains over prior inference-time scaling baselines.Qwen-Image-2512 further validates effectiveness across a different foundational architecture.Source: §4.2 · p. 6
• 14.03 average normalized-score improvement on downstream tasks with Z-Image-Turbo surpasses the best inference-time scaling baseline’s +8.92.Prompt-rewriting baselines degrade on some tasks, particularly text rendering.Source: §4.2 · p. 6

Ablation Studies

• 63.5 GenEval2 score results from adding Agent Loop, Agent Memory, and Agent Skill to a 31.0 baseline, with respective gains of 21.4, 9.0, and 2.1 points.The ablation uses Z-Image-Turbo and averages results over three independent runs.Source: §4.3 · pp. 6–7
• 71.4% passed criteria on GenEval2 after rising from 62.2% across iterations, indicating directed optimization rather than merely random variation.GEMS shows a consistent upward trajectory, unlike the fluctuating gains of a basic Agent Loop Only approach.Source: §4.3 · pp. 7–8

Memory Analysis

• 2.5 points are gained by compressing raw reasoning logs into concise Experiences, whereas including full thoughts produces negligible gains.Historical prompts and feedback add 3.4 points, while generated images add a further 3.1 points.Source: §4.3 · p. 8

Efficiency and Skills

• 2.80 average iterations, down from 3.26, follows Agent Memory and Agent Skill improvements while GEMS maintains lower overhead through early stopping.At approximately three generated images per task, GEMS substantially outperforms other inference-time scaling methods.Source: §4.3 · pp. 8–9

• GEMS reframes text-to-image generation as an iterative optimization problem.Source: §5 · p. 9
• The framework unifies iterative refinement, persistent trajectory-level memory, and domain-specific skills.Source: §5 · p. 9
• 14.22 average gain across five mainstream benchmarks and 14.03 across four specialized downstream tasks are achieved with Z-Image-Turbo.Source: §5 · p. 9
• Z-Image-Turbo surpasses Nano Banana 2 on GenEval2 despite using a 6B foundational model.Source: §5 · p. 9

• Detailed prompts are provided for the Planner, Decomposer, Verifier, Refiner, and Compressor modules.Source: §A.1 · p. 15, Figure 12 · p. 16, Figure 13 · p. 16
• The Skill Manager and Memory Manager are implemented as programmatic modules rather than LLM agents.Source: §A.1 · p. 15

• Evaluations follow the official settings of Z-Image and Qwen-Image.Source: §A.2 · p. 17
• Nano Banana 2 configuration is also presented for comprehensive comparison.Source: §A.2 · p. 17

• Table 3 summarizes the evaluation details for the compared models.Source: Table 3 · p. 17
• LongText-Bench, SpatialGenEval, CREA, and ArtiMuse evaluate text rendering, spatial intelligence, creative drawing, and aesthetic drawing, respectively.Source: §A.3 · p. 17
• The study compares GEMS with inference-time scaling baselines including Rewrite, Promptist, Random Search, Maestro, and CRAFT.Source: §A.4 · p. 17
• Rewrite enhances initial prompts with an MLLM, while Promptist automatically rephrases prompts using a fine-tuned language model and beam search.Source: §A.4 · p. 17
• Random Search generates multiple images and selects the best according to verifier-assigned scores.Source: §A.4 · p. 17
• Maestro iteratively generates two images per round and uses pairwise comparison to evolve outputs.Source: §A.4 · p. 17
• CRAFT refines prompts from MLLM feedback and targeted updates without specialized memory management.Source: §A.4 · p. 17

• Scores are reported separately for GenEval, GenEval2, DPG-Bench, OneIG, WISE, SpatialGenEval, and CREA.The breakdown covers seven benchmarks and follows each benchmark’s official evaluation scripts.Source: B Result Details · p. 17
• The analysis examines how different inference-time scaling strategies affect individual generation capabilities.Source: B Result Details · p. 17
• Benchmark subsets are used to provide a more comprehensive performance comparison than aggregate results alone.Source: B Result Details · p. 17

Qualitative comparisons

• GEMS and the baseline are compared on generated outputs using the same prompts.Examples include an Earth-from-the-moon view and a liquid-water butterfly formed from a splash.Source: C Qualitative Results · p. 17, Figure 16 · p. 21, Figure 17 · p. 22

Mainstream benchmark dimensions

• GenEval measures single-object, two-object, counting, color, position, and attribute-binding capabilities.Source: Table 4 · p. 18
• GenEval2 evaluates object, attribute, count, position, and verb dimensions.Source: Table 5 · p. 18
• DPG-Bench and OneIG evaluate global/entity/relation factors and alignment/text/reasoning/style/diversity factors, respectively.Source: Table 6 · p. 19, Table 7 · p. 19, Table 8 · p. 20

Downstream benchmark dimensions

• WISE, SpatialGenEval, and CREA cover scientific domains, spatial relations, and creative-quality dimensions with benchmark-specific categories.WISE includes cultural, temporal, spatial, biological, physical, and chemical dimensions; SpatialGenEval and CREA use their listed fine-grained categories.Source: Table 9 · p. 20, Table 10 · p. 21, Table 11 · p. 21

Efficiency

• Agent Loop’s iterative workflow causes noticeable inference latency despite using lightweight, distilled Z-Image-Turbo.Future work targets workflow optimization to reduce computational overhead and improve efficiency.Source: D Limitations and Future Work · p. 22

Agent autonomy

• The current system depends on predefined workflows to coordinate agent collaboration and module interactions.The authors propose investigating tool interfaces for more autonomous memory and skill management.Source: D Limitations and Future Work · p. 22

Modality scope

• GEMS is primarily designed for image generation, with future research considering more complex multimodal domains such as video generation.Source: D Limitations and Future Work · p. 22

Editing support

• Z-Image-Turbo and Qwen-Image-2512 currently do not support image editing.The proposed direction is to use more versatile models for a unified reasoning, generation, and editing loop.Source: D Limitations and Future Work · p. 22