PaperlayerView exact revision on arXiv

Source-linked AI summary

Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation

Jun He, Junyan Ye, Zilong Huang, Dongzhi Jiang, Chenjue Zhang, Leqi Zhu, Renrui Zhang, Xiang Zhang, Weijia Li

arXiv:2602.01756v1cs.CV

TL;DR

Existing image generators often miss implicit intentions, lack complex reasoning, and cannot verify evolving real-world facts. Mind-Brush adds multimodal retrieval and reasoning to generation, raising Qwen-Image accuracy from 0.02 to 0.31 on Mind-Bench and improving WISE and RISEBench results.

  • Problem

    Existing image generators struggle with implicit intent, complex reasoning, and factual verification of real-time or evolving contexts.

  • Method

    Mind-Brush is a unified agentic framework that retrieves multimodal evidence and applies logical reasoning in a human-like Think-Research-Create workflow, evaluated with 500-sample Mind-Bench.

  • Results

    0.02 to 0.31 accuracy: Mind-Brush substantially improves Qwen-Image on Mind-Bench, with WISE WiScore +25.8% and RISEBench Accuracy +27.3%.

  • Takeaways & Limitations

    The results support an agentic generative paradigm combining active retrieval and logical deduction for knowledge-intensive, reasoning-dependent visual synthesis.

  • Takeaways & Limitations

    Existing methods remain limited by reliance on internalized knowledge without external tools, hindering factual verification of real-time events or evolving contexts.

Abstract

from arXiv · show

While text-to-image generation has achieved unprecedented fidelity, the vast majority of existing models function fundamentally as static text-to-pixel decoders. Consequently, they often fail to grasp implicit user intentions. Although emerging unified understanding-generation models have improved intent comprehension, they still struggle to accomplish tasks involving complex knowledge reasoning within a single model. Moreover, constrained by static internal priors, these models remain unable to adapt to the evolving dynamics of the real world. To bridge these gaps, we introduce Mind-Brush, a unified agentic framework that transforms generation into a dynamic, knowledge-driven workflow. Simulating a human-like 'think-research-create' paradigm, Mind-Brush actively retrieves multimodal evidence to ground out-of-distribution concepts and employs reasoning tools to resolve implicit visual constraints. To rigorously evaluate these capabilities, we propose Mind-Bench, a comprehensive benchmark comprising 500 distinct samples spanning real-time news, emerging concepts, and domains such as mathematical and Geo-Reasoning. Extensive experiments demonstrate that Mind-Brush significantly enhances the capabilities of unified models, realizing a zero-to-one capability leap for the Qwen-Image baseline on Mind-Bench, while achieving superior results on established benchmarks like WISE and RISE.

1. Introduction

Mind-Brush reframes image generation as a dynamic, knowledge-driven workflow that combines multimodal retrieval with reasoning to address implicit intentions, evolving knowledge, and complex visual constraints. Mind-Bench evaluates these capabilities, and Mind-Brush substantially improves the Qwen-Image baseline and performance on WISE and RISEBench.

  • Motivation: Static text-to-pixel decoders often fail to grasp implicit, high-level user intentions.Their outputs are constrained to mapping explicit user instructions to pixels.
  • Motivation: Temporal knowledge cutoffs prevent current image generation models from adapting to evolving real-world dynamics, including real-time news and novel IP concepts.Their cognitive boundaries remain static because pre-training data has a temporal knowledge cutoff.
  • Limitations of Existing Methods: Existing agentic methods emphasize prompt refinement or sequential drawing steps but lack external tools for factual verification and complex reasoning.These approaches remain largely confined to standard T2I benchmarks and rely solely on internalized knowledge.
  • Mind-Brush: Mind-Brush unifies intent analysis, multimodal search, and knowledge reasoning in a dynamic ’think-research-create’ workflow.It retrieves multimodal evidence for out-of-distribution concepts and uses logical reasoning to deduce implicit visual constraints.
  • Mind-Bench: Mind-Bench evaluates generative capabilities involving dynamic external knowledge and complex reasoning beyond instruction following, internal knowledge recall, and rudimentary reasoning.Existing benchmarks fall short of testing active information retrieval and complex reasoning.
  • Results: 0.02 to 0.31: Mind-Brush elevates the Qwen-Image baseline on Mind-Bench; +25.8 % WiScore and +27.3 % Accuracy are achieved on WISE and RISEBench, respectively.These results substantially outperform existing baselines on the established knowledge-driven WISE and reasoning-driven RISEBench benchmarks.

2. Related Work

Related work has progressed from agentic prompt refinement toward unified multimodal architectures that combine understanding and image generation. However, fidelity, understanding–generation balance, and evaluation of deeper comprehension remain open challenges.

  • Agentic Image Generation: Multimodal LLMs increasingly act as agentic decision-makers that translate vague user intents into precise image-synthesis instructions.Prompt-optimization approaches include multi-agent refinement in T2I-Copilot and PromptSculptor, alongside ImAgent’s test-time policy scaling for semantic alignment.
  • Unified Multimodal Models: Unified multimodal models integrate cross-modal understanding and generation, but early visual-token approaches suffered fidelity limitations from lossy VQ-VAE compression.Chameleon and Emu3 introduced image generation into the LLM paradigm through visual signal discretization.
  • Unified Multimodal Models: Balancing understanding and generation remains difficult, motivating decoupled designs that use multimodal language models to guide external diffusion heads.This strategy is adopted by methods including OmniGen2 and BLIP-o3, following architectural innovations such as Mixture-of-Tokens and Mixture-of-Experts in Bagel and OneCat.
  • Evaluation: Existing benchmarks mainly measure text-image alignment and instruction following, often restricting evaluation to shallow, explicitly stated comprehension.GenEval assesses compositional integrity by measuring whether models bind prompt-specified attributes such as counts and positions.

3. Mind-Brush

Mind-Brush models image generation as a hierarchical, context-aware decision process that detects cognitive gaps, routes requests through search or reasoning tools, and accumulates verified evidence. A final review consolidates this evidence into a structured master prompt for generation.

  • Framework: Mind-Brush formalizes inference as a Hierarchical Sequential Decision-Making Process that produces a structured cognitive trajectory from user intent to visual realization.The process is defined by M = ⟨S, A, π, E⟩.
  • Framework: The cognitive state combines the instruction, an optional reference image, and a dynamic evidence buffer accumulating retrieved knowledge and reasoning chains.At step t, the state is st = {I, Iimg, Et}.
  • Cognitive Gap Detection: Cognitive Gap Detection uses 5W1H analysis to identify entities or logical dependencies requiring external verification and converts missing knowledge into atomic questions Qgap.The resulting questions determine whether the workflow follows factual grounding or logical reasoning.
  • Evidence Acquisition: For out-of-distribution entities or dynamic events, Cognition Search generates textual and visual queries, retrieves factual documents, and updates both textual context and reference-image retrieval.The dual update injects retrieved concepts into I′ and calibrates Q′img so Iref aligns with validated facts.
  • Evidence Acquisition: For complex deductions, CoT Knowledge Reasoning processes the instruction, input image, and accumulated search evidence to resolve conflicts or interpret visual data into explicit conclusions Rcot.The final evidence set combines search evidence and reasoning conclusions as E = Esearch ∪ Rcot.
  • Information Consolidation and Conditional Synthesis: The Concept Review Agent filters redundant or irrelevant evidence and synthesizes verified facts, logical conclusions, and creative intent into a structured Master Prompt Pmaster.This consolidation phase prepares the final prompt for precise image generation.

4. Mind-Bench

Mind-Bench evaluates cognitive image generation beyond static text-to-pixel translation by testing multimodal knowledge retrieval and implicit-constraint reasoning across diverse domains. It is built with human-curated evidence and strict checklist-based evaluation to improve factual reliability and penalize partial correctness.

  • Motivation: Mind-Bench targets cognitive generation by addressing benchmarks’ limited temporal sensitivity, multimodal reasoning depth, and reliance on static knowledge.It is designed around a “Research-then-Create” paradigm for complex image generation.
  • Benchmark Structure: Mind-Bench comprises two primary clusters spanning 10 sub-domains: Knowledge-Driven Tasks and Reasoning-Driven Tasks.Knowledge-driven tasks cover Special Events, Weather, Character, Object, and World Knowledge; reasoning-driven tasks cover Life, Geo, Math, Science & Logic, and Poem.
  • Knowledge-Driven Tasks: Knowledge-Driven Tasks assess retrieval and integration of external information for precise visual grounding while mitigating hallucinations about out-of-distribution concepts.The five sub-domains include breaking news, real-time weather, specific IP characters, long-tail artifacts, and common-sense world knowledge.
  • Reasoning-Driven Tasks: Reasoning-Driven Tasks test whether models can deduce implicit constraints from simple instructions across life, geographic, mathematical, scientific, logical, and poetic domains.These tasks require latent reasoning results for visualization and comprehension.
  • Construction: Mind-Bench uses a human-machine collaborative pipeline in which 6 AI graduate students curate difficult prompts, collect multimodal evidence, and verify executable evaluation checklists.Each final sample contains an instruction, multimodal reference evidence, and a rigorous checklist.
  • Evaluation Metric: Checklist-based Strict Accuracy (CSA) judges each generated image against all checklist items, counting a sample as correct only when every sub-item passes.CSA uses an MLLM judge and penalizes generations that are partially correct but logically flawed.

5. Experiments

Mind-Brush is evaluated against proprietary and open-source image-generation baselines on Mind-Bench, WISE, and RISEBench under consistent implementation settings. It improves knowledge- and reasoning-dependent generation, with gains attributed to complementary search and reasoning agents.

  • Evaluation setup: Mind-Bench evaluates dynamic external knowledge and multi-step reasoning with Checklist-based Strict Accuracy, while WISE uses WiScore for semantic understanding and world knowledge integration.The evaluation uses three benchmarks with distinct focuses, including RISEBench.
  • Evaluation setup: Mind-Brush is compared with mainstream proprietary UMMs and state-of-the-art open-source T2I models or UMMs, all using official default settings.The baselines include GPT-Image-1, GPT-Image-1.5, Nano Banana, Nano Banana Pro, FLUX-2 Pro, FLUX-2 Max, FLUX.1 dev, FLUX 1 Kontext, FLUX 1 Krea, Z-Image, GenAgent, and Qwen-Image.
  • Main results: 30.0% and 29.0% are Mind-Brush’s overall generation-accuracy improvements over SD-3.5 Large and Qwen-Image, respectively, on Mind-Bench.The framework enables open-source base models to match or exceed proprietary methods despite Qwen-Image’s lower baseline accuracy.
  • Main results: 25.8% and 0.78 are Mind-Brush’s WiScore improvement over Qwen-Image and overall WISE score, respectively; 61.5 is its RISEBench Instruction Reasoning score.On WISE it matches GPT-Image-1, while on RISEBench it surpasses Nano Banana and exceeds Bagel by 68.5%.
  • Qualitative analysis: Mind-Brush retrieves visual references for out-of-distribution concepts and dissects implicit spatial or mathematical attributes to produce logically coherent, visually faithful images.These qualitative behaviors distinguish its knowledge-driven and reasoning-driven results from baselines that fail to recognize relevant concepts or infer implicit constraints.
  • Ablation study: 0.28 is the accuracy improvement from the Cognition Search Agent on Knowledge-Driven tasks, while jointly deploying both agents improves accuracy by 0.17 and 0.06 over single-agent configurations.The Knowledge Reasoning Agent improves Reasoning-Driven tasks, and the combined agents show a synergistic effect.

6. Conclusion … A.1. Workflow of Mind-Brush

Mind-Brush is a training-free agentic framework that transforms text-to-image generation into an active cognitive workflow combining intent analysis, multimodal grounding, and explicit reasoning. Its workflow retrieves evidence, resolves cognitive gaps, and synthesizes a factually and logically precise master prompt for image generation.

  • 6. Conclusion: Mind-Brush transforms text-to-image generation from passive decoding into an active cognitive workflow.The framework is training-free and designed to align vague user intents with objective reality.
  • 6. Conclusion: The framework orchestrates intent analysis, multimodal grounding, and explicit chain-of-thought reasoning to bridge vague intents and precise, factually grounded visual synthesis.Mind-Bench evaluates these capabilities on knowledge-intensive and reasoning-dependent tasks.
  • 6. Conclusion: Mind-Bench is established as a benchmark for stress-testing models on knowledge-intensive and reasoning-dependent image-generation tasks.The supplied conclusion passage states that empirical results demonstrate significant outperformance, but provides no complete quantitative result here.
  • A.1. Workflow of Mind-Brush: The workflow begins with a user instruction, an optional user image, and initialized foundation models and toolsets.These inputs provide the starting context for subsequent agentic processing.
  • A.1. Workflow of Mind-Brush: During intent analysis, Mind-Brush decomposes inputs using 5W1H, identifies cognitive gaps, and formulates a dynamic execution strategy.The resulting variables are denoted Qgap for cognitive gaps and Splan for the execution strategy.
  • A.1. Workflow of Mind-Brush: Cognition Search performs two-stage retrieval by gathering textual evidence and refining visual queries for semantically accurate reference images.Textual evidence grounds factual concepts, while retrieved images provide multimodal references.
  • A.1. Workflow of Mind-Brush: The Knowledge Reasoning agent activates for tasks requiring deep logic or cross-modal understanding, while Concept Review synthesizes evidence into a coherent Master Prompt.The Image Generator uses this prompt to produce a final image aligned with user creativity and real-world logic.

A.2. Additional Experimental Results

Mind-Brush is additionally evaluated on the instruction-following GenEval++ and creative-generation Imagine-Bench benchmarks. On GenEval++, it outperforms Agentic baselines, with improvements that approach GPT-4o performance.

  • Benchmark Evaluation: GenEval++ evaluates complex instruction following with Accuracy across categories including Color, Count, and Position, while Imagine-Bench targets creative generation.The benchmarks are used for additional evaluations omitted from the manuscript due to space limitations.
  • Inference Workflow: Mind-Brush decomposes instructions with 5W1H, searches for multimodal evidence when needed, applies knowledge reasoning, and rewrites a master prompt for image generation.The workflow optionally accepts a user image and conditions final generation on retrieved visual references.
  • GenEval++: 41.7% Accuracy improvement is achieved by Mind-Brush over GenAgent on GenEval++ Pos/Count.GenAgent is identified as the best-performing baseline among the compared Agentic methods.
  • GenEval++: 13.3% Accuracy improvement is achieved by Mind-Brush over GenAgent on GenEval++ Multi-Count.The resulting performance is reported as close to GPT-4o.

A.3. Additional Ablation Study

The ablation study evaluates Mind-Brush across different MLLM backbones and image-generation models, separating agentic intelligence from visual synthesis. Results show that stronger reasoning backbones and visual executors both improve performance, with the MLLM exerting the dominant influence.

  • A.3. Additional Ablation Study: Mind-Brush ablations vary the MLLM backbone and image-generation model to assess robustness across computational regimes and disentangle agentic intelligence from visual synthesis.The study reports quantitative results in Table 6.
  • Quantitative Comparison with Different MLLM Baselines: 29.2% relative gain: replacing Qwen3-VL with GPT-5.1 while keeping Qwen-Image fixed improves overall performance from 0.24 to 0.31.The result identifies the underlying MLLM as the dominant factor, improving retrieval, verification, and generation fidelity.
  • Quantitative Comparison with Different Image Generation Models: CSA score increases from the baseline of 0.17 to 0.34 when GPT-Image-1 is integrated with Mind-Brush driven by GPT-5.1.Under the same GPT-5.1 backbone, the GPT-Image-1 configuration outperforms the Qwen-Image variant.
  • Quantitative Comparison with Different Image Generation Models: The framework scales with stronger generation engines while preserving the MLLM’s role in intent alignment and factuality.Performance benefits from both the intelligence substrate and the visual decoder.

A.4. Evaluation Protocols · B. Additional Details of Mind-Bench · B.1. Additional task description of Mind-Bench

The paper standardizes evaluation across benchmarks using official protocols and specialized judges or metrics. Mind-Bench is further specified through strict factual verification and a 10-category taxonomy spanning knowledge-driven and reasoning-driven tasks.

  • A.4. Evaluation Protocols: Official benchmark protocols and LLM judge settings are used for standardized evaluation across cognitive dimensions.Existing-benchmark automated evaluations use the official LLM settings as the core judge.
  • A.4. Evaluation Protocols: WISE performance is reported with WiScore, which aggregates Consistency, Realism, and Aesthetic Quality on a discrete 3-point scale.The metric measures knowledge-image alignment across Scon, Sreal, and Saes.
  • A.4. Evaluation Protocols: WiScore = α1 · ¯Scon + α2 · ¯Sreal + α3 · ¯Saes combines the three WISE sub-dimensions through a weighted linear formulation.The formula is defined as the benchmark’s composite score.
  • A.4. Evaluation Protocols: The WISE weights are α1 = 0.7, α2 = 0.2, and α3 = 0.1, prioritizing semantic consistency while penalizing physical-law violations and weak aesthetics.Consistency receives the greatest weight, followed by Realism and Aesthetic Quality.
  • A.4. Evaluation Protocols: RISEBench evaluates reasoning-informed visual editing with GPT-4.1 scores from 1 to 5 across instruction reasoning, appearance consistency, and visual plausibility.The dimensions measure directive execution, preservation of irrelevant attributes, and plausibility of the edited result.
  • A.4. Evaluation Protocols: Mind-Bench uses Checklist-based Strict Accuracy with Gemini-3.0-Pro to verify human-verified atomic factual claims through binary judgments.The protocol is designed to quantify cognitive-gap bridging while mitigating self-evaluation bias in GPT-series models.
  • A.4. Evaluation Protocols: The Mind-Bench protocol is intended to ensure that accuracy reflects comprehensive external-knowledge grounding rather than superficial semantic overlap.Its strict verification procedure emphasizes precise grounding and resistance to hallucination.
  • B.1. Additional task description of Mind-Bench: Mind-Bench comprises 10 task categories organized into Knowledge-driven and Reasoning-driven domains, with taxonomy details covering distributions, modalities, and definitions.The expanded specification appears in Table 7 because of space limitations in the main manuscript.

B.2. Comparison with Existing Benchmarks

Mind-Bench is positioned as a more temporally dynamic and rigorous alternative to existing text-to-image benchmarks. It uniquely evaluates real-time information retrieval while providing an objective standard for agentic generation systems.

  • Benchmark comparison: Mind-Bench’s comparison with existing T2I benchmarks highlights advantages in temporality, modality, and evaluation rigor.These distinctions are presented in Table 8.
  • Benchmark comparison: Mind-Bench is the sole platform described as assessing real-time information retrieval and targeting dynamic concepts.GenEval and WISE instead rely on static, frozen knowledge distributions.
  • Benchmark comparison: Its scoring framework provides a precise, objective standard for validating agentic generation systems.This extends evaluation beyond simple text alignment.

B.3. Data Sources and Copyright Statement · C. Additional Visualization of Inference Process

Mind-Bench uses publicly available or non-commercial data sources under applicable copyright terms, with redistribution adapted to each source’s licensing restrictions. Additional figures visualize Mind-Brush’s step-by-step inference and image-synthesis process across diverse benchmark tasks.

  • B.3. Data Sources and Copyright Statement: Mind-Bench primarily uses publicly available academic datasets or public information sources permitting non-commercial use.The benchmark construction explicitly follows the copyright policies of its data sources.
  • B.3. Data Sources and Copyright Statement: Wikipedia supplies reference images and descriptions for News, Historical Events, Specific Characters, Trendy IPs, World Knowledge, and Geographical Understanding under CC BY-SA 3.0.The cited Wikipedia content is used under the Creative Commons Attribution-ShareAlike 3.0 Unported License.
  • B.3. Data Sources and Copyright Statement: Weather data comes from world-weather.info, life-reasoning images from recipetineats, and Math data from MathVerse under its academic citation license.These sources cover historical weather conditions, life reasoning, and mathematical images and queries, respectively.
  • B.3. Data Sources and Copyright Statement: Mind-Bench directly releases content under compatible open-source licenses but provides metadata and original URLs when secondary redistribution is restricted.This policy allows users to download restricted content independently while complying with the relevant terms of service.
  • C. Additional Visualization of Inference Process: Additional qualitative examples in Figures 5–24 visualize Mind-Brush’s complete cognitive trajectory from initial intent analysis to final image synthesis.The examples are intended to provide an intuitive understanding of the framework across diverse scenarios.
  • C. Additional Visualization of Inference Process: Figures 19–24 visualize generation processes for Geo Math, Math, Science & logic, and Poem tasks.These examples extend the visualized inference coverage to mathematical, scientific, logical, and poetic tasks.
Loading 2602.01756v1…