PaperlayerView exact revision on arXiv

Source-linked AI summary

Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation

Haozhe Wang, Weijia Feng, Jinpeng Yu, Che Liu, Ping Nie, Fangzhen Lin, Jiaming Liu, Ruihua Huang, Jimmy Lin, Wenhu Chen, Cong Wei

arXiv:2607.05382v4cs.CVcs.AI

TL;DR

Visual generators struggle with open-ended, evolving world knowledge despite strong rendering ability, while existing benchmarks underrepresent these failures. This paper introduces SearchGen benchmarks and a teach-then-search co-training framework, achieving monotonic gains from no search to generator-adaptive search. The results show that a generator-specific knowledge boundary can guide more effective external knowledge use.

  • Problem

    Visual generators face a world-knowledge bottleneck because fixed training corpora cannot cover users’ open-ended, evolving, and long-tailed requests.

  • Method

    The paper constructs SearchGen datasets and discovers generator-specific knowledge boundaries through co-training generators to internalize stable knowledge and reasoners to search remaining gaps.

  • Results

    Monotonic gains arise from NO SEARCH through BLIND SEARCH to GENERATOR-ADAPTIVE SEARCH, while frontier open generators score only 21–28 out of 100 on SearchGen-Bench.

  • Takeaways & Limitations

    A generator-specific knowledge boundary is an operational design variable for coordinating internal knowledge and external search in visual generation.

  • Takeaways & Limitations

    Naive search can cause concept corruption and copy effects when references override accurate internal knowledge or become copying templates.

Abstract

from arXiv · show

Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed multimodal SearchGen-Corpus-1M to support offline, reproducible research. On SearchGen-Bench, frontier open generators score only 21 to 28 out of 100, a 40-point collapse invisible to existing benchmarks. The natural remedy is to employ search tools, enabling agentic visual generation. However, we find that naive search fails: it retrieves indiscriminately, injecting noise into prompts the generator already handles. We trace the root cause to a generator-specific, evolving knowledge boundary: the divide between what a generator can internalize through training and what must remain in external context. Although this boundary is hard to specify in advance, we show that it is discoverable through a teach-then-search co-training framework. Even a minimal version of this co-training recipe produces monotonic improvement, laying the foundation for recursive self-improvement in visual generation that can meet world-knowledge-grounded requests. We release the full dataset, co-training corpus, and search corpus as a replayable harness for tool-augmented, world-knowledge-grounded visual generation.

1 Introduction

Modern image generators render complex scenes well but fail on open-ended, evolving world knowledge, revealing a knowledge bottleneck rather than a visual-synthesis bottleneck. SearchGen-20K and SearchGen-Bench expose this gap, while noise-resistant search and teach-then-search co-training enable generator-adaptive improvement.

  • Problem: Generators produce polished visuals yet fabricate unfamiliar or historically grounded content because fixed training corpora impose knowledge cutoffs on unbounded, evolving user requests.Examples include fabricating the 2025 Osaka Expo mascot and rendering anachronistic armor for a historically accurate Spartan phalanx.
  • Agentic visual generation: Naive search degrades many prompts because indiscriminate retrieval injects irrelevant detail, stylistic contamination, and spurious structure into generator inputs.Open-weight generators treat all conditioning signals as authoritative and cannot distinguish useful references from noise.
  • Method and results: A three-stage gate–filter–integrate reasoner and minimal teach-then-search co-training produce monotonic gains from NO SEARCH through BLIND SEARCH to GENERATOR-ADAPTIVE SEARCH.The framework targets the evolving boundary between knowledge generators can internalize and knowledge that must remain in external context.
  • Benchmark: 20,939 records span twelve failure categories and twenty-two domains, paired with pre-executed multimodal SEARCHGEN-CORPUS-1M for reproducible research.SEARCHGEN-20K is annotated with fine-grained visual checklists for automated assessment.
  • Benchmark: Up to 40 points separate frontier open generators’ performance on SEARCHGEN-BENCH from existing evaluations, exposing a substantial world-knowledge evaluation gap.The benchmark reports frontier open generators achieving only 21–28 points.

2 The World-Knowledge Bottleneck

The world-knowledge bottleneck arises because fixed-corpus visual generators face open-ended, evolving, long-tailed requests and cannot recognize what they do not know. Search-intensive evaluation exposes this gap, while naive search can further damage prompts the generator already handles, motivating selective, modality-aware search.

  • The World-Knowledge Bottleneck: Fixed training corpora cannot cover open-ended, evolving, long-tailed user requests, and generators lack a mechanism to recognize missing knowledge.This structural blind spot limits faithful generation more than rendering ability.
  • The World-Knowledge Bottleneck: 20,840 production prompts revealed twelve recurring failure categories, showing that real-user demand is more diverse, long-tailed, and rapidly evolving than existing benchmarks suggest.The categories span distinct knowledge gaps, including visual identity, factual precision, and requests requiring both modalities.
  • The World-Knowledge Bottleneck: Nearly 40 points separate comparable standard-prompt performance from open-generator performance on search-intensive world-knowledge prompts.Open and commercial generators score 67–75 out of 100 on parametric prompts, while the same open generators collapse to 22–28 on search-intensive prompts.
  • The World-Knowledge Bottleneck: BlindSearch degrades every generator on NoSearch prompts, whereas ReasonedSearch raises Qwen-Image-2 from 70.7 to 76.5 and TextualSearch from 22.9 to 34.1.Selective search must learn when to abstain so that knowledge-intensive gains do not damage parametric knowledge.
  • The World-Knowledge Bottleneck: Search failures arise from concept corruption and copy effect, motivating a gate–filter–integrate protocol and modality choice based on each knowledge gap.Visual references address underspecified identities, textual search supplies structured facts, and many requests require both modalities.

3 Co-Training Agentic Visual Generation

Co-training enables agentic visual generation by pairing a noise-resistant, three-stage search protocol with sequential generator teaching and reasoner recalibration. This process discovers and tracks a generator-specific knowledge boundary, reducing unnecessary search while preserving search for knowledge that remains external.

  • Framework: The framework combines a reasoner that controls search and evidence integration with co-training that expands the generator before recalibrating its search boundary.The reasoner handles visual and web knowledge, while co-training first teaches the generator and then updates the reasoner.
  • Three-stage agentic search: Three sequential stages suppress search noise by gating knowledge gaps, filtering references that fill only the identified gap, and integrating evidence through enriched natural-language specifications.Gate emits at most 3 queries or SKIP; filtering reduces visual-copy shortcuts, while integration limits leakage of style, background, and layout.
  • Knowledge boundary: The knowledge boundary partitions internalizable knowledge from contextual knowledge, and it is generator-specific, shifts under training, and determines when search is structurally necessary.Search should not fire for internalizable knowledge but remains necessary for contextual knowledge outside the generator’s capacity.
  • Co-training: The boundary is discoverable through co-training because Phase 1 strengthens the generator, reducing prompts that require search, before Phase 2 recalibrates the reasoner to the expanded boundary.Figure 9b measures the per-prompt quality gap with and without search, whose distribution shifts under co-training.
  • Two-phase co-training: Phase 1 uses search-augmented inputs and online Diffusion-DPO to internalize stable world knowledge and build robustness to imperfect visual references, while Phase 2 moves search inward to remaining gaps.DPO reinforces the generator’s best outputs; recalibration uses rejection-sampling finetuning after the generator’s boundary has shifted.

4 Validating Evolving Knowledge Boundary through Co-Training

Co-training validates that the generator’s knowledge boundary is discoverable, shifting through generator improvement and reasoner calibration. Across two generators, the resulting calibrated reasoner improves monotonically, selectively avoids harmful search, and approaches frontier-oracle performance.

  • Monotonicity: Co-training improves monotonically across phases and difficulty tiers, with the hardest Set III showing the largest gains.Figure 9 reports monotonic improvement for all three search-intensive tiers, while Set III has the most room for boundary-shift gains.
  • Monotonicity: 31.8 overall on Klein-4B after Phase 2 slightly exceeds the 31.2 frontier oracle, with DPO adding +2.8 and RFT adding +2.6.The same monotonic pattern also holds for Bagel.
  • Selectivity: 56.9 on NoSearch is a +7.0 gain over the 49.9 no-search DPO baseline, showing that calibration learns when to abstain from harmful search.This directly tests whether search-intensive improvements sacrifice prompts the generator already handles.
  • Generator-specificity: 26.8 overall with base Klein-4B versus 31.8 with Klein-4B-DPO demonstrates that the calibrated reasoner’s behavior is generator-specific.The cross-check supports treating the knowledge boundary as an operational property of the generator–reasoner pair.
  • Structural interpretation: 31.8 from the co-trained 8B reasoner paired with a 4B generator slightly exceeds the 31.2 frontier VLM oracle on that generator.The matched-compute comparison suggests generator-specific calibration can approach frontier-scale reasoning at a fraction of the cost.

5 Related Work

Prior work spans search-augmented language modeling, selective retrieval, tool-augmented image generation, and knowledge-intensive evaluation. This work differs by discovering a generator-specific knowledge boundary through deliberately ordered teach-then-search co-training.

  • Search-augmented generation: RAG and REALM ground language models in external documents at inference time, while Atlas and REPLUG jointly train searchers and generators end-to-end.The latter align retrieval distributions with generator needs and outperform frozen-searcher pipelines.
  • Selective and adaptive retrieval: Self-RAG, FLARE, Toolformer, and CRAG study retrieval control, confidence-triggered search, API invocation, and post-hoc retrieval correction.These approaches address when to search or how to assess retrieved evidence in text systems.
  • Tool-augmented and agentic image generation: GenAgent, RPG-DiffusionMaster, LLM-grounded Diffusion, and IterComp use LLM planning to decompose prompts and improve compositional generation within existing knowledge.Instruct-Imagen adds multimodal instructions, while RationalRewards uses preference-calibrated prompt rewriting.
  • Knowledge-intensive evaluation: GenAI-Bench, T2I-CompBench, DALL-Eval, and HEIM evaluate prompt adherence, spatial reasoning, attribute binding, object relationships, and holistic quality within known visual concepts.TIFA and human-grounded automated metric suites extend VLM-based evaluation of visual generation.
  • Self-improving and co-training systems: Self-Rewarding Language Models and SPIN motivate self-improvement, whereas this work teaches the generator with search-augmented demonstrations before recalibrating the reasoner to its shifted boundary.The deliberate teach-then-search ordering and asymmetric generator–reasoner roles distinguish the approach from generic symmetric self-play.

6 Conclusion · Appendix overview · A Benchmark and dataset details

The paper frames visual generation around a generator-specific knowledge boundary that co-training can discover and progressively refine. Its released replayable resources and appendix document the benchmark, analyses, protocols, and implementation evidence supporting this framework.

  • 6 Conclusion: Co-training discovers a generator-specific knowledge boundary that is difficult to specify in advance, with self-awareness emerging from co-training dynamics rather than scale alone.The central objective is building a system that knows what it does not know.
  • 6 Conclusion: Online DPO expands the generator’s internal knowledge and improves its use of imperfect references, while rejection finetuning recalibrates search toward knowledge the generator still cannot render.Together, these stages co-train the generator and reasoner around their shared boundary.
  • 6 Conclusion: A minimal recipe using one DPO pass, one RFT pass, a 4B generator, and an 8B reasoner validates the boundary-learning principle at the smallest useful scale.Further iterations can expand the internalized boundary and narrow search toward genuinely contextual knowledge.
  • 6 Conclusion: Open questions include how the boundary scales across failure categories, whether model internals can predict it, and how the internalizable/contextual split changes with model scale.The passage specifically questions whether culturally specific knowledge resists internalization.
  • 6 Conclusion: The gate–filter–integrate protocol and teach-then-search co-training extend beyond search to image editing, render-as-code, 3D-asset retrieval, and structural control.Each tool supplies a different portion of the contextual knowledge set Kctx.
  • A Benchmark and dataset details: 20,939 prompt records, 96,848 reasoning trajectories, 283,493 generated images, and 159,027 archived search sessions support replayable research without live search API access.The records include 20,789 unique prompt strings.
  • Appendix overview: The appendix follows the paper’s evidence chain, covering benchmark construction, failure analysis, judge–human agreement, score breakdowns, workflow and evaluation, hardware, and implementation details.Its sections organize supporting evidence from benchmark heatmaps through limitations.

A.1 Benchmark construction details · B Additional empirical analysis

The benchmark is constructed from real user prompts through a reliable failure taxonomy, a long-tailed entity database, and staged prompt synthesis. It preserves global cultural diversity and uses fixed train, validation, and test partitions to target knowledge absence.

  • A.1 Benchmark construction details: Cohen’s κ > 0.85 was achieved between two annotators labeling 2,000 sampled failure cases before taxonomy application to all 10,840 production prompts.Iterative schema refinement supported treating the taxonomy as structural rather than an artifact of labeling choices; the taxonomy appears in Table 1.
  • A.1 Benchmark construction details: 31,537 seed entities were extracted from 10,840 real user prompts across 22 primary domains, with names, training-frequency estimates, visual references, and distinguishing attributes.The entity distribution intentionally follows real user demand’s long tail.
  • A.1 Benchmark construction details: SEARCHGEN-20K is synthesized by instantiating templates with entities, relations, and stylistic constraints distilled from the production scrape.The supplied passage also describes a second stage in which a frontier large language model rewrites instantiated templates into naturalistic user-style requests while preserving grounded entities and modality requirements.
  • A.1 Benchmark construction details: The benchmark includes entities from East Asia, South Asia, the Middle East, Africa, Latin America, and beyond to expose long-tail cultural knowledge gaps.A Western-centric benchmark would underestimate the severity of these gaps.
  • A.1 Benchmark construction details: The corpus uses train (20,000), validation (128), and test (751) partitions for supervised training and model selection.These partitions match Section 2.2.
  • A.1 Benchmark construction details: SEARCHGEN-BENCH targets knowledge absence rather than compositionality within known concepts.The supplied passage contrasts this focus with prior benchmarks cited in the text.

B.1 Extended failure analysis · B.2 Judge–human correlation and judge–reasoner independence

The extended analysis identifies generator-specific failure modes and shows that surface-triggered search can target the wrong prompts. Human agreement is high overall but weakens on TextualSearch, especially where glyph-level and layout correctness dominate.

  • B.1 Extended failure analysis: Concept corruption arises because conditioning lacks reliable confidence-based gating, so references receive substantial weight even when native rendering could be more accurate.This can corrupt concepts on prompts where internal knowledge was already sufficient.
  • B.1 Extended failure analysis: Pixel-level visual conditioning causes reference copying, reproducing incidental background, lighting, and framing instead of composing specified semantic attributes.Without an integration layer, outputs resemble filtered search results rather than new composite generations.
  • B.1 Extended failure analysis: Named entities are only loosely correlated with Qwen-Image’s inferred knowledge-boundary strata, making cheap surface-cue gates trigger search on the wrong subset.This repeats the tradeoff between VisualSearch/TextualSearch gains and NoSearch regressions.
  • B.1 Extended failure analysis: For Klein-4B-DPO, both BLIND SEARCH and GENERATOR-ADAPTIVE SEARCH score below the no-search baseline on the TextualSearch stratum.The hardest prompts disproportionately require compositional contextual reasoning involving typography, cultural composition, and implicit interpretation.
  • B.2 Judge–human correlation and judge–reasoner independence: The human study evaluates the same 500 prompt–image pairs with checklist, rubric, and holistic judgments, using multiple raters and adjudication for high-disagreement items.The protocol uses the same anchors as the automated judge and stabilizes gold labels under resampling.
  • B.2 Judge–human correlation and judge–reasoner independence: 0.87 Spearman ρ is the Gemini-3-Flash judge’s agreement with consolidated human labels across the evaluation set.Agreement is tightest on object-centric NoSearch prompts but weakens on TextualSearch, where glyph-level correctness and fine layout dominate rubric variance.

C Extended quantitative results … D.1 End-to-end workflow and reasoner I/O

The supplementary material expands evaluation to a complete nine-component breakdown and specifies a staged, replayable workflow in which a reasoner gates search, filters references, integrates grounded context, and enables the generator and judge to complete the loop.

  • D.1 End-to-end workflow and reasoner I/O: The reasoner’s gate, filter, and integrate stages are stored in the SFT corpus as trajectory types Task A, Task B, and Task C.The supplementary protocol traces the overall workflow and representative input–output pairs, while the released corpus preserves verbatim JSON in the exact schema.
  • C.1 Full nine-component breakdown by stratum: Every generator drops sharply on Search-Intensive, indicating that the bottleneck reflects missing knowledge rather than rendering ability.Knowledge-sensitive components measure knowledge presence, while knowledge-invariant components measure rendering competence.
  • D.1 End-to-end workflow and reasoner I/O: The end-to-end pipeline runs gate →filter →integrate, selects whether and what to search, emits an enriched prompt, renders an image, and judges it against a checklist and rubric.The Yang Chaoyue example traces these steps using scene, costume, and likeness references.
  • D.1 End-to-end workflow and reasoner I/O: The gate searches when a named entity has a critical visual-identity gap and skips generic prompts the generator already renders reliably.For Shanghai Tower, it searches an exterior photograph; for a generic porcelain teapot, it skips because no critical or important gap survives.
  • D.1 End-to-end workflow and reasoner I/O: The filter selects the single candidate that most directly fills the identified gap while minimizing extraneous content.It uses a 0-based index when only one of three retrieved references carries the required location-specific layout.
  • D.1 End-to-end workflow and reasoner I/O: The integrate stage converts selected visual grounding into language by specifying what to borrow with a grounded citation, avoiding unrequested style or layout copying.A polar-bear-in-the-Sahara example borrows an acacia tree’s branching silhouette, cracked bark texture, and hard midday desert lighting.

D.2 Evaluation protocol · E Hardware and implementation details

The evaluation protocol uses an auditable automated judge that scores prompt-specific and generic image qualities on a fixed [0, 3] scale, with reference images serving only as comparison evidence. Difficulty is partitioned by no-search quality, judge fidelity is validated against human ratings, and experiments run on NVIDIA H20 GPUs.

  • D.2 Evaluation protocol: The automated judge receives the prompt, checklist, rubric, and annotated references, while scoring only the designated image.References provide comparison evidence for entity focus and severity but are not themselves scored.
  • D.2 Evaluation protocol: The judge extracts observable evidence, compares reference-dependent checklist items, assigns anchored half-point scores, and emits blocks for nine aggregatable components.Emitted blocks cover reference alignment, checklist, rubric, visual-reference fidelity, and text-reference fidelity.
  • D.2 Evaluation protocol: All components use a [0, 3] scale in 0.5 steps, with each score documented through Anchor, signed Adjustments, and a clamped Computed final.The anchors define 0 as unmet or contradicted and 3 as fully met.
  • D.2 Evaluation protocol: The nine components comprise two prompt-specific components, six fixed generic dimensions, and one reference-conditional component.The conditional component applies when reference images are provided.
  • D.2 Evaluation protocol: The 651 search-intensive prompts are divided into three fixed tercile-based difficulty sets using Nano Banana Pro no-search quality, with a separate 100-prompt NoSearch column.Set I is the easiest third, Set II is intermediate, and Set III is the hardest third.
  • D.2 Evaluation protocol: The judge achieves Spearman ρ = 0.87 with human ratings on 500 held-out prompt–image pairs.The Anchor/Adjustment/Computed diagnostic is intended to encourage falsifiable critiques of individual scores.
  • E Hardware and implementation details: All experiments run on NVIDIA H20 GPUs with 80 GB HBM3, with compute configurations detailed separately for each training phase.The hardware specification applies across the reported experiments.

E.1 VLM reasoner training (Phase 0 and Phase 2) … E.4 Total compute budget

The training pipeline combines supervised reasoner finetuning, generator DPO, and rejection-sampling recalibration, with separate judge serving and distributed-compute infrastructure. Training uses approximately 20,000 annotated trajectories, filtered Phase 2 rollouts, and explicitly separated compute accounting.

  • E.1 VLM reasoner training (Phase 0 and Phase 2): The Qwen3-VL-8B reasoner receives full-parameter supervised finetuning through ModelScope-Swift in Phase 0 and Phase 2.The Phase 0/2 configuration is specified in Table 9.
  • E.1 VLM reasoner training (Phase 0 and Phase 2): Approximately 20,000 expert-annotated gate–filter–compress trajectories train the reasoner, with a 1% held-out validation split.Preprocessing uses 200 workers and 4 dataloader workers.
  • E.2 Generator DPO training (Phase 1): The Flux.2-Klein-4B generator undergoes online iterative DPO with Flow-Factory in Phase 1.The generator configuration is specified in Table 10.
  • E.2 Generator DPO training (Phase 1): The Qwen3-VL-8B judge scores candidates with the SEARCHGEN-20K protocol’s nine dimensions, using 4,096 output tokens, temperature 0.2, and 262,144 pixels per image.Up to 24 requests run concurrently, with the judge served through vLLM over HTTP on a separate node.
  • E.2 Generator DPO training (Phase 1): Each prompt yields 5 candidate images, and preference pairs use the highest- and lowest-scored generations within each group.An SSIM penalty of 0.95 discourages near-identical chosen and rejected images.
  • E.3 Reasoner rejection finetuning (Phase 2): Phase 2 recalibrates the reasoner with rejection-sampling finetuning, retaining only rollouts whose search produces positive group-relative advantage for the strengthened generator.The infrastructure matches Phase 0, including ModelScope-Swift, 8× H20, and the same hyperparameters.
  • E.4 Total compute budget: Table 11 summarizes end-to-end training compute, while the concurrently served VLM judge runs on a separate GPU and is excluded from training GPU-hour totals.Phase 1 GPU-hours equal 8 GPUs × 24 h wall-clock; runs use expandable CUDA segments and NCCL over NVLink within each 8-GPU node.

F Limitations and future directions · G Broader impact

The paper prioritizes depth of insight over exhaustive optimization, identifying iterative co-training, generator scaling, architecture, and reward signals as key future directions. It also provides evaluation resources for an underserved problem while acknowledging misinformation risks and proposed mitigations.

  • F Limitations and future directions: The work prioritizes depth of insight over exhaustive optimization, leaving several promising avenues for future research.This framing motivates the limitations and future-direction agenda.
  • F Limitations and future directions: A single DPO pass followed by a single RFT pass already yields monotonic improvement, but multiple co-training rounds could further sharpen the knowledge boundary.The proposed multi-round protocol alternates generator and reasoner improvement over several cycles.
  • F Limitations and future directions: Experiments use 4B-parameter Flux.2-Klein-4B and 7B-parameter Bagel generators, motivating studies of larger or architecturally different models.Such changes may broaden internalizable knowledge and alter which prompts require search.
  • F Limitations and future directions: Gemini-3-Flash serves as the automated VLM judge and correlates strongly with human ratings at Spearman ρ = 0.87, but automated rewards can remain noisy.The paper notes that automated scoring introduces noise when substituting for human judgment.
  • F Limitations and future directions: DPO is intentionally chosen to resist reward noise by constructing preference pairs from the best- and worst-ranked generations.This design uses relative rankings rather than relying directly on potentially noisy automated scores.
  • G Broader impact: SEARCHGEN-20K and SEARCHGEN-BENCH address an evaluation gap by cataloging twelve failure categories grounded in over 10,000 real user prompts.The resources expose failures that existing benchmarks largely overlook because they emphasize prompts within generators’ parametric knowledge.
  • G Broader impact: Improved rendering of public figures, cultural symbols, and historical events could lower barriers to convincing visual misinformation, while the benchmark measures AIGC fakeness.Released assets are evaluation prompts and search results rather than trained generator checkpoints, and downstream users are encouraged to apply further safeguards.

G.1 Third-party models, data access, and terms of use · G.2 Responsible release and safeguards

The paper uses third-party open-weight generators and commercial APIs under providers’ terms, while releasing research artifacts without model weights or proprietary content. SEARCHGEN-20K and SEARCHGEN-BENCH are released with safeguards that exclude unsafe artifacts and discourage harmful, deceptive, or infringing use.

  • G.1 Third-party models, data access, and terms of use: Open-weight evaluations use Bagel, FLUX.2 checkpoints, and Qwen-Image.The cited passage identifies these as open-weight generators and provides their associated authors and years.
  • G.1 Third-party models, data access, and terms of use: Public documentation does not specify whether commercial endpoints use retrieval, tools, or proprietary knowledge bases during inference.This uncertainty limits direct interpretation of the systems’ internal information-access mechanisms.
  • G.1 Third-party models, data access, and terms of use: Experiments follow each provider’s publicly posted terms and documentation, and releases exclude model weights and proprietary API payloads.The work releases prompts and cached search artifacts instead.
  • G.2 Responsible release and safeguards: The public release provides prompts, annotations, evaluation checklists, rubrics, search queries, and permissible search metadata or derived attributes.It does not include raw scraped images, proprietary web content, or proprietary API payloads without permission.
  • G.2 Responsible release and safeguards: Unsafe search artifacts are filtered or excluded, including explicit sexual content, graphic violence, hateful or extremist material, and personally sensitive content.The benchmark is intended for evaluating search-intensive visual generation, not producing deceptive, infringing, or harmful content; users should respect original-source licenses and terms.
Loading 2607.05382v4…