PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling

Keming Wu, Zuhao Yang, Kaichen Zhang, Shizun Wang, Haowei Zhu, Sicong Leng, Zhongyu Yang, Qijie Wang, Sudong Wang, Ziting Wang, Zili Wang, Hui Zhang, Haonan Wang, Hang Zhou, Yifan Pu, Xingxuan Li, Fangneng Zhan, Bo Li, Lidong Bing, Yuxin Song, Ziwei Liu, Wenhu Chen, Jingdong Wang, Xinchao Wang, Xiaojuan Qi, Shijian Lu, Bin Wang

arXiv:2604.28185v2cs.CV

TL;DR

Visual generation models remain fragile when correctness depends on exact structure, persistent state, verification, or causal grounding. This paper introduces a five-level capability taxonomy and combines benchmark review, stress tests, and case studies to map progress and expose these weaknesses. Across diverse tasks, current systems are strong at semantic and visual plausibility but remain unreliable on structurally, temporally, and causally grounded generation.

  • Problem

    Current visual generation models remain fragile when correctness depends on exact structure, persistent state, verification, or causal grounding.

  • Method

    The paper introduces a five-level capability taxonomy and combines benchmark review, in-the-wild stress tests, and expert-constrained case studies.

  • Results

    Across spatial, physical, editing, diagrammatic, interface, and embodied tasks, models are increasingly plausible visually and semantically but remain fragile on exact structure, persistent state, verification, and causal grounding.

  • Takeaways & Limitations

    The capability-centered framework clarifies what visual generation systems improve and where their correctness remains limited.

  • Takeaways & Limitations

    Sequential in-context editing can cause cumulative pixel-level quality degradation because repeated encode/decode cycles introduce irreversible visual drift.

Abstract

from arXiv · show

Recent visual generation models have made major progress in photorealism, typography, instruction following, and interactive editing, yet they still struggle with spatial reasoning, persistent state, long-horizon consistency, and causal understanding. We argue that the field should move beyond appearance synthesis toward intelligent visual generation: plausible visuals grounded in structure, dynamics, domain knowledge, and causal relations. To frame this shift, we introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation, progressing from passive renderers to interactive, agentic, world-aware generators. We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, synthetic data distillation, and sampling acceleration. We further show that current evaluations often overestimate progress by emphasizing perceptual quality while missing structural, temporal, and causal failures. By combining benchmark review, in-the-wild stress tests, and expert-constrained case studies, this roadmap offers a capability-centered lens for understanding, evaluating, and advancing the next generation of intelligent visual generation systems.

1 Introduction

The paper frames visual generation as a shift from one-shot appearance synthesis toward controllable, persistent, interactive, and physically grounded visual intelligence. It organizes this evolution through a five-level taxonomy, analyzes its technical and data drivers, and argues that stress tests reveal gaps hidden by aesthetic metrics.

  • Current Capabilities and Blind Spots: Modern systems improve in photorealism and editing while still failing puzzle-like spatial reconstruction, multi-step state-transition editing, and persistent character identity.The introduction also identifies physical reasoning as a persistent blind spot.
  • Capability Taxonomy: The roadmap proposes five capability levels: Atomic, Conditional, In-Context, Agentic, and World-Modeling Generation.The taxonomy clarifies what it means for visual models to become more intelligent.
  • Modeling and Training Drivers: It synthesizes evolution drivers spanning flow matching, unified understanding-and-generation systems, visual representations, alignment, reward modeling, and efficiency engineering.The paper emphasizes both architectural changes and pre-training, post-training, and inference improvements.
  • Applications as Constraints: Applications increasingly require structure, memory, state tracking, and closed-loop control, progressing from conditional generation and domain adaptation toward embodied interaction.The application progression includes reasoning-driven editing and embodied interaction.
  • Evaluation and Stress Testing: High aesthetic quality does not imply mastery of spatial logic, physical consistency, or long-horizon visual reasoning.Real-world stress tests complement benchmark-based review by exposing these limitations.

2 The Evolution of Visual Intelligence

The paper proposes a nested five-level taxonomy of visual intelligence, progressing from one-shot rendering through controllable, context-rich, agentic, and causally grounded generation. The taxonomy distinguishes capabilities by forward-pass context, multi-call orchestration, and anchoring in physical and causal world knowledge.

  • Taxonomy: The five levels characterize visual-generation competence rather than architecture or task labels, with each level subsuming its predecessors and adding one qualitatively new capability.The taxonomy frames progress as a transition from passive statistical rendering to goal-directed, physically informed visual intelligence.
  • L4: Agentic Generation: L4 introduces an external controller and multiple calls, but systems still fail when deciding, verifying, and correcting actions in open-ended loops.Memory, retrieval, and tool-grounded editing expose noisy evidence, brittle verification, proxy-optimized trajectories, and silent drift.
  • L5: World-Modeling Generation: L5 requires causal faithfulness: generated future states must reflect intervention consequences rather than statistical correlates, as demonstrated by interactive neural game engines and embodied visual systems.Ignoring commanded steering or contact dynamics can corrupt downstream policy-learning signals, especially under unusual mechanisms.
  • L1: Atomic Generation: L1 performs independent, one-shot prompt-to-image mapping, prioritizing distributional plausibility over precise spatial layout, object count, identity, or structural constraints.Its central limitation is uncontrolled variation: relations such as left-right placement may be violated despite visually pleasing outputs.
  • L2: Conditional Generation: L2 adds explicit structural or multimodal conditions, making generation controllable while leaving spatial precision and attribute binding vulnerable to leakage and confusion.Examples include depth maps, edge sketches, segmentation layouts, reference images, and identity embeddings.
  • L3: In-Context Generation: L3 remains a single forward pass but absorbs multiple references, conditions, or accumulated editing history, with memory implicit in the growing input rather than an external controller.This operational definition places multi-turn editing at L3 because each round processes the accumulated sequence in one shot.

3 Model, Architecture, and Method

The section presents visual generation as a converging architecture space built around latent representations, diffusion/flow backbones, unified generation-editing pathways, and complementary autoregressive planning. It also distinguishes renderer capabilities from system-level control, arguing that closed-source systems increasingly add agentic multi-call orchestration beyond open models’ single-pass designs.

  • Hybrid architectures: Autoregressive and diffusion families solve complementary subproblems, making hybrid designs a central pattern in which AR plans semantics and diffusion or flow produces high-fidelity images.Transfusion exemplifies this division by combining causal attention for text with bidirectional attention for image patches and a hybrid cross-entropy/diffusion loss.
  • Unified generation and editing: Recent systems increasingly unify text-to-image, image-to-image, and instruction-based editing through one backbone, treating tasks as different input and conditioning flows.By spring 2026, 60% of recent frontier reports reportedly used fully unified architectures.
  • Visual representations: Latent generation makes high-resolution synthesis tractable by mapping pixels into compact representations, with 8× spatial compression reducing backbone computation by 64×.A 512 × 512 RGB image contains 786,432 dimensions, motivating the encoder’s central compression–fidelity trade-off.
  • Backbone design: MM-DiT’s separate text and image streams with joint attention improved text comprehension and typography fidelity, establishing a template later adopted by systems such as FLUX.This design enables bidirectional information exchange while preserving modality-specific parameter streams.
  • Frontier convergence: A review of ten frontier reports found convergence around Qwen-VL text encoding, MM-DiT or hybrids, and flow matching, while U-Net was effectively retired at the frontier.Qwen-VL appeared in 6 of 10 reports, particularly for Chinese rendering and long-prompt comprehension.
  • System-level control: The open–closed gap is framed primarily as system architecture: closed systems plausibly achieve L4 agentic generation through multi-call control flow, whereas open systems remain L3-bounded by single forward passes.This distinction places orchestration around the renderer, rather than the renderer alone, at the center of the capability gap.

4 Training and Inference

Training pipelines for frontier visual generators are converging on staged curricula, aggressive data refinement, architecture-dependent SFT, and preference-based post-training. Remaining differentiation lies mainly in caption quality, resolution-specific filtering, reinforcement-learning variants, checkpoint merging, and finer-grained credit assignment.

  • Training-pipeline consensus: Resolution graduation begins at 256^2 for semantic composition, then scales through 512^2 → 1024^2 stages for detail refinement and production-resolution training.Low-resolution training consumes more than half of Z-Image’s total compute, while CT crosses the production-resolution threshold through intermediate mixed-aspect-ratio sub-stages.
  • Training-pipeline consensus: Frontier training recipes converge on four stages, specialist-checkpoint merging, and a DPO + GRPO reinforcement-learning wrapper.Across ten reports, macro-level recipe discovery has largely given way to execution quality, with variation concentrated in captioning, CT filters, RL variants, and merging.
  • Data curation and SFT: Data quality, rather than parameter count, increasingly sets SFT’s capability ceiling, motivating profiling, semantic balancing, geometric grounding, and failure-aware refinement.The curation pipeline converts noisy web-scale data into stronger supervision, while final-stage expert-quality data has disproportionate impact on user-visible capabilities.
  • Data curation and SFT: SFT recipes depend on the backbone and increasingly supervise editing, multi-view consistency, reasoning, planning, verification, refinement, memory, and skill selection.Different architectures expose different bottlenecks, while trajectory-based datasets extend supervision beyond prompt-to-image pairs to intermediate reasoning and action traces.
  • Preference optimization: DPO updates generators directly toward preferred outputs, while GRPO variants support online exploration, unified multimodal optimization, non-paired rewards, and flow-matching alignment.Examples include Unified-GRPO, UniGRPO, DiffusionNFT, and Flow-GRPO, with denoising reduction enabling efficient online exploration at approximately T ≈ 10 steps.
  • Preference optimization: Preference supervision should be distributed across denoising trajectories because early steps determine global semantics and layout, whereas later steps refine local details.This reframes diffusion alignment as process-level credit assignment rather than a uniform terminal-reward or RL-versus-DPO choice.

5 Resources and Infrastructure

Visual-generation infrastructure increasingly engineers training signals through active curation, synthetic-data distillation, relabeling, and multi-source construction rather than relying on naturally occurring data. State-of-the-art pipelines balance scale, quality, cost, and reproducibility through aggressive filtering, hybrid datasets, frontier-model teachers, and complementary data modalities.

  • Data Construction: Active engineering uses foundation models to construct training signals, creating a data flywheel where stronger models produce synthetic data for stronger successors.91K frontier-quality samples can match millions of web-scraped samples, but proprietary-model distillation ties the quality ceiling to access terms.
  • Data Construction: Data construction spans complementary modalities: real images provide realism, user edits capture intent, videos provide temporal pairs, 3D assets enable disentanglement, and designs cover specialized needs.No single modality satisfies all requirements, motivating combinations across diffusion editing, video extraction, automated pipelines, expert edits, and community pairs.
  • Data Construction: Caption and VLM-driven relabeling has become a dominant training lever, raising capability ceilings even when architecture and compute remain roughly fixed.The passage attributes Z-Image’s “6B competitive with 20B+” result and LongCat-Image’s advantage to aggressive captioning and tight curation.
  • Scale and Quality: Pipelines trade scale against quality, ranging from 20M candidates retaining 1M to 12K ultra-high-resolution images and 1K expert-level prompts.Hybrid approaches dominate, combining synthetic samples with real-world expert edits, multi-turn sequences, or real-image anchors to reduce generation bias.
  • Frontier Distillation: Frontier distillation achieves competitive performance at modest dataset scale by exchanging inference cost for reusable training signal, while preserving dependence on implicit teachers.ShareGPT-4o-Image uses 91K samples, and Pico-Banana-400K uses 386K samples at ∼$100K USD.
  • Industrial Curation: Industrial pipelines favor aggressive overgeneration and strict selection, with retention ranging from under 45% to ∼6% and 5%, while filtering strategy depends on generation costs and quality floors.Retention choices span aggressive overgeneration with strict multi-stage selection and targeted generation with lighter curation; open pipelines remain challenged by error accumulation and homogenization.

6 Applications and Evolving Frontiers

The section organizes visual-generation applications by the constraints they impose, spanning conditional control, domain-specific rule adherence, reasoning-driven editing, and embodied interaction. Across these settings, the field is shifting from perceptual plausibility toward controllable, structurally consistent, and temporally grounded generation.

  • Application taxonomy: Applications are organized into four branches: conditional image generation, custom domain adaptation, conditional image editing, and embodied-domain generation.These branches correspond to fine-grained controllability, constructionist domain rules, reasoning-driven state transitions, and support for physical agents.
  • Conditional image generation: Conditional generation reduces language ambiguity by adding spatial maps, reference images, identities, 3D cues, or other auxiliary control signals.Identity customization has progressed from rigid subject preservation toward articulated whole-body character consistency, but pose changes, occlusion, deformation, and cross-view variation remain challenging.
  • Conditional image generation: As identity and spatial control improve, multi-entity compositions and long-horizon narratives increasingly require explicit state management to limit feature bleeding across subjects or frames.Region-specific identity routing is one example of using masks to reduce cross-subject interference.
  • Custom domain adaptation: Custom domain adaptation addresses the mismatch between continuous foundation-model priors and discrete rules governing layout grammar, topology, coordinates, layers, and 3D structure.Layout conditioning and layered representations expose spatial, occlusion, transparency, and editability constraints, while recent developments remain tied to narrowly defined domains.
  • Embodied-domain generation: Embodied visual generation and editing enrich interaction datasets at low real-world collection cost, while visual prediction encodes environment dynamics and temporal structure for sequential decision-making.The goal is to support informed decisions in interactive settings rather than produce visually accurate frames in isolation.

7 Assessment for the Future: Stress Testing the Limits · 7.1 Methodology: From Benchmarks to "In-the-Wild" Scenarios

The paper argues that standard visual-generation benchmarks mainly test lower-level realism and prompt following, so it supplements them with structured in-the-wild stress tests mapped to the five-level taxonomy. It also treats AI-assisted paper figures as embedded probes requiring expert verification rather than formal benchmarks.

  • 7 Assessment for the Future: Stress Testing the Limits: Standard metrics such as FID, CLIP-Score, and isolated prompt-following measures predominantly assess atomic and conditional generation capabilities.These benchmarks do not directly establish progress toward in-context, agentic, or world-modeling generation.
  • 7 Assessment for the Future: Stress Testing the Limits: The evaluation therefore adds carefully designed in-the-wild stress tests targeting the boundary conditions of each taxonomy level.The tests are intended to probe higher-level visual capabilities beyond conventional benchmark settings.
  • 7 Assessment for the Future: Stress Testing the Limits: Each stress-test dimension is explicitly mapped to the level it primarily challenges, turning frontier evaluation into structured evidence about the taxonomy’s current limits.This design distinguishes systematic capability assessment from anecdotal failure reporting.
  • 7.1 Methodology: From Benchmarks to "In-the-Wild" Scenarios: Nano Banana and GPT-Image-2 are evaluated with identical prompts and settings, enabling direct comparison without changing task definitions.Generator identity is reported in each figure caption.
  • 7.1 Methodology: From Benchmarks to "In-the-Wild" Scenarios: Several explanatory figures are generated from expert-authored descriptions and used as embedded probes of structured visual-generation capability, not formal benchmarks.Their prompts and selection process are not treated as standardized evaluation procedures.
  • 7.1 Methodology: From Benchmarks to "In-the-Wild" Scenarios: These AI-assisted figures can reveal missing relations, scientifically inaccurate structure, weak symbolic grounding, and unstable text rendering despite visual plausibility.The cited failure modes overlap with the issues targeted by the stress tests.
  • 7.1 Methodology: From Benchmarks to "In-the-Wild" Scenarios: Final correctness of AI-assisted figures still requires expert verification.The figures are informative probes, but their outputs are not accepted as self-validating evidence.

7.2 Dimension I: Spatial Structuring & Layout Precision

This section evaluates spatial structuring through rigid jigsaw reconstruction and explicit layout tasks. Current models often reproduce semantic or visual structure while failing exact geometric, coordinate, topological, and object-permanence constraints.

  • Jigsaw Reconstruction: Jigsaw reconstruction exposes a gap between semantic success and geometric failure: models recognize balloons and sky but hallucinate textures instead of matching rigid puzzle edges.Outputs fuse distinct pieces, dissolve tabs and blanks, and prioritize a plausible completed image over spatially valid reconstruction.
  • Jigsaw Reconstruction: The jigsaw case indicates that current models rely on probabilistic correlation rather than causal or physical reasoning over persistent image patches.Even an Agentic prompt does not make models treat patches as rigid bodies with permanent properties.
  • Explicit Layout Constraints: Metro-map generation captures professional visual grammar and several high-level requirements but violates graph-level topology, including the four-line central-station constraint.The output is visually plausible yet not topologically valid.
  • Explicit Layout Constraints: 13m15s of generation still produced multiple metro-map constraint violations, whereas GPT 5.5 identified all major mismatches in 9s when given the image and prompt jointly.This contrast highlights the potential value of post-hoc verification for detecting failures that generation does not prevent.
  • Explicit Layout Constraints: The tactical-map example is visually convincing but shifts requested objects from F5 and G6 to approximately F6 and G7, and places D7 closer to D8.These subtle errors preserve map coherence while violating discrete coordinate-placement rules.
  • Explicit Layout Constraints: Across structured layouts, models treat symbolic coordinates and relations as soft cues rather than deterministic addresses, yielding coherent images without verifiable grid state.The recurring failure is procedural rather than aesthetic: visual patterns are synthesized, but discrete spatial relations are not reliably maintained.

7.3 Dimension II: Physical Reasoning & Causal Fidelity

The section evaluates physical reasoning through counterfactual visual state changes, manipulation, action planning, functional persistence, and volumetric transformations. Results show emerging causal and geometric understanding, but failures in temporal consistency and executable physical reasoning remain.

  • Fluid Dynamics: In a citrus-slice intervention, the model adds trailing bubbles and altered light refraction when the object sinks, indicating causal artifacts beyond positional plausibility.The test also evaluated spatially aligned vector annotations for physical concepts such as surface tension and Archimedes’ principle.
  • Fluid Dynamics: The sinking case suggests a bridge from L2 Conditional Generation toward L5 World-Modeling Generation through reasoning-conditioned document editing.The model’s final output supports this direction despite a redundant search for consistency in its Thinking Trace.
  • Robotic Manipulation: During robotic manipulation, the model preserves object permanence, recognizes a cup’s cylindrical affordance, and wraps fingers around a mug without mesh interpenetration.These behaviors suggest sensitivity to friction and surface normals during visually generated contact.
  • Visual Action Planning: The generated spoon trajectory follows a natural joint-constrained arc and correctly models spoon containment inside a bowl, but its orientation is wrong in the first two frames.This contrast separates emerging kinematic intuition from persistent visual-detail consistency.
  • Functional Causal Persistence: In humanoid replacement, identity persists across frames, but the model fails to preserve the pouring action’s functional causal persistence.The example exposes a gap between pixel-space control and physical simulation, with sequential actions rendered as texturally consistent static states.
  • Volumetric Reasoning: Vegetable transformations preserve internal material properties, including zucchini diameter and interior color and carrot-shaving thickness and curvature, while slice arrangements remain stochastic.The case advances counterfactual state synthesis but still lacks a rigid kinematic path grounded in specific tool movement.

7.4 Dimension III: Visual-Textual Integration & Logic

This section presents a physics-tutoring case that tests whether a model can combine OCR, diagram grounding, symbolic reasoning, and visual re-rendering in one coherent workflow. The results suggest that polished annotated outputs may conceal iterative reasoning, with the capability best understood as interactive reasoning on images rather than image synthesis alone.

  • Experimental Setup: The case requires solving a Chinese Gaokao-style electromagnetism problem and returning an annotated solution image instead of plain text.The workflow combines parsing the question and diagram, identifying physical laws, deriving equations, and writing the solution onto the original visual artifact.
  • Experimental Setup: The task jointly tests dense-document OCR, diagram grounding, symbolic physics reasoning, layout planning, and readable equation rendering.Variables and field directions must remain aligned with the figure, while annotations preserve the source question and diagram.
  • Observed Behavior: The generated image appears to reconstruct an expert workflow, deriving entry velocity, induced current, Ampere force, and coupled physical quantities.The output is more than decorative annotation: it presents a visually integrated worked solution spanning multiple stages of the physics problem.
  • What the Thinking Trace Reveals: The reasoning trace is less smooth than the polished image, repeatedly revisiting velocity relations, friction assumptions, and force balances at magnetic-field transitions.This contrast indicates that the final visual presentation can mask substantial internal re-analysis.
  • Capability Analysis: The task’s cognitive core is OCR → diagram grounding → symbolic reasoning → structured visual re-rendering, while image editing supplies document-level placement and preservation decisions.The system must decide where to place derivations, whether to extend the canvas, how to preserve the diagram, and how to maintain contrast and annotation style.
  • Insight: The case reveals a capability frontier in using images as interactive reasoning canvases, requiring coherent composition of OCR, math reasoning, and rendering rather than isolated benchmark skills.A likely mechanism is VLM-first reasoning followed by renderer-second projection into the image, whether explicit or unified.

7.5 Dimension IV: Multi-Turn Editing — Markovian Chaining and Silent Drift

Multi-turn editing tests whether systems preserve full edit history, selectively retrieve relevant past states, and maintain cross-turn invariants rather than relying on a Markovian last-image shortcut. Pixel-level re-rendering and semantic reference-frame shifts produce cumulative silent drift, exposing limits of current in-context editing.

  • Core Ability: Multi-turn editing requires remembering history, retrieving relevant turns, and preserving invariants, but many systems retain only the last image and current prompt.This Markovian shortcut causes locally acceptable edits to accumulate silent drift across turns.
  • Observed Behavior: Nano Banana re-renders the full four-panel page each turn, causing compression artifacts, text drift, and micro-shifts in non-edited regions.The top-left girl’s facial expression at T2 no longer matches its T0 rendering despite T2 targeting another panel.
  • Insight: Every round incurs a VAE round-trip with non-zero quantization loss, so long context preserves editing capacity but cannot guarantee pixel-level fidelity or frozen regions.The resulting degradation is structural to current single-forward in-context systems rather than an engineering bug.
  • Observed Behavior: In semantic editing, the cat succeeds under an explicit size change but silently changes during an unrelated mouse addition, then fails to restore its original state.The model enlarges the cat in the wrong direction and drops the mouse when “original” is interpreted from the drifted frame.

7.6 Dimension V: Human-Centric Heredity & Aesthetic Editing

This section tests whether visual generation can reason about heredity, coordinated cosmetic modification, and culturally specific styling while preserving identity. The cases show strong underlying capabilities, but hereditary blending remains highly sensitive to prompt wording.

  • Heredity: Explicit blending cues elicit genuine trait blending, whereas minimal prompts produce a compositional shortcut that combines source regions rather than hereditary features.The minimal prompt largely copies the mother’s appearance while borrowing the father’s clothing, gesture, and background; the explicit cue yields a plausible blend accepted as their child.
  • Cosmetic editing: A vague beautification request is decomposed into coordinated edits across hairline, facial contour, eyes, and skin while preserving identity.The modified portrait simultaneously advances the hairline, slims and defines the face, enlarges the eye region, and smooths the skin.
  • Cosmetic editing: An “analysis chart” request produces a surgical-consultation document with annotated measurements, procedures, clinical terminology, and educational disclaimers rather than a modified portrait.This output adapts the modality to the prompt and reflects substantial plastic-surgery domain knowledge beyond surface-level beautification.
  • Culturally specific styling: The model resolves the culturally specific term “foil perm” into short, tight, lofted curls, integrates the hairstyle naturally, and leaves facial identity, expression, and clothing largely unchanged.The case demonstrates culturally grounded world knowledge and clean localized editing without visible hair–face boundary artifacts.

7.7 Dimension VI: Low-level Vision Tasks

The model demonstrates broad low-level editing and object-recognition abilities, but depth estimation and faithful signal recovery remain unreliable. Its restoration outputs often prioritize perceptual cleanliness and detail over reconstructing the precise underlying image.

  • Object Recognition and Depth Estimation: On an unseen, complex real-world image, the model identifies most foreground objects and their outlines but fails substantially at depth estimation.It assigns many objects the same depth-indicating shade and misses several background people’s outlines.
  • Image Restoration: Across super-resolution, low-light enhancement, denoising, deraining, and deblurring, the model produces substantially cleaner, sharper, and more visually pleasing outputs.The evaluation spans synthetic and real-image degradations from Set5, LOL, BSD68, Rain100H, and GoPro.
  • Restoration Fidelity: Despite strong perceptual improvements, restoration outputs frequently add, alter, or reimagine details instead of faithfully recovering the clean-image signal.This detail hallucination is especially evident in low-light enhancement and differs from simple over-smoothing.
  • Underlying Mechanism: The model behaves more like a prior-guided generative image corrector than a classical inverse-problem solver performing exact physical restoration.Degraded inputs act as constraints, while prior-driven image synthesis appears to supply much of the final visual quality.
  • Overall Implication: These results indicate general low-level editing capability across heterogeneous degradations, while reconstruction fidelity remains distinct from making degraded images look clean, sharp, and natural.The distinction matters because perceptual success and precise recovery are not equivalent in low-level vision.

7.8 Dimension VII: Cross-Disciplinary Real-World Applications

The section tests frontier image models as practical tools across historical planning, sports interfaces, programming, mathematics, culinary communication, and biomedical infographics. These cases show strong structural, semantic, and document-rendering capabilities, while exposing persistent weaknesses in fine-grained logical consistency and robustness.

  • Historical City Planning: The model accurately reconstructs an ancient capital’s orthographic view, city walls, and grid-like ward system, but produces blurry Chinese characters and repeated gate names.Its visual layout and historical style are strong, while precise text rendering and textual logic remain unreliable.
  • Sports Interface Generation: The model creates a polished football UI with real Bournemouth players and readable English, yet introduces incorrect disciplinary icons, generic positions, and a poorly drawn budget bar.The errors reveal a gap between broad world knowledge and strict fine-grained consistency.
  • Programming Workflows: GPT-Image-2 solves both Fibonacci Number and Trapping Rain Water while preserving two-panel LeetCode layouts, syntax highlighting, indentation, and readable high-resolution text.The examples demonstrate read-solve-render behavior across problem comprehension, algorithm selection, Python formatting, and visual composition, but remain a capability probe rather than a complete coding benchmark.
  • Mathematical Diagram Synthesis: GPT-Image-2 synthesizes a structured proof diagram with organized geometry, repeated 108◦ angle annotations, labeled segments, side ticks, and boxed algebraic identities.The case indicates structure-aware diagram synthesis that couples spatial construction with symbolic semantics, though visual plausibility alone does not establish correctness.
  • Knowledge-to-Communication Applications: The model produces polished multilingual culinary documents and biomedical infographics that organize specialized knowledge into visually coherent, professionally designed communication artifacts.The culinary example supports largely correct English, Chinese, Japanese, and Korean meanings, while the biomedical example combines topic understanding, information prioritization, hierarchy, iconography, and long-form text rendering.

7.9 Dimension VIII: High-level Vision Tasks

High-level vision tasks test whether multimodal models convert visual understanding into explicit, structured, actionable predictions rather than merely plausible images. Across OCR, keypoints, segmentation, and detection, models capture global structure but remain less reliable at fine-grained symbolic, geometric, and instance-level precision.

  • Task scope: Four tasks—OCR, keypoint estimation, semantic segmentation, and object detection—evaluate transitions from holistic perception to structured visual prediction.They span text decoding, articulated pose modeling, dense region parsing, and instance-level localization.
  • OCR: OCR preserves dominant text layout and substantial salient content, but small or degraded text still suffers character omissions and corruption.The model better preserves text existence and arrangement than exact symbolic content.
  • Keypoint estimation: Keypoint estimation recovers global pose structure and dominant articulation, while occluded, overlapping, or ambiguous joints reduce landmark precision.Predicted landmarks may drift toward nearby structures even when the overall pose remains plausible.
  • Semantic segmentation: Semantic segmentation produces coherent broad masks and scene partitioning, but boundaries become coarse and leak across adjacent structures in cluttered regions.Semantic coherence is stronger than geometric fidelity.
  • Object detection: Object detection localizes many salient entities with globally sensible boxes, yet crowded scenes produce duplicate boxes, overlapping predictions, and ambiguous labels.The model more reliably identifies an object’s presence than its exact extent and competing-instance identity.

7.10 Summary: Mapping Stress Tests to the Visual Intelligence Taxonomy · 8 Positions and Frontiers

The paper frames visual generation as progressing through a five-level taxonomy toward agentic, tool-augmented, synthetic-data-trained, and world-modeling systems. Its frontier positions emphasize moving beyond passive image synthesis toward structured reasoning, closed-loop action, real-world grounding, and causal simulation.

  • 8 Positions and Frontiers: The roadmap organizes open visual-generation problems within a five-level taxonomy and identifies a focused set of frontiers expected to shape near-term research.Each frontier names an unsolved problem and locates it within the L1–L5 progression.
  • 8.1 Visual Chain-of-Thought (vCoT): Thinking Before Rendering: Visual Chain-of-Thought replaces one-shot text-to-pixel mapping with inspectable intermediate states that support constraint checking, retrieval, and revision.Intermediate representations include textual analyses, layouts, regional decompositions, coarse renderings, symbolic edit programs, and executable visual operations.
  • 8.1 Visual Chain-of-Thought (vCoT): Thinking Before Rendering: vCoT is most useful for hard compositional constraints, external knowledge, or user-steerable planning, but its trace faithfulness, length, latency, and hallucination risks remain unresolved.Explicit reasoning can cost more than it saves on familiar or purely aesthetic prompts.
  • 8.2 Closed-Loop Visual Agents: Generation as Action: Closed-loop visual agents treat generation as an action that can be observed, verified, refined, committed, or rolled back during an agent trajectory.Systems use typed executable edits, reversible creative operations, persistent memory, and learned verifiers.
  • 8.3 Agentic Visual Generation: Tool-Augmented Rendering: Agentic Visual Generation embeds rendering inside execution graphs that combine retrieval, OCR, structured data, geometric engines, localization tools, planning, verification, and refinement.This orchestration addresses world knowledge, accuracy, and real-time information gaps that latent visual priors cannot reliably solve alone.
  • 8.4 Training with Synthetic Data and Visual Self-Play: Synthetic-data training spans frontier-model distillation, reward-model-guided generation, and visual self-play, with domain-specific synthesis targeting structurally constrained generation under sparse real supervision.The central risk of self-training without external grounding is distribution narrowing toward outputs favored by the generator and its self-judge.
  • 8.5 Visual Generation as World Simulation: World modeling advances visual generation from appearance synthesis to predicting scene evolution under intervention, making causal consequences and action faithfulness more important than perceptual quality alone.Neural game engines show that learned generators can replace hand-coded rendering and physics in environments with well-defined visual regularities.
  • 8.5 Visual Generation as World Simulation: At the taxonomy’s apex, interactive world simulation treats images as states in a dynamic closed loop, subsuming controllable rendering, narrative consistency, and interaction while motivating structured-data integration.Databases, spreadsheets, and document corpora are identified as promising first-class inputs for data-centric visual intelligence.

9 Conclusion

The conclusion frames visual generation as a transition from plausible image synthesis toward visual intelligence, organized by a five-level taxonomy from Atomic Generation to World-Modeling Generation. It presents a capability-centered roadmap that connects methods, training, data, applications, stress tests, and a future agenda focused on verifiable visual reasoning.

  • Conceptual framework: The five-level taxonomy organizes progress from one-shot rendering to causal simulation under intervention.Its levels are Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation.
  • Paper scope: The paper reviews visual-generation architectures and the full training lifecycle, including data curation, alignment, reward modeling, and inference acceleration.It covers GANs, diffusion, flow matching, autoregressive, hybrid, and unified multimodal systems, alongside VLM-driven relabeling, continued training, SFT, and preference-based alignment.
  • Contribution: The main contribution is a capability-centered framework for interpreting a rapidly expanding field rather than a catalogue of methods.The framework links the taxonomy to how improvements are produced and uses stress tests to expose remaining shortcomings.
  • Future agenda: The next agenda calls for stronger intermediate reasoning, closed-loop agents, tool-augmented rendering, synthetic-data feedback loops, and evaluations of structural and causal correctness.The conclusion characterizes this agenda as a shift from image synthesis to verifiable visual reasoning, rather than simply scaling existing systems.
Loading 2604.28185v2…