Source-linked AI summary
Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models
Ke Hao, Yuanzhi Liang, Tingxi Chen, Rui Li, Haibin Huang, Chi Zhang, Yun Gu, Xuelong Li
TL;DR
Unified models bring visual understanding and generation together, but their self-generated images can be unreliable training experiences. GGF grounds each generated dream through flow-level feedback and replay; across three backbones, it improves text-to-image generation and yields modest visual-understanding gains.
Problem
Self-generated images may omit or distort content, so unified models need to ground their visual experience before learning from it.
Source: Introduction · p. 2Method
GGF generates a visual dream, transfers image-grounded generation directions through flow-level feedback, and replays the dream through captioning and re-imagination to ground understanding.
Source: Introduction · p. 2Results
Across three unified multimodal models with different integration designs, GGF consistently improves text-to-image generation and yields modest gains in visual understanding.
Source: Conclusion · p. 8Takeaways & Limitations
Unified multimodal models can self-evolve from text prompts alone when their generated visual experience is grounded before consolidation.
Source: Conclusion · p. 8Repeated training on self-generated samples may cause drift from pretrained behavior, motivating retention of the initial model θ0.
Source: Flow-Level Feedback to Generation · p. 4
Abstract
from arXiv · showhide
Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks. We introduce Generative Grounding Feedback(GGF), a self-evolving post-training framework that uses only text prompts and the model's own visual experience. Given a prompt, the model first generates a visual ``dream.'' Flow-level feedback compares text-, image-, and repair-conditioned predictions at the same noisy latent state, transferring image-grounded generation directions to the prompt condition. Dream replay grounding replays this dream through captioning and re-imagination, training claim-level evidence to remain consistent across the replay while separating unrelated visual experiences. Jointly optimized, these two directions let generation provide visual grounding for understanding and understanding refine subsequent generation without paired image--text supervision. Experiments across unified models with different understanding--generation integration designs show consistent improvements in text-to-image generation together with modest gains in visual understanding.
Introduction
Unified architectures create an opportunity for generation and understanding to learn from the same visual experience, but self-generated images must be grounded before reuse. GGF couples flow-level feedback with dream replay grounding, and improves both capabilities across three unified backbones.
Introduction
- A generated image can train understanding when paired with its prompt, while semantic evidence from the understanding branch can guide future generation.This creates a bidirectional learning loop from one self-generated visual experience.Source: Introduction · p. 1
- Self-generated images may omit objects, distort attributes, violate spatial relations, or contain artifacts, so blindly training on them risks amplifying model errors.GGF addresses the need to assess whether a dream supports the intent that produced it before consolidating signals.Source: Introduction · pp. 1–2
- GGF generates a visual dream, then replays it through captioning and re-imagination to ground claims in evidence consistent across the replay and separate unrelated visual experiences.Source recovery, replay consistency, and semantic separation provide supervision for visual understanding.Source: Introduction · p. 2
- Flow-level feedback compares text-, image-, and repair-conditioned predictions at the same noisy latent state, transferring image-grounded, repair-enhanced directions to prompt-conditioned generation.The generated image also serves as a grounding sample for understanding.Source: Introduction · p. 2
- Across three unified multimodal backbones, GGF improves text-to-image generation and visual understanding.The introduction presents this as validation across different architectural families.Source: Introduction · p. 2
Related Work
Prior work has tightened architectural integration between visual understanding and generation and explored transferring supervision across capabilities. The related work distinguishes those efforts from reciprocal self-evolution using text prompts alone, which remains underexplored.
Unified multimodal models
- Unified multimodal architectures range from shared autoregressive frameworks to designs with decoupled visual pathways or diffusion- and flow-based synthesis, alongside systems connected through modality bridges or aligned representations.Examples include Chameleon (Team 2024), Janus (Wu et al. 2025a), and JanusFlow.Source: Related Work · p. 2
Cross-capability transfer between understanding and generation
- Cross-capability transfer includes understanding features or feedback for generation and generated visual evidence or auxiliary supervision for understanding, but these approaches largely remain one-way.Examples include RecA (Xie et al. 2025a), TIFA, DeGF, and GAS (Guo et al. 2026).Source: Related Work · p. 2
Self-improvement and reciprocal evolution
- Self-improvement methods filter generated experience, use understanding-derived preferences or rewards, or jointly optimize both capabilities, often relying on curated supervision, external images, or reward construction.The review identifies reciprocal evolution from text prompts alone, without external visual experience or task-specific supervision, as underexplored.Source: Related Work · pp. 2–3
Method
GGF uses a prompt-generated visual dream as shared experience for feedback between visual understanding and generation. It combines flow-level feedback with dream replay grounding and optimizes both directions jointly.
Self-Generated Visual Experience
- The model encodes its generated dream as a latent, samples a flow timestep and noise, and evaluates image, prompt, combined, and repair-conditioned predictions at the same noisy state.The four contexts are image, prompt, image plus prompt, and image plus prompt plus a quality-repair instruction.Source: Overview · p. 3, Self-Generated Visual Experience · p. 3
Overview
- GGF couples the understanding and generation branches by comparing how text, image, and repair conditions steer a shared visual dream, then replaying its captioned semantics.The three-step framework dreams a visual experience, refines generation with understanding feedback, and grounds understanding through generative replay.Source: Overview · p. 3
Flow-Level Feedback to Generation
- Flow-level feedback aligns prompt-conditioned generation with image-conditioned and repair-enhanced directions, while an anchor preserves the pretrained generation prior.The repair-enhanced direction transfers condition-induced guidance rather than simply reproducing the generated image.Source: Flow-Level Feedback to Generation · pp. 3–4
Dream Replay Grounding
- Replay grounds understanding through captioning and re-imagination, preserving semantic agreement, returning to the initial dream, recovering its source prompt, and separating unrelated dreams.The independently generated dream serves as an off-manifold reference for separating unrelated visual experience.Source: Dream Replay Grounding · pp. 4–5
Bidirectional Grounding Optimization
- Joint optimization uses understanding-encoded visual evidence to refine generation while generative replay improves visual understanding, with both directions originating from the same dream.Both branches are updated within each training step.Source: Bidirectional Grounding Optimization · p. 5
Experiments
Across three unified backbones, GGF improves text-to-image generation and visual understanding, while ablations and comparisons support the value of its coupled feedback objectives. The experiments also show qualitative generation changes and improving benchmark trends during BAGEL training.
Experimental Setup
- Experiments use three backbones spanning mixture-of-transformers, shared-transformer, and bridge-style integration, with generation and understanding evaluated on the same post-trained checkpoint.GGF requires no paired image–text data; each backbone is trained for 1,000 steps using 1,000 dreaming prompts and a disjoint set of 1,000 replay prompts.Source: Experimental Setup · p. 5
Impact on Understanding
- On BAGEL and Show-o2, GGF raises MME by more than 20 points; BAGEL HallusionBench aAcc increases from 51.8 to 59.4 and fAcc from 24.3 to 31.2.POPE accuracy and F1 improve slightly on both backbones, while SEED-Bench remains essentially unchanged; Show-o2 improves across all three HallusionBench evaluation levels.Source: Impact on Understanding · p. 6, Table 2 · p. 7
Ablation Study
- On BAGEL, the complete objective performs best on GenEval and MME, and removing any component degrades both metrics.The generation anchor has the largest GenEval effect within flow-level feedback, cross-condition alignment has the strongest MME effect in that group, and removing semantic separation lowers MME by nearly 19 points; replay alignment and cycle consistency also contribute clear gains.Source: Ablation Study · pp. 6–7, Table 3 · p. 7
Text-to-Image Generation
- BAGEL’s GenEval score rises from 84.30 to 86.97, and GGF consistently improves overall GenEval and DPG-Bench across all three backbones.Gains are clearest on compositional dimensions; single-object accuracy changes only marginally, while DPG-Bench gains extend to denser prompts.Source: Text-to-Image Generation · p. 6, Table 1 · p. 7
Qualitative Results
- Compared with the pretrained model on identical prompts and sampling settings, GGF generations more often correct malformed details and match requested counts, spatial relations, and attributes.These qualitative changes align with gains on GenEval counting, position, and color-attribution dimensions.Source: Figure 3 · p. 6, Qualitative Results · p. 7
Comparison with Post-Training Methods
- GGF achieves the strongest joint result across GenEval, DPG-Bench, MME, and POPE, while reproduced SRUM slightly improves DPG-Bench but lowers GenEval and MME.GGF completes 1,000 steps in approximately 4.1 hours on two H100 GPUs without a separate scoring stage.Source: Comparison with Post-Training Methods · p. 7, Table 4 · p. 8
Training dynamics
- On BAGEL, smoothed training loss decreases while GenEval and MME rise overall during optimization.Smoothing suppresses fluctuations from prompt sampling and stochastic self-generated trajectories.Source: Training dynamics · p. 8, Figure 4 · p. 8
Conclusion
GGF couples understanding and generation through grounded reuse of the model’s own visual dreams, using flow-level feedback and replay grounding without paired image–text supervision. Across three unified models, it consistently improves text-to-image generation and modestly improves visual understanding.
Conclusion
- Across three unified multimodal models, GGF consistently improves text-to-image generation and yields modest gains in visual understanding.The results support self-evolution from text prompts alone when generated visual experience is grounded before consolidation.Source: Conclusion · p. 8
- GGF reuses each generated dream through flow-level feedback and replay grounding to couple visual understanding and generation without paired image–text supervision.Replay grounding trains claim-level evidence to remain consistent across captioning and re-imagination while separating unrelated experiences.Source: Conclusion · p. 8
- The joint feedback directions let generation ground understanding while understanding refines subsequent generation.Flow-level feedback transfers image-grounded and repair-enhanced directions to prompt-conditioned generation while preserving the pretrained generation prior.Source: Conclusion · p. 8