PaperlayerView exact revision on arXiv

Source-linked AI summary

Unlocking Complex Visual Generation via Closed-Loop Verified Reasoning

Hanbo Cheng, Limin Lin, Ruo Zhang, Yicheng Pan, Jun Du

arXiv:2605.14876v2cs.CVcs.AI

TL;DR

Complex-semantic T2I generation is limited by single-step capacity, unverified reasoning, long-context optimization instability, and iterative-inference latency. CLVR addresses these issues with verified trajectory synthesis, PPRL, and DSWM, and reports stronger performance than open-source baselines while approaching proprietary models across benchmarks.

  • Problem

    Single-step T2I systems struggle with complex semantics, while multi-step reasoning faces unverified trajectories, long-context optimization instability, and prohibitive inference latency.

  • Method

    CLVR combines an automated step-verified data engine, Proxy Prompt Reinforcement Learning for long-context alignment, and DSWM for efficient inference.

  • Results

    Across multiple benchmarks, CLVR outperforms most compared open-source baselines and narrows the gap to proprietary models.

  • Takeaways & Limitations

    CLVR provides a system-level approach for general test-time scaling in complex visual generation.

  • Takeaways & Limitations

    DSWM’s theoretical compatibility relies on local linear perturbations, infinitesimal updates, and absence of reward hacking.

Abstract

from arXiv · show

Despite rapid advancements, current text-to-image (T2I) models predominantly rely on a single-step generation paradigm, which struggles with complex semantics and faces diminishing returns from parameter scaling. While recent multi-step reasoning approaches show promise, they are hindered by ungrounded planning hallucinations lacking verification, monolithic post-hoc reflection, long-context optimization instabilities, and prohibitive inference latency. To overcome these bottlenecks, we propose the Closed-Loop Visual Reasoning (CLVR) framework, a comprehensive system that deeply couples visual-language logical planning with pixel-level diffusion generation. CLVR introduces an automated data engine with step-level visual verification to synthesize reliable reasoning trajectories, and proposes Proxy Prompt Reinforcement Learning (PPRL) to resolve long-context optimization instabilities by distilling interleaved multimodal histories into explicit reward signals for accurate causal attribution. Furthermore, to mitigate the severe latency bottleneck caused by iterative denoising, we propose $Δ$-Space Weight Merge (DSWM), a theoretically grounded method that fuses alignment weights with off-the-shelf distillation priors, reducing the per-step inference cost to just 4 NFEs without requiring expensive re-distillation. Extensive experiments demonstrate that CLVR outperforms existing open-source baselines across multiple benchmarks and approaches the performance of proprietary commercial models, unlocking general test-time scaling capabilities for complex visual generation.

1 Introduction

Single-step T2I systems struggle with complex semantics, while scaling capacity yields diminishing returns. CLVR addresses these limitations with verified trajectories, PPRL, and DSWM, and reports stronger cross-benchmark performance than open-source baselines.

  • Single-step T2I generation often mishandles complex prompts, producing attribute confusion, missing entities, and misaligned spatial relations.
  • Increasing model capacity provides diminishing marginal returns, with linear capability gains typically requiring exponential increases in parameters and compute.
  • CLVR connects data synthesis, model alignment, inference mechanisms, and deployment acceleration in a unified framework for visual generation.
  • PPRL stabilizes optimization over extended multimodal contexts and unlocks more general test-time scaling capabilities for visual generation.
  • An automated data engine generates verified, high-quality CLVR trajectories to address the lack of reliable visual Chain-of-Thought training data.
  • DSWM leverages distillation priors to accelerate multi-step visual reasoning and support practical deployment.
  • Across multiple benchmarks, CLVR outperforms most compared open-source baselines and narrows the gap to proprietary models.

2 Related work

Prior reasoning-enhanced T2I methods improve complex semantic alignment but remain limited by insufficient trajectory verification, monolithic coupling, and reward-model or data bottlenecks.

  • Existing visual reasoning trajectories may contain diffusion-side execution failures, mixing reliable and erroneous steps during supervision.
  • Unified multimodal models incur substantial joint training costs and cannot readily exploit advances in independent VLM and diffusion foundations.
  • In multi-step reasoning, conventional reward models struggle to evaluate interleaved multimodal histories, while specialized trajectory data remains scarce for re-distillation.

3 Method

CLVR combines verified trajectory synthesis, proxy-prompt alignment, interactive inference, and weight merging for efficient closed-loop visual reasoning. Its components constrain execution, stabilize long-context training, preserve historical consistency, and reuse distillation priors.

  • CLVR comprises trajectory synthesis with step-level validation, PPRL-based diffusion alignment, and efficient deployment using historical conditioning and DSWM.
  • Closed-Loop Visual Reasoning Data Synthesis: The data engine treats the VLM as a closed-loop controller that assesses the canvas, reasons about semantic gaps, and invokes discrete generation, editing, validation, or termination tools.
  • Closed-Loop Visual Reasoning Data Synthesis: Passive verification gates every generative tool call and discards failed trajectory contexts, preventing generative errors from contaminating the dataset.
  • Closed-Loop Visual Reasoning Data Synthesis: Global filtering retains a trajectory only when two independent judge VLMs agree that its multi-step result exceeds a single-step baseline in instruction following and visual quality.
  • Proxy Prompt Reinforcement Learning: SFT first adapts the VLM and diffusion model to multi-step planning, establishing policy initialization for subsequent reinforcement learning.
  • Proxy Prompt Reinforcement Learning: The training objective truncates complete trajectories into local multimodal contexts, conditioning diffusion generation on prior reasoning and images at each step.
  • Proxy Prompt Reinforcement Learning: PPRL uses an offline foundation VLM to distill long interleaved histories into explicit scene descriptions, editing instructions, and reference-image indices for reward evaluation.
  • Inference: During inference, a VLM router iteratively plans actions from accumulated multimodal history while a context-aware diffusion generator updates the canvas.

4 Experiment

CLVR is evaluated across standard T2I benchmarks and diagnostic probes, with ablations separating prompt rewriting, alignment, and deployment acceleration. Results show stronger complex-semantic performance, resilience to increasing task complexity, and substantial inference-cost reduction.

  • 4 Experiment: CLVR is evaluated on GenEval, GenEval++, ImagineBench, PRISM, and WiseBench against open-source and proprietary reference models.The setup includes SD3.5, T2I-R1, Uni-CoT, GPT-4o, and Gemini 2.5.
  • 4.2 Main results on standard T2I benchmarks: CLVR (9B) substantially outperforms FLUX.2 and surpasses Uni-CoT and T2I-R1 on GenEval, especially for spatial positioning, counting, and multi-object generation.These gains target complex compositional categories rather than only overall benchmark performance.
  • 4.2 Main results on standard T2I benchmarks: 8.830 on ImagineBench, 82.1 on PRISM, and 0.76 on WiseBench show strong cross-benchmark performance for CLVR (9B).On PRISM, CLVR exceeds Qwen-Image at 79.9 and approaches GPT-4o at 86.3; WiseBench approaches GPT-4o at 0.80.
  • 4.3 Semantic complexity probe: CLVR maintains a resilient pass rate across high-complexity tiers, improving AUCpass over FLUX.2 without expanding the DiT backbone.Single-step baselines degrade sharply as Ctask increases and require exponential Ieff scaling for marginal gains.
  • 4.4 Ablation studies: CLVR reaches 0.87 on GenEval and 0.74 on WiseBench, outperforming open-loop prompt rewriting at 0.78 and 0.64 respectively.Prompt rewriting improves WiseBench from 0.48 to 0.64 but degrades GenEval from 0.81 to 0.78.
  • 4.4 Ablation studies: 0.87 on GenEval after DSWM exceeds RL-only at 0.83 and distill-only at 0.81, while reducing denoising from 28 × 2 to 4 NFEs per step.The ablation attributes progressive gains to SFT, PPRL, and then DSWM.

5 Conclusion

CLVR addresses complex-semantic degradation in T2I generation through a system spanning data, training, inference, and deployment. Its components verify trajectories, stabilize long-context alignment, preserve multimodal history, and reduce denoising cost.

  • 5 Conclusion: CLVR uses state-constrained control and step-level visual verification to synthesize reliable training trajectories.The data engine targets ungrounded planning hallucinations in visual reasoning.
  • 5 Conclusion: PPRL distills interleaved image-text histories into explicit single-step rewards for stable long-context alignment.This addresses optimization bottlenecks in long-horizon planning.
  • 5 Conclusion: Trajectory-accumulative conditioning preserves multimodal historical memory during globally consistent inference.The mechanism is intended to mitigate loss of long-horizon dependencies.
  • 5 Conclusion: DSWM reuses distillation priors to reduce denoising cost to 4 NFEs without expensive end-to-end re-distillation.The method decouples CLVR training from distillation acceleration.

A.1 Local Analysis and Geometric Motivation for Weight Merge

The appendix motivates DSWM as a parameter-space merge of distillation and alignment updates. Its analysis treats these updates as approximately compatible under local geometric assumptions.

  • A.1 Local Analysis and Geometric Motivation for Weight Merge: DSWM analyzes whether distillation and alignment increments can be added linearly in parameter space while remaining approximately decoupled in output space.The analysis is local and first-order around the base model weights.
  • A.1 Local Analysis and Geometric Motivation for Weight Merge: Alignment is defined as the combined increment from supervised fine-tuning and reinforcement learning: ∆WAlign = ∆WSFT + ∆WRL.SFT and RL are treated as unified alignment operations.
  • A.1 Local Analysis and Geometric Motivation for Weight Merge: The proof outline assumes local linear superposition and argues that distillation and alignment updates occupy approximately orthogonal normal and tangent subspaces.These assumptions support additive treatment of the two update types.

A.1.1 Linear Superposition under First-Order Perturbation

The first-order analysis justifies merging distillation and alignment perturbations by linearizing the diffusion model around base weights. Empirical norm measurements support the small-perturbation assumption used in this argument.

  • A.1.1 Linear Superposition under First-Order Perturbation: The analysis assumes twice-continuous differentiability and sufficiently small distillation and alignment increments near Wbase.This permits truncating higher-order terms within a local linear perturbation region.
  • A.1.1 Linear Superposition under First-Order Perturbation: 2.79% distilled-model shift, 2.30% full-parameter SFT shift, and 0.0075% RL-LoRA shift support the small-update assumption.The shifts are measured using global relative Frobenius norm on FLUX.2 4B models.
  • A.1.1 Linear Superposition under First-Order Perturbation: The Taylor expansion represents the merged output as the base output plus the Jacobian applied to the summed distillation and alignment increments, plus a second-order remainder.The Jacobian is evaluated at Wbase.
  • A.1.1 Linear Superposition under First-Order Perturbation: Linearity separates the Jacobian response to the merged perturbation into independent distillation and alignment output increments after truncating O(∥∆W∥2).This is the algebraic step completing the first-order superposition argument.

A.1.2 Compatibility for Distribution-based Distillation

The paper argues that distribution-based distillation and alignment updates are approximately compatible because they occupy orthogonal normal and tangent directions around the data manifold. Under stated manifold, optimization, and small-noise assumptions, distillation pulls outputs toward the manifold while alignment redistributes density along it.

  • Assumptions: Distribution-based distillation is analyzed under a low-dimensional data-manifold assumption and healthy, KL-regularized alignment optimization.The analysis assumes alignment remains near the manifold while distillation corrects off-manifold predictions.
  • Normal-space update: Distillation updates primarily reside in the manifold’s normal space because score-based correction pulls off-manifold predictions toward their nearest projected points.The normal component is linked to the shortest correction from a generated point to the manifold.
  • Tangent-space update: SFT and RL alignment updates primarily reside in the tangent space by increasing likelihood or reward while preserving support near the data manifold.SFT favors high-density manifold regions, whereas KL-regularized RL discourages departures from the base-model support.
  • Compatibility: Because normal and tangent spaces are orthogonal, the inner product of distillation and alignment increments is approximately zero.The paper uses this geometric relation to explain compatibility between the two updates.

A.1.3 Compatibility for Trajectory-based Distillation

The trajectory-based analysis treats alignment as a slowly varying vector field active within a limited time window and combines it with trajectory distillation. Under local smoothness and linearity assumptions, the merged model approximates the multi-step aligned trajectory while reducing truncation-error accumulation.

  • Assumptions: Trajectory-based compatibility is analyzed for an alignment field concentrated in a time window and varying slowly within that window.This time-locality assumption supports replacing the alignment integral with a local estimate.
  • Error decoupling: Under the stated premise, distillation compensates truncation error from the nonlinear base PF-ODE while the alignment integral is approximated by a single-step estimate.The proposition compares the merged model with precise multi-step solving of the pure alignment model.
  • Merge mechanism: The merged model approximates the alignment-field integral through a single-step jump under local smoothness assumptions.This is the central conclusion of the trajectory-based merge analysis.
  • Practical boundary: The analysis reports that DSWM accelerates inference in the evaluated setting without requiring expensive closed-loop distillation data.Its theoretical error bounds are local approximations, and effectiveness depends on structural similarity and parameter proximity to the base model.

A.2 Benchmark Descriptions

The evaluation uses benchmarks spanning compositional alignment, imaginative generation, multi-dimensional instruction following, and world-knowledge reasoning. Together, these benchmarks test both fine-grained visual semantics and broader reasoning capabilities.

  • GenEval: GenEval evaluates compositional text-to-image alignment using object-focused detection and segmentation across six atomic instruction tasks.Its binary component-level signals provide granular assessment of semantic composition.
  • GenEval++: GenEval++ increases instruction complexity by coupling more objects and attributes and uses vision-language-model checklist verification instead of only rule-based detectors.It was introduced to address score saturation and evaluation inaccuracy.
  • ImagineBench: ImagineBench tests surreal generation by requiring fantastical modifications that preserve core object identity, measuring fantasy fulfillment, identity preservation, and aesthetics.The benchmark targets creative generation beyond straightforward reproduction of real-world scenes.
  • PRISM-Bench: PRISM-Bench covers seven tracks, including a Long Text track that uses Generation Chain-of-Thought descriptions for complex multi-step instructions.Its tracks span imagination, entities, text rendering, style, affection, composition, and long text.
  • WiseBench: WiseBench evaluates world-knowledge integration with 1,000 prompts across 25 subdomains and reports a weighted consistency metric called WiScore.Its domains include cultural common sense, spatio-temporal reasoning, and natural sciences.

A.3 Detail of Data Source and Pipeline

The pipeline combines constrained trajectory generation, multi-model verification, and alignment methods for closed-loop visual reasoning. Experiments report improved compositional performance, adaptive reasoning length, and scope boundaries tied to the evaluation protocols.

  • Data Pipeline: The automated pipeline uses a state-constrained controller and diffusion agent to generate, inspect, refine, validate, and finalize reasoning trajectories.Gemini 2.5 Pro manages reasoning and state, while Seedream 4 generates and refines images.
  • Data Pipeline: Joint validation by Gemini 2.5 Pro and Seed 1.8 retains trajectories only when both judges approve each reasoning step and the final image over a single-step baseline.Blind A/B testing and failure-triggered termination further filter incoherent or degrading steps.
  • Alignment Results: On GenEval, VLM-SFT-only CLVR raises the FLUX.2 4B Distill baseline from 0.81 to 0.86 overall, while open-loop rewriting reaches 0.78.On WiseBench, prompt rewriting raises performance from 0.48 to 0.64, while VLM-SFT-only CLVR reaches 0.6577.
  • Alignment Results: PPRL improves both benchmarks after diffusion SFT, reaching 0.87 on GenEval and 0.74 on WiseBench, whereas simple RL is unstable.Simple RL changes GenEval from 0.85 to 0.84 and WiseBench from 0.62 to 0.64.
  • Complexity Scaling: 98.79 AUCpass for CLVR (FLUX2) exceeds 73.89 for its base model in the semantic-complexity probe.Single-step models follow a power-law relationship between AUCpass and effective capacity, while CLVR deviates from that trend.
  • Inference Efficiency: Approximately 68% of GenEval tasks finish within 2 iterations, 28% require 3, and only a few reach 4–5 iterations below the 8-iteration limit.CLVR uses shorter trajectories on GenEval and longer ones on the more complex PRISM benchmark.
Loading 2605.14876v2…