PaperlayerView exact revision on arXiv

Source-linked AI summary

Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget

Guoxuan Chen, Chufeng Xiao, Haoran Yang, Siyue Xie, Binxiao Huang, Ming Zhang, Cheuk Him Chau, Xinyu Fu, Yingzhao Lian, Tom S. Y. Li, Jintao Lin, Bowen Dong, Zian Qian, Yuhao Liu, Yuxuan Hu, Weikang Shi, Bin Zou, Bowen Zheng, Haoxuan Che, Chang Chen, Yuyang He, Heyang Sun, Tianyu Huang, Chong Hou Choi, Cheng Gong, Han Shi, Haoli Bai, Xihui Liu, Hongsheng Li, Qifeng Chen, Chao Huang, Rui Liu, Chenyang Lei

arXiv:2607.13125v2cs.CVcs.AI

TL;DR

The paper addresses how open-source image-generation systems can interpret increasingly complex user requirements while operating under constrained compute and limited disclosure of closed-source practices. It combines stronger multimodal understanding, agentic prompt rewriting and model selection, careful data curation, and inference-time scaling. Boogu-Image-0.1 matches or surpasses open-source models across standard benchmarks and approaches leading closed-source systems using 208.62 million unique images and approximately $400K in theoretical base-model training cost.

  • Problem

    Image generation is shifting from Text-to-Image toward Requirement-to-Image, but complex intentions, implicit constraints, cross-modal context, and varied latency requirements demand stronger understanding and system-level orchestration.

  • Method

    Boogu-Image-0.1 integrates stronger text encoding, agentic prompt rewriting, model selection, inference-time techniques, quality-focused data curation, and training optimization.

  • Results

    Boogu-Image-0.1 achieves top-tier Boogu Arena performance and state-of-the-art open-source results on Qwen-Image-Bench, using 208.62 million unique images and roughly $400K in training cost.

  • Takeaways & Limitations

    Understanding, evaluation practice, data filtering, and caption design are presented as practical ingredients for advancing open-source understanding-driven image generation.

  • Takeaways & Limitations

    Boogu trails leading closed-source systems on tasks requiring extensive common-sense and domain knowledge, optimizes text rendering only for Chinese and English, and remains limited by anatomical and fine-detail failures.

Abstract

from arXiv · show

We introduce Boogu-Image-0.1, an open-source unified multimodal understanding and generation model family, comprising Base, Turbo, Edit, and Edit-Turbo variants. It delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual (Chinese-English) text rendering. Closed-source multimodal systems like Nano-Banana-Pro and GPT-Image-2 achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. In this work, we demonstrate that strengthening the understanding capability of the system, through a stronger multimodal encoder, agentic prompt rewriting, and related techniques, together with improvements in data quality, training pipelines, and agentic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets. Comprehensive evaluations show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard benchmarks, and achieves results approaching leading closed-source systems. Notably, this is accomplished with only 208.62 million unique images. The base model's theoretical training cost is only approximately \$400K. We share practical discussions that we believe are valuable to the broader research community, and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. Our code is available here: https://github.com/Boogu-Project/Boogu-Image.

1 Introduction

Boogu-Image-0.1 treats understanding as a first-class part of image generation, targeting complex requirements through stronger language perception, agentic inference, and efficient training. The resulting open-source system reports competitive or top-tier performance while using 208.62 million unique images and roughly $400K in training cost.

  • Motivation: Image generation is framed as Requirement-to-Image, requiring interpretation of complex intentions, implicit constraints, multi-level instructions, and cross-modal context.The paper argues that understanding bridges ambiguous human requirements to precise visual generation and strategies suited to different cost and latency constraints.
  • Understanding: Stronger text encoders improve language-encoding capability, motivating the adoption of Qwen3-VL-8B as a balance between capability and parameter budget.The authors systematically analyze text encoders of varying scales and report stronger capabilities from stronger LLMs.
  • Agentic inference: Agentic Image Generation rewrites prompts, selects Base or Turbo variants, and applies inference-time techniques such as Reflection.The agent interprets user requests and orchestrates generation rather than treating the model as a passive text-to-image mapping.
  • Training efficiency: 208.62 million unique images and roughly $400K in training cost support a text-to-image base model with performance competitive with the state of the art.The efficiency strategy prioritizes data quality and training optimization under a constrained compute budget.
  • Practical lessons: Evaluation protocols, data filtering, and caption design are presented as decisive practical details whose opacity creates hidden engineering costs for open-source development.The paper documents these details as practical reference points for future work.
  • Results: Boogu-Image-0.1 achieves top-tier performance on Boogu Arena and state-of-the-art results among open-source models on Qwen-Image-Bench.The authors release the model and practical lessons to support understanding-driven generation research.

2 Experimental Results

Boogu-Image-0.1 performs strongly across human-preference, text-to-image, text-rendering, and image-editing evaluations, while the paper argues that several conventional benchmarks misrepresent real-world capability. Results also expose important scope limits, including benchmark saturation, incomplete baseline coverage, and weaknesses in automated evaluation.

  • Benchmark validity: GPT-Image-2 ranks only mid-pack on GenEval and DPG-Bench despite leading human preference, revealing rank inversions between public benchmarks and generation quality.
  • Benchmark validity: Public benchmarks can diverge from deployment because they use synthetic, isolated tasks, saturate near ceiling, and may suffer data contamination or leakage.
  • Text-to-image generation: Boogu models lead the open-source tier across Boogu Arena categories, with Turbo-Thinking most clearly narrowing the gap to closed-source models in photorealistic and stylized content.
  • Text-to-image generation: 53.57 and 53.73 are Boogu-Image-0.1-Base-Thinking's best overall Qwen-Image-Bench scores for Chinese and English prompts, respectively, among evaluated open-source models.
  • Text rendering: 0.985 places Boogu-Image-0.1-Turbo-Thinking second overall on Chinese text rendering, behind Seedream-4.5.
  • Image editing: 4.64 makes Boogu-Image-0.1-Edit-Thinking the best overall ImgEdit-Bench model, ahead of JoyAI-Image-Edit at 4.57 and FireRed-Image-Edit at 4.56.
  • Image editing: Human evaluations place Nano-Banana-Pro above Boogu models in real editing scenarios despite its lower ImgEdit-Bench score, while the benchmark's coarse taxonomy omits many real operations.

3 Experimental Analysis and Discussion

Boogu centers system design on understanding, combining stronger encoders, prompt rewriting, and complexity-aware inference to improve quality–efficiency trade-offs. The discussion also argues that evaluation must account for latency, application realism, leakage, and unresolved challenges in measuring real capability.

  • System design: Boogu decomposes understanding into complementary dimensions targeting distinct errors in text-to-image pipelines.This decomposition forms the backbone of its quality–efficiency trade-off.
  • Understanding and generation: Scaling the frozen instruction encoder consistently improves GenEval scores when training data, hyperparameters, and DiT architecture remain fixed.The gains are attributed to instruction-understanding capacity rather than parameter count alone.
  • Understanding and generation: A reasoning-capable prompt rewriter significantly boosts overall performance, with stronger vision-language models producing better rewrites.The reported correlation links rewriting gains to intrinsic world knowledge and reasoning capacity.
  • Understanding and generation: Per-aspect captioning produces more accurate image descriptions than any single-VLM baseline and improves corresponding generation capabilities.The strategy evaluates models across counting, spatial relations, attributes, style, and text rendering before selecting configurations.
  • Evaluation: Quality-improving techniques such as best-of-N sampling, self-reflection, prompt rewriting, and ensembling increase inference cost, making latency essential to evaluation.Reported quality without an accompanying latency budget omits the deployment trade-off.
  • Evaluation: Public benchmarks can misrepresent practical capability because they may use synthetic settings, suffer from leakage, and miss world knowledge or ambiguous semantics.The paper calls for ImageNet-like benchmarks, world-knowledge evaluation, dynamic protocols, and stronger leakage controls.

4 Conclusion, Limitations, and Future Directions

Boogu-Image-0.1 unifies understanding and generation for image creation and editing, combining data-efficient training with inference-time agentic scaling. The authors report strong open-source performance while identifying limitations in world knowledge, language coverage, anatomy, and fine-grained reconstruction.

  • Boogu-Image-0.1 jointly designs multimodal understanding and image generation within one open-source system.
  • 208.62M unique images and roughly $400K support training under constrained resources.
  • Agentic prompt rewriting and model selection further improve inference-time generation performance.
  • Limitations: Boogu still trails leading closed-source systems on extensive common-sense and domain-knowledge tasks, with the gap difficult to measure reliably.
  • Limitations: Current optimization covers Chinese and English text rendering, while complex anatomy and fine-grained details remain vulnerable to inconsistencies and VAE reconstruction error.
  • Future Directions: Future directions emphasize transparent evaluation, higher-fidelity data curation, and stronger agentic reasoning, verification, and tool use.

A Contributors

The paper lists the project’s sponsors, leadership, contributors, affiliations, and collaborating institutions, followed by background on diffusion, editing, VLMs, and unified multimodal generation.

  • Contributors: The project includes sponsors, a project leader, core contributors, contributors, and named correspondence contacts.
  • Affiliations: The primary affiliation is Huawei Technologies’ Artificial Intelligence Laboratory in Hong Kong.
  • Affiliations: Collaborating institutions include six Hong Kong and mainland Chinese universities.
  • Related Work: The related work background covers text-to-image diffusion, controllable editing, vision-language models, and unified multimodal generation.

B.2 Macro-Architecture Design

The macro-architecture uses an Instruction Reasoner and Instruction Encoder to interpret tasks before a Diffusion Transformer performs generation or editing. Reference-image presence determines the task modality, while visual and textual inputs are integrated through shared processing stages.

  • Instruction Reasoner: The Instruction Reasoner analyzes, decomposes, and rewrites user instructions before conditioning the generation pipeline.
  • Task Modality: The system defaults to text-to-image generation without reference images and to editing when reference images are provided.
  • Overall Pipeline: The macro-pipeline uses an autoregressive Transformer as the Instruction Encoder and a DiT for iterative high-fidelity synthesis.
  • Instruction Encoder: A tokenizer embeds rewritten text, while a ViT encodes reference images and bypasses processing when no reference image is present.
  • Instruction Encoder: An optional three-layer Prompt Tuning Transformer prepends 32 trainable prompt embeddings to enhance instruction parsing.
  • Diffusion Transformer: The DiT combines reference-image and noise streams with instruction representations, then decodes the result through a VAE into the final image.

B.3 Micro-Architecture Details

The micro-architecture adapts established components into a unified framework supporting both text-to-image generation and image editing.

  • The proposed micro-architecture modifies and extends prior designs to support text-to-image generation and image editing within one framework.

B.3.1 Dual-Stream Layer

The dual-stream layer separately processes instruction and image information before fusing them, using residual main paths and computational branches to stabilize and shape representations. Subsequent single-stream layers unify the deeply modeled streams for comprehensive multimodal analysis.

  • Dual-stream processing: The instruction and image streams are processed separately, with the image stream modeling noisy latent and reference-image interactions.The other stream fuses instruction and image modalities.
  • Dual-stream processing: An independent image attention module deepens visual comprehension, while the complementary stream performs instruction-image fusion.Together, the streams provide fine-grained multimodal comprehension before convergence.
  • Main path and branches: Residual main paths stabilize layer parameter distributions, while branch paths apply attention and modulation before merging into the main path.Retaining only the main path would reduce the layer to an identity mapping.
  • Single-stream fusion: After convergence, the single-stream layer merges instruction and image hidden states into one representation for comprehensive multimodal fusion and analysis.It uses modulation and sandwich normalization following established designs.

B.5 Training Process

Training uses a progressive curriculum that moves from low to high resolution and from text-to-image generation to text-and-image-to-image editing. The editing phase combines mixed-resolution and mixed-task data.

  • Progressive curriculum: Training first uses low-resolution images, then scales to higher resolutions, before fine-tuning text-to-image training into editing.This progressive curriculum is summarized across tasks and resolutions.
  • Resolution handling: Images exceeding spatial thresholds are dynamically downscaled, while smaller images remain unchanged to support mixed-resolution training.The thresholds are Max Output Pixels, Max Input Pixels, and Max Side Length.
  • Editing training: Approximately 22M editing samples combine 11.29M pure text-to-image samples with 10.70M augmented editing samples.The editing model is therefore optimized through mixed-resolution and mixed-task training.

C Boosted Orthogonal Guidance

Boosted Orthogonal Guidance is a training-free inference method designed to improve fine-grained detail and cinematic atmosphere while addressing appearance degradation from aggressive guidance. It treats DiT predictions as image-aligned matrices rather than flattened vectors.

  • Motivation: Aggressive classifier-free guidance can improve alignment while causing oversaturation, flattened tones, over-smoothing, and implausible artifacts.BOG targets this failure mode during inference.
  • Core idea: BOG emphasizes the information-preserving orthogonal component of guidance updates to avoid color blow-up and texture collapse.The method is inspired by decomposing guidance into parallel and orthogonal components.
  • Conventional formulation: Standard DiT inference treats each prediction as a flattened one-dimensional update direction toward a clean-image hidden state.This viewpoint underlies conventional guidance and normalization strategies.
  • Matrix representation: The DiT output has shape [B, C, H, W], and its [H, W] structure forms a meaningful image-aligned grid rather than an arbitrary reshaping.BOG uses this spatial structure as part of its matrix-based formulation.

C.1.2 Matrix Orthogonal Decomposition

The paper formulates diffusion generation as ODE integration with DiT predictions and reviews classifier-free guidance as conditional-unconditional extrapolation. Its matrix decomposition then separates spatial updates into parallel and orthogonal components across columns and rows.

  • Diffusion ODE: Diffusion generation numerically integrates an ODE from Gaussian noise toward the data distribution, using a DiT to approximate the intractable score.Conditional generation augments the DiT with auxiliary inputs such as text.
  • Classifier-free guidance: Classifier-free guidance extrapolates from the unconditional DiT prediction toward the conditional prediction using scale ω.The conditional-minus-unconditional difference provides the guidance update.
  • Matrix representation: The DiT output is treated as an [H, W] matrix after omitting batch and channel dimensions, enabling spatially structured decomposition.The matrix difference is denoted ∆D.
  • Spatial decomposition: Parallel-orthogonal decomposition is applied column-wise and row-wise to obtain separate spatial update matrices.The parallel-component magnitude is controlled by μ, with μ ≤1 used in practice to mitigate oversaturation when guidance is large.

C.2 Rolling-Sum Momentum

The method adds momentum to the guidance trajectory during text-to-image inference, combining the current and previous guidance updates. Empirically, negative momentum causes directional jitter, motivating the positive default settings.

  • Rolling-Sum Momentum: Positive momentum smooths the guidance trajectory across inference iterations.The approach is motivated by the intuition that trajectory inertia within the flow-matching vector field influences the final generation.
  • Rolling-Sum Momentum: The momentum update combines the current guidance update with the previous iteration using η = 0.9 and ρ = 0.1.The iteration index i maps to diffusion time through t = scheduler(i).
  • Rolling-Sum Momentum: Negative momentum causes directional jitter in the guidance update across iterations.This contrasts with reverse-momentum proposals that use ρ < 0 to move away from previous classifier-free-guidance directions.

C.3 Putting It All Together

The complete pipeline applies Boosted Orthogonal Guidance to the current inference state and returns a modified prediction. BOG combines momentum, matrix normalization, and orthogonal decomposition, but remains optional because it can worsen structural distortions and text rendering.

  • C.3 Putting It All Together: BOG refines guidance by smoothing the trajectory, preserving spatial rank through matrix normalization, and isolating orthogonal components.The orthogonal operation diminishes projections parallel to the conditional prediction to prevent oversaturation.
  • C.3 Putting It All Together: BOG takes the inference step, latent state, condition, unconditional state, and guidance scale as inputs, producing a modified prediction.Algorithm 1 summarizes this single-step computation.
  • C.3 Putting It All Together: BOG increases the probability of structural distortions and reduces text-rendering success, so the authors treat it as an optional filter.The interval setting alleviates these artifacts with a slight trade-off in enhanced BOG-styled photographic texture.
  • C.3 Putting It All Together: The BOG Interval applies BOG every ΔBOG diffusion steps while using standard classifier-free guidance on the remaining steps.With ΔBOG = 2 in a 10-step process, BOG runs on steps 2, 4, 6, 8, and 10; this is the recommended default.

C.4 Optimal Semi-Orthogonal Approximation

The section characterizes the optimal semi-orthogonal approximation as a trace-maximization problem solved in SVD coordinates. It also describes a Newton-Schulz alternative for computing the matrix normalization without explicitly performing SVD.

  • C.4 Optimal Semi-Orthogonal Approximation: Using D = UΣV^⊤, the optimization is expressed through Q = U^⊤OV in SVD coordinates.Q is an n×n orthogonal matrix because U and O are semi-orthogonal and V is orthogonal.
  • C.4 Optimal Semi-Orthogonal Approximation: The diagonal bound |q_ii| ≤ 1 implies the trace upper bound, which is achieved by O = UV^⊤.The optimizer therefore minimizes the Frobenius approximation error.
  • C.4 Optimal Semi-Orthogonal Approximation: The semi-orthogonal approximation minimizes the Frobenius distance by maximizing tr(O^⊤D).The proof reduces the squared Frobenius norm to trace maximization.
  • C.4 Optimal Semi-Orthogonal Approximation: Von Neumann’s trace inequality independently recovers the same optimizer and minimum value.Equality occurs at Q = I_n, equivalently O = UV^⊤.
  • C.4 Optimal Semi-Orthogonal Approximation: MNorm(D) can be estimated with Newton-Schulz iterations instead of explicitly computing an SVD when norm(σ_i, i) = 1.The implementation normalizes matrices by their Frobenius norm and performs batched matrix multiplications.
Loading 2607.13125v2…