Source-linked AI summary
BBQ-to-Image: Numeric Bounding Box and Qolor Control in Large-Scale Text-to-Image Models
Eliran Kachlon, Alexander Visheratin, Nimrod Sarid, Tal Hacham, Eyal Gutflaish, Saar Huberman, Hezi Zisman, David Ruppin, Ron Mokady
TL;DR
Text-to-image models remain imprecise for workflows requiring exact numeric control of object location, size, and color. BBQ conditions generation on numeric bounding boxes and RGB values through enriched structured captions, achieving strong spatial alignment and improved color fidelity while supporting structured user control.
Problem
Descriptive text remains imprecise for professional workflows requiring deterministic control of object size, location, and color.
Method
BBQ augments structured training captions with numeric bounding boxes and RGB values without architectural modifications or inference-time optimization.
Results
BBQ achieves strong box alignment and improves RGB color fidelity over state-of-the-art baselines.
Takeaways & Limitations
Structured intermediate language can translate user intent into precise, deterministic numeric control while preserving expressiveness.
Takeaways & Limitations
Evaluation uses a neutral VLM captioning pipeline and compares models using different native or alternative captioning schemes.
Abstract
from arXiv · showhide
Text-to-image models have rapidly advanced in realism and controllability, with recent approaches leveraging long, detailed captions to support fine-grained generation. However, a fundamental parametric gap remains: existing models rely on descriptive language, whereas professional workflows require precise numeric control over object location, size, and color. In this work, we introduce BBQ, a large-scale text-to-image model that directly conditions on numeric bounding boxes and RGB triplets within a unified structured-text framework. We obtain precise spatial and chromatic control by training on captions enriched with parametric annotations, without architectural modifications or inference-time optimization. This also enables intuitive user interfaces such as object dragging and color pickers, replacing ambiguous iterative prompting with precise, familiar controls. Across comprehensive evaluations, BBQ achieves strong box alignment and improves RGB color fidelity over state-of-the-art baselines. More broadly, our results support a new paradigm in which user intent is translated into an intermediate structured language, consumed by a flow-based transformer acting as a renderer and naturally accommodating numeric parameters.
BRIA AI
BBQ enables precise spatial and color control using numeric bounding boxes and RGB values, including targeted refinement through parameter edits.
- BBQ conditions image generation on explicit numeric bounding boxes and RGB values for precise spatial and color control.
- Editing bounding boxes or RGB values and regenerating produces targeted changes while preserving the rest of the scene.
1 Introduction
BBQ addresses the imprecision of descriptive language by introducing numeric control for object size, location, and color. It achieves this through enriched structured captions and supports a structured-language interface between user intent and image rendering.
- 1 Introduction: Professional workflows require explicit RGB values and pixel-accurate bounding boxes rather than subjective descriptors such as “crimson” or “bottom-right.”
- 1 Introduction: Bounding boxes support object dragging and RGB values integrate with color pickers, replacing ambiguous prompting with precise interfaces.
- 1 Introduction: BBQ achieves numeric parametric control by augmenting training captions, without architectural modifications, special grounding tokens, or inference-time optimization.
- 1 Introduction: A fine-tuned VLM converts short natural-language prompts into detailed parametric descriptions that BBQ executes.
- 1 Introduction: User intent is translated into an intermediate structured language consumed by a flow-based transformer acting as a renderer.
- 1 Introduction: BBQ achieves strong precision for object location, size, and color control within a unified text-based framework.
2 Related Works
Prior work improves text-to-image controllability through structured captions and specialized spatial or color mechanisms. BBQ instead incorporates exact numeric parameters into a unified structured textual representation without architectural or inference-time modifications.
- 2 Related Works: Transformer and flow-matching advances, together with long structured captions, have improved visual fidelity and prompt alignment.
- 2 Related Works: FIBO enables native disentanglement through structured JSON, but descriptive strings such as “red” or “top-left” remain semantically ambiguous.
- 2 Related Works: Region-controlled methods such as ReCo, GLIGEN, InstanceDiffusion, and Ranni use specialized position tokens, architectural changes, or inference-time modifications.
- 2 Related Works: BBQ unifies high-precision spatial control in structured text, providing exact coordinate guidance without modifying the underlying architecture.
- 2 Related Works: Color-control methods often rely on specialized adapters, task-specific losses, or inference-time optimization, whereas BBQ encodes RGB triplets directly in textual conditioning.
- 2 Related Works: BBQ’s refinement examples use structured JSON edits and re-generation with the same random seed to produce localized changes while preserving the scene.
3 Method
BBQ adapts a large-scale text-to-image model to accept numeric bounding boxes and RGB colors through caption augmentation rather than architectural changes or extra inference procedures. A VLM bridge translates short prompts and edits into structured parametric prompts, enabling controlled generation and interaction.
- 3 Method: BBQ conditions generation on numeric bounding boxes and RGB triplets, aligning images with specified spatial and chromatic parameters.Bounding boxes use normalized top-left and bottom-right coordinates, while colors use RGB values in [0, 255]^3.
- 3 Method: The model achieves parametric control through dataset augmentation alone, without architectural changes or additional loss functions.Training extends structured captions with numeric bounding boxes and colors.
- 3.3 The Parametric Bridge: From Short Captions to Long, Structured, Parametric Prompts: A fine-tuned VLM translates short prompts into full structured parametric descriptions and supports interactive edits such as dragging, resizing, and recoloring.Its modes include generation, refinement of existing JSON, and reference-image-based parametric description.
- 3.1 Enriching the Training Data with Bounding Boxes and Colors: Training data replaces semantic locations and qualitative colors with explicit coordinates and RGB triplets, using automated parametric extraction for visual grounding.The pipeline starts from FIBO-style captions and adds dominant-object colors and a global RGB palette.
- 3.2 BBQ: Large-Scale Training to Control Bounding Boxes and Qolors: BBQ continues training an 8B-parameter FIBO backbone on 25M images paired with parametric captions.The training follows FIBO’s hyperparameters and includes later aesthetic finetuning and DPO training for text rendering.
- 3.2 BBQ: Large-Scale Training to Control Bounding Boxes and Qolors: BBQ preserves targeted attribute changes while keeping the rest of the scene largely unchanged under same-seed structured edits.The reported behavior demonstrates preservation of FIBO’s native disentanglement.
4 Experiments
BBQ is evaluated for expressiveness, spatial grounding, and color fidelity using complementary qualitative and quantitative protocols. Across these evaluations, it preserves scene structure, aligns objects with numeric boxes, and achieves strong chromaticity control against competing models.
- Evaluation Design: The evaluation isolates expressiveness, spatial accuracy under numeric box constraints, and color fidelity under explicit RGB specifications.TaBR measures caption-to-generation reconstruction, bounding-box accuracy measures spatial grounding, and color accuracy measures parametric color fidelity.
- Color-conditioning Accuracy: Color evaluation extracts foreground object pixels from 200 single-object images and clusters them in CIELab space with K = 5 and K = 8.This protocol isolates the target object and reduces interference from other image regions.
- Expressiveness: BBQ preserves original pose, object relationships, and overall scene layout in Text-as-a-Bottleneck reconstructions.Image-level pairwise preferences also show BBQ consistently outperforming FIBO, Nano Banana Pro, and Flux.2 Pro.
- Bounding-box Accuracy: BBQ consistently aligns object placement with specified regions, outperforming Nano Banana Pro, Flux.2 Pro, and GLIGEN while trailing InstanceDiffusion.It achieves this without grounding-specific architectural modifications or inference-time alignment mechanisms.
- Color-conditioning Accuracy: BBQ achieves the lowest a–b chromaticity errors across mean, median, and 90th-percentile statistics for both K = 5 and K = 8.The a–b metric isolates hue and saturation differences while ignoring lighting; BBQ remains competitive under CIEDE2000, which penalizes lightness variation.
5 Conclusion
BBQ addresses the gap between descriptive prompting and deterministic numeric control by conditioning on structured bounding boxes and RGB values. The conclusion presents structured intermediate representations as a route to interactive, programmable image synthesis while maintaining global scene coherence.
- Conclusion: BBQ enables precise control over object location, size, and color through explicit numeric bounding boxes and RGB values.The model achieves this through large-scale training on enriched structured captions without architectural modifications or inference-time optimization.
- Conclusion: Structured intermediate representations translate user intent into numeric parameters for direct manipulation while preserving global scene coherence.The framework supports interfaces such as object repositioning and precise color selection.
- Conclusion: The approach suggests a path toward programmable, professional-grade image synthesis systems that can integrate additional precise attributes.It moves beyond descriptive prompting toward controllable generative modeling.
A Additional Refinement Examples
Structured parametric editing refines generated images by changing numeric box coordinates and object color, producing updated layouts and appearances while preserving scene coherence.
- A Additional Refinement Examples: Editing structured parametric captions changes object position, extent, and color before regeneration.Each example explicitly enforces the target color #DD20A7 while updating spatial layout and appearance.
- A Additional Refinement Examples: The regenerated results preserve overall scene coherence despite simultaneous spatial and chromatic edits.