PaperlayerView exact revision on arXiv

Source-linked AI summary

Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models

Ruchit Rawal, Reza Shirkavand, Sayak Paul, Yuxin Wen, Heng Huang, Yizheng Chen, Tom Goldstein, Gowthami Somepalli

arXiv:2607.04461v1cs.CV

TL;DR

Inference-time scaling for text-to-image generation often emphasizes intermediate verification while overlooking generation cost and verifier overhead in efficiency comparisons. Flash-BoN expands exploration with inexpensive draft candidates, efficient verification, and full-quality refinement, consistently outperforming baselines under fixed wall-clock budgets, with gains reaching +8% AUC at larger model scales.

  • Problem

    Existing diffusion scaling methods emphasize intermediate verification while generation costs and verifier overhead remain incompletely represented by NFE-based comparisons.

  • Method

    Flash-BoN combines timestep truncation, layer skipping, and activation proxies to generate many drafts, then verifies and fully refines the most promising candidate.

  • Results

    +8% AUC at larger model scales, with Flash-BoN consistently outperforming baselines across three benchmarks and three model scales under fixed wall-clock budgets.

  • Takeaways & Limitations

    The results support broader, more diverse candidate exploration under wall-clock budgets and show compatibility with orthogonal scaling methods and RL post-training.

  • Takeaways & Limitations

    VLM-based verifiers can be noisy and poorly calibrated, with tied top scores making pointwise selection effectively random for many prompts.

Abstract

from arXiv · show

Inference-time scaling for text-to-image generation has progressed from simple Best-of-$N$ (BoN) sampling to guided search methods that verify and steer candidate trajectories at intermediate denoising steps. These approaches focus on when and how often to verify during denoising but largely treat the cost of generation itself as fixed. Moreover, the standard practice of comparing methods by number of function evaluations (NFEs) counts only denoising forward passes and ignores verifier overhead, which can distort efficiency rankings. We show that under wall-clock evaluation, simple BoN already matches or outperforms several guided search techniques, suggesting that compute is better spent on broader exploration than on repeated intermediate verification. This motivates Flash-BoN, which generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs: timestep truncation, layer skipping, and activation proxies into a single configuration optimized once per model. An efficient multi-stage verification procedure then identifies the most promising draft, which is refined at full quality. Across three benchmarks and three model scales, Flash-BoN consistently outperforms all baselines under fixed wall-clock budgets, with gains that grow at larger model scales (+8% AUC). We further show that our strategy combines well and improves existing orthogonal techniques such as reflection-based prompt optimization (+16% AUC). The gains correlate with increased candidate diversity, which also enables draft-guided selection to accelerate RL post-training convergence.

1 Introduction

The introduction argues that inference-time scaling for diffusion T2I should account for generation and verifier costs under wall-clock budgets, not NFEs alone. It presents Flash-BoN as a draft-generation and multi-stage verification pipeline that consistently improves scaling across benchmarks and model scales.

  • Motivation: Inference-time scaling for T2I balances generation of candidates against verification that scores and selects outputs.BoN concentrates compute on full trajectories and verifies once, whereas guided search verifies during denoising to prune, guide, or backtrack.
  • Motivation: NFE comparisons can misrank methods because they omit verifier overhead, while wall-clock evaluation shows simple BoN matches or exceeds guided search.Verifier steps can be more compute intensive than individual denoising steps, so frequent verification reduces exploration within a fixed runtime.
  • Flash-BoN: Flash-BoN generates inexpensive draft candidates, verifies promising ones, and refines only selected candidates at full quality.Its draft policy jointly configures complementary acceleration knobs and is solved once per model.
  • Flash-BoN: Flash-BoN combines fast pointwise pruning with targeted pairwise comparisons to identify the best draft efficiently.Pointwise scoring narrows the pool, while pairwise comparisons provide stronger discrimination without comparing every candidate pair.
  • Results: +8% AUC: Flash-BoN’s margins grow at larger model scales while it leads across model–benchmark combinations.The paper reports consistent outperformance across three benchmarks and three model scales.

2 Related Works

Prior inference-time scaling methods improve diffusion generation by searching noise-space trajectories or steering sampling with verifiers, but typically measure compute using NFEs or FLOPs while overlooking verifier and control-logic costs. Flash-BoN instead treats wall-clock time as the primary constraint and uses efficiency techniques to explore more candidates within a fixed runtime.

  • Prior inference-time scaling: Existing methods allocate additional inference-time compute by searching for better diffusion trajectories or steering sampling with verifiers or reward models.Inference-time scaling is framed as a search over noise space guided by learned verifiers, with increasing compute yielding consistent gains.
  • Limitations of existing evaluation: Prior approaches primarily measure compute through NFEs or FLOPs, abstracting away verifier and control-logic costs.This accounting can omit runtime spent on intermediate verification and search decisions.
  • Flash-BoN perspective: Flash-BoN treats wall-clock time as the primary constraint and repurposes timestep truncation and architectural simplifications to expand candidate exploration under a fixed runtime.The approach uses efficiency techniques not merely to preserve baseline quality, but to enlarge the set of explored candidates.

3 Problem Setup

The problem setup seeks the highest-scoring image from candidates generated within a fixed wall-clock budget. It formalizes diffusion sampling as iterative denoising and counts generation, verification, and data-loading costs in the budget.

  • Sampling process: Diffusion sampling starts from Gaussian noise and iteratively denoises across S discrete timesteps to produce an image.Each step applies a neural network fθ to the current latent, prompt, and timestep.
  • Sampling process: The denoising network consists of L sequential U-Net or DiT blocks, and the formulation also applies to flow-matching models.The paper evaluates both diffusion and flow-matching settings.
  • Optimization objective: Given prompt x and wall-clock budget T, the method selects the candidate y* maximizing verifier score V(y, x), which proxies benchmark metric E(y, x).Inference-time scaling generates a pool of N candidate images before selection.
  • Optimization objective: The total-cost constraint includes all generation, verification, and applicable data reading/loading costs, not only denoising computation.This cost is represented by T_total({y_j}_{j=1}^{N}, x) ≤ T.

4 Flash-BoN

Flash-BoN accelerates inference-time scaling by generating many inexpensive draft candidates with a jointly optimized configuration, then verifying and fully refining the selected candidate. Its multi-stage selection addresses noisy pointwise verification while preserving end-to-end validity through cached latent states and full-computation refinement.

  • Draft acceleration: Flash-BoN combines timestep truncation, layer skipping, and activation proxies in a per-model configuration optimized once for speed and sufficient fidelity.The configuration jointly specifies preview steps, skipped layers, skipping onset, and full-computation frequency.
  • Three-stage pipeline: It generates M cheap previews with different random seeds, verifies them, selects the best draft, and resumes generation with full computation.The three stages are draft generation, verification and selection, and refinement.
  • Verification and selection: Pointwise verifier scores are cheap and parallelizable but unreliable because noisy, integer-valued outputs cause frequent ties and effectively random top-candidate selection.Over 80% of prompts have tied highest scores, and roughly 30% have 20+ candidates sharing the top score.
  • Verification and selection: A multi-stage Elo filter combines pointwise pruning, sparse adjacent pairwise comparisons, and a dense final tournament to select the highest-rated candidate.The procedure progressively retains survivors while concentrating comparisons where rankings are most uncertain.
  • Selective refinement: Selective refinement is valid because diffusion sampling is Markovian: future outputs depend only on the selected latent, allowing cached-state continuation with the standard unaccelerated configuration.Cached tensors can be offloaded to CPU, and wall-clock evaluation captures the resulting transfer overhead.

5 Experiments

Across three model scales and benchmarks, Flash-BoN leads under fixed wall-clock budgets, with broader exploration outperforming guided search and gains widening on larger models. Its draft strategy also improves compositional performance, existing scaling methods, verifier robustness, and RL convergence.

  • Wall-clock performance: Flash-BoN leads every model–benchmark combination on normalized AUC/Time, while BoN-style exploration outperforms guided search across Wan2.1 1.3B, Wan2.1 14B, and FLUX.1-dev.The comparison covers GenAI-Bench, GenEval, and UniGenBench under end-to-end wall-clock accounting.
  • Prompt categories: Flash-BoN leads all ten Wan2.1 1.3B prompt categories, with its largest gains on compositional prompts and narrower gaps on Attribute and Counting.Broader draft pools help sample valid layouts, while fine-grained distinctions are harder to resolve from draft previews.
  • Combining scaling methods: Flash-Reflection-Tuning raises AUC from 0.46 to 0.62, while Flash-BFS improves it from 0.49 to 0.55.Cheaper draft cycles enable more prompt revisions, whereas locally constrained BFS benefits less from faster intermediate steps.
  • Candidate diversity: Pearson r=0.75 links candidate-pool diversity with final quality, with Flash-BoN at the high-diversity end and guided methods in the lower-diversity, lower-performance region.Diversity is measured using the Vendi Score over candidates generated within the fixed budget.
  • Verifier and evaluation metric: ImageReward shows 270% improvement over no scaling, compared with 40% for HPSv3 and 26% for VQAScore; general VLM verifiers deliver consistent 6–14% gains across metrics.Matched verifier–evaluator pairs yield the largest gains, but distinct proxy verifiers improve cross-metric generalization and avoid metric-specific bias loops.
  • RL post-training: Flash-Flow-GRPO reaches 0.699 versus 0.692 for the baseline and matches the baseline’s final performance by step 60, indicating roughly 10× faster convergence.The gap opens within the first 60 training steps and persists throughout training on Wan2.1 1.3B.

6 Discussion

Flash-BoN generates many inexpensive draft candidates, verifies them efficiently, and refines only the most promising candidate at full quality. It consistently outperforms baselines under fixed wall-clock budgets, with gains increasing at larger model scales and broader exploration driving the improvements.

  • Method: Flash-BoN generates a large pool of inexpensive draft candidates before selecting and fully refining only the most promising one.Selection uses an efficient multi-stage verification procedure.
  • Results: Flash-BoN consistently outperforms baselines across multiple benchmarks and models under fixed wall-clock budgets.Its gains grow at larger model scales.
  • Implications: Flash-BoN’s improvements are driven by broader and more diverse exploration.Flash-BoN rollouts also make Flash-Flow-GRPO converge faster and maintain a consistent advantage throughout training on GenAI-Bench with Wan2.1 1.3B.

Supplemental Material · A Additional Experiment Setup Details · A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion

Guided search requires stochastic branching, but flow-matching models use deterministic ODE sampling, so shared-state branching produces identical children. The paper enables BFS, DFS, and ZOS by converting the probability-flow ODE into a marginal-preserving SDE and discretizing its reverse-time dynamics.

  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: Guided search baselines BFS, DFS, and ZOS require stochastic branching or resampling at intermediate denoising steps.Diffusion models provide this noise natively, whereas flow-matching models do not.
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: Deterministic ODE sampling maps each initial noise state to one output, making children spawned from a shared intermediate state identical.Consequently, guided search is inapplicable to flow models without modification.
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: The method converts the probability-flow ODE into an equivalent SDE that preserves the marginal distribution p_t(x) at every timestep while adding branching stochasticity.The construction follows the stochastic interpolant framework and prior ODE-to-SDE conversion work.
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: The required SDE drift adds a score-dependent correction to the learned velocity field: f_SDE(x_t,t) = v_t(x_t) + (σ_t^2/2)∇ log p_t(x_t).This drift is obtained by matching the ODE and SDE marginal-density equations.
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: For rectified flow, the score is expressed through the velocity field and substituted into the reverse-time SDE before Euler–Maruyama discretization.The resulting practical update combines a drift term with Gaussian noise ε ~ N(0, I).
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: Setting a = 0 recovers deterministic ODE sampling, whereas larger a increases diversity at the cost of sample quality.The experiments use a = 0.3 to support trajectory branching while preserving image fidelity.
  • A.1 Enabling Guided Search on Flow Models via ODE-to-SDE Conversion: The ODE-to-SDE conversion is applied only to BFS, DFS, and ZOS, whose intermediate branching requires stochasticity.It is not applied to the underlying flow model’s standard deterministic sampling procedure.

A.2 Evaluation Protocol Details & Hyper-parameters · A.3 Combining Drafts with Existing Scaling Methods: Additional Details

The evaluation uses fixed benchmark subsets, full GenEval coverage, and controlled H200 experiments with separate generation and verification GPUs. Additional details specify model-specific Flash-BoN draft configurations and how drafts integrate with reflection and particle-based scaling methods.

  • A.2 Evaluation Protocol Details & Hyper-parameters: 300 prompts are uniformly sampled from GenAI-Bench and UniGenBench, while all 553 GenEval prompts are evaluated.The fixed subsets are reused across experiments to reduce exhaustive evaluation cost.
  • A.2 Evaluation Protocol Details & Hyper-parameters: GenEval assesses compositional constraints, whereas GenAI-Bench uses VQAScore for text-image alignment.The reported GenEval constraints include object presence, counting, spatial relations, and attribute binding.
  • A.2 Evaluation Protocol Details & Hyper-parameters: All methods run on NVIDIA H200 GPUs with batch size 8, exploring 8 candidates or using particle groups of 8.BoN and Flash-BoN explore candidates in parallel, while BFS, DFS, and ZOS use particle groups.
  • A.2 Evaluation Protocol Details & Hyper-parameters: Two GPUs per node separately handle generation and verification so verifier inference does not affect generation wall-clock measurements.This allocation reserves one GPU for each function.
  • A.2 Evaluation Protocol Details & Hyper-parameters: Flash-BoN’s optimized draft configurations are S′=49, nskip=3 for Wan2.1 1.3B, S′=49, nskip=2 for Wan2.1 14B, and S′=48, nskip=11 for FLUX.1-dev.The corresponding layer skips and full-quality frequencies are {22, 23}/5, {34, 35}/4, and {40, ..., 44}/3, respectively.
  • A.3 Combining Drafts with Existing Scaling Methods: Additional Details: The same draft configuration supports standalone Flash-BoN, reflection using Qwen2.5-VL-7B, and accelerated denoising between BFS verification steps.Reflection iterates on draft images until revisions stop or the maximum iteration count is reached, then produces a full-quality generation.

A.4 RL Post-Training: Additional Details. · B Additional Experiments

Flash-Flow-GRPO and baseline Flow-GRPO share core training and evaluation settings, while differing in rollout selection and regularization. Flash-Flow-GRPO uses draft-selected trajectories with stricter calibration requirements and a larger KL penalty to limit policy divergence.

  • A.4 RL Post-Training: Additional Details.: Flow-GRPO and Flash-Flow-GRPO share LoRA α=64, r=32, learning rate 10−4, GRPO clip range 10−3, noise level a=0.7, and CFG scale 5.0.Both use training denoising steps T=20 and evaluation denoising steps T=50.
  • A.4 RL Post-Training: Additional Details.: Both methods train on prompts from the GenAI-Bench train set combined with the HPSv3 dataset.
  • A.4 RL Post-Training: Additional Details.: The two runs differ only in rollout selection and regularization.
  • A.4 RL Post-Training: Additional Details.: Flash-Flow-GRPO generates 2G=16 draft rollouts.
  • A.4 RL Post-Training: Additional Details.: Drafts must preserve verifier ranking fidelity and keep reference-model log-probabilities from full forward passes on selected trajectories well-calibrated.
  • A.4 RL Post-Training: Additional Details.: Flash-Flow-GRPO increases the KL penalty to β=0.1 versus 0.05 for the baseline to prevent excessive policy divergence under the draft-selected distribution.

B.1 Wall-Clock vs. NFE on Additional Benchmarks … Effect of Verifier Model Choice

Under matched wall-clock budgets, frequent verifier calls reduce exploration enough to reverse fixed-NFE rankings, while verifier choice and selection strategy materially affect quality. Multi-stage verification with appropriately scaled pairwise comparison improves selection, but overly small or large verifiers underperform because of discrimination or cost limits.

  • B.1 Wall-Clock vs. NFE on Additional Benchmarks: Under equal runtime, the fixed-NFE advantage of Breadth-First Search reverses because frequent verifier calls delay reaching the same NFE budget and reduce exploration.This reversal is reported on both GenEval and UniGenBench with Wan2.1 1.3B.
  • B.2 Reward Score Trajectories: 1.22: ImageReward verification rises from the 0.331 no-scaling baseline within the first candidate batch, then remains effectively flat over the remaining 300 s.The trajectory suggests early saturation and nearly indistinguishable scores among later candidates.
  • Effect of Verification Budget on Multi-Stage Filtering: Increasing pairwise verification budgets consistently improves selection quality over pointwise-only scoring, with the benefit widening as candidate pools grow.Exhaustive and high budgets outperform pointwise-only scoring across the full time range, whereas minimal and low budgets provide limited benefit.
  • Effect of Verifier Model Choice: Qwen2.5-VL-7B and InternVL3-9B provide the best verifier performance under wall-clock constraints, balancing discrimination with inference cost.The comparison uses Flash-BoN with VLMs spanning different model families and scales.
  • Effect of Verifier Model Choice: The Qwen2.5-VL-72B verifier’s higher inference cost reduces the number of candidates explored and verified, producing lower overall performance despite stronger per-comparison judgments.This trade-off reinforces that verifier cost matters under fixed wall-clock constraints.
  • Effect of Verification Budget on Multi-Stage Filtering: Pairwise-budget scaling provides meaningful gains complementary to scaling exploration through cheaper drafts.The evidence links improved verification compute with the broader exploration axis used by Flash-BoN.
  • Alternative Pairwise Ranking Strategies.: Hybrid and Multi-Stage ranking, which combine pointwise screening with pairwise refinement, consistently outperform alternatives under wall-clock constraints.Full Pairwise is penalized by quadratic cost at small budgets, while Knockout and Top-k Maintenance lack sufficient comparison structure.
  • Effect of Verifier Model Choice: The Qwen2.5-VL-3B verifier lacks sufficient discrimination for reliable pairwise judgments, despite its reduced size and corresponding wall-clock savings.The smaller model is evaluated as a 2× reduction relative to the default verifier.

B.4 Component Ablations: Contribution of Each Acceleration Knob … D Qualitative Examples

Flash-BoN’s three acceleration knobs expand the draft candidate pool, but their speed–fidelity trade-off must be calibrated per model. The method remains effective for four-step distilled models and benefits from discrete optimization, Pareto selection, and diverse calibration and verification choices.

  • B.4 Component Ablations: Contribution of Each Acceleration Knob: Flash-BoN combines timestep stopping, layer skipping, and activation proxies, progressively showing that each knob contributes to the full draft configuration.The ablation isolates each knob and tests whether discrete optimization is necessary.
  • B.4 Component Ablations: Contribution of Each Acceleration Knob: 2.58× activation proxies and 1.43× layer skipping raise Wan2.1’s iso-quality speedup, while random shortcuts underperform BoN with AUC 0.551 / 0.548.Early timestep stopping recovers most of the random-configuration gap, and additional knobs improve both AUC and speedup.
  • B.4 Component Ablations: Contribution of Each Acceleration Knob: A larger candidate pool improves exploration within fixed budgets, while pairwise verification preserves reliable selection as the pool grows.The combination links acceleration to broader search without abandoning verification.
  • B.4 Component Ablations: Contribution of Each Acceleration Knob; C.1 Configuration for Precomputation: ∼3 hours per model is required for discrete calibration, which generalizes across benchmarks but transfers only partially across models; FLUX.1-dev applied to Wan2.1 14B yields AUC 0.61.The random-configuration collapse shows that stacking acceleration knobs without model-specific calibration is insufficient.
  • B.5 Few-Step Distilled Models: 7.5× faster, Flash-BoN reaches BoN’s 150 s score on FLUX.1-schnell, a four-step distilled model evaluated under matched wall-clock budgets.The dedicated configuration uses S′=3, Lskip={40, . . . , 44}, and ffull=2.
  • C Additional Optimization Details; C.1 Configuration for Precomputation; Discrete Optimization: Flash-BoN selects ϕ∗ through black-box discrete optimization over generation speed and draft quality, using 120 held-out calibration prompts and wall-clock speedup plus LPIPS similarity.The optimization balances runtime and output fidelity against the full baseline.
  • Discrete Optimization; Pareto Selection: 200 iterations per model typically complete in 2–4 hours, and Pareto selection maximizes speedup subject to Sim(ϕ) ≥0.6.Dual annealing handles the non-differentiable mixed discrete and ordinal search space.
  • Effect of Discrete Optimizer; Effect of Calibration Prompts; D Qualitative Examples: Dual annealing, Bayesian optimization, and differential evolution converge to remarkably similar configurations, while in-distribution calibration produces comparable 3.9× speedup with slightly higher fidelity.Using ImageReward for both verification and evaluation can select prompt-violating images, illustrating same-metric bias.

D.1 Correlation between Verifier and Generation Performance … E System Prompts

The supplementary analyses show that verifier reliability and candidate-pool diversity are central to effective selection and training, while the verification prompts enforce strict prompt adherence through pointwise and pairwise scoring. Expanded rollouts improve difficult-prompt training signals by broadening output-space coverage and increasing the chance of finding positive examples.

  • D.1 Correlation between Verifier and Generation Performance: Verifier scores can mislead when the verifier is disproportionately weaker than the generator, so alignment with actual generation quality should be manually inspected.This includes cases where a recent generator is evaluated by a much older reward model.
  • D.1 Correlation between Verifier and Generation Performance: Figure 19 shows that expanded drafts can supply correct rear-view giraffe examples absent from the original rollouts, giving the optimizer positive examples to reinforce.The example concerns the prompt “The rear end of a giraffe with a bird sitting on it.”
  • D.2 GRPO Expanded Rollout: Expanded 2G=16 draft pools cover a wider output-space region than baseline G=8 random rollouts, increasing the chance of finding a correct positive example.Such examples provide an informative training signal for the optimizer when baseline rollouts miss high-signal candidates.
  • D.3 Diversity Qualitative Examples: Candidate-pool diversity correlates strongly with final selection quality, with Pearson r=0.75 across time budgets.Flash-BoN explores visually diverse cheap drafts before selectively refining the best candidate, whereas guided search produces fewer, more similar candidates.
  • E System Prompts: Pointwise and pairwise verification outputs follow fixed formats: pointwise responses use exactly three newline-separated lines, and pairwise responses report reasoning, winner, and confidence.The pairwise format specifies Winner as A, B, or TIE and Confidence as high, medium, or low.
  • D.3 Diversity Qualitative Examples: BFS particles converge to near-identical outputs, while BoN-style candidates explore diverse compositions, layouts, and styles from the same prompt.The contrast is illustrated through imperceptible pixel differences among BFS outputs versus visibly varied BoN-style candidates.
  • E System Prompts: The pairwise verifier compares two images on requested objects and visual elements, focusing only on prompt adherence rather than general image quality.It selects the slightly better image when both are close and reserves TIE for truly indistinguishable cases.
Loading 2607.04461v1…