PaperlayerView exact revision on arXiv

Source-linked AI summary

Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps

Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, Saining Xie

arXiv:2501.09732v1cs.CV

TL;DR

Diffusion models can scale inference computation through denoising steps, but performance gains often plateau. The paper searches for better sampling noises using verifier and algorithm choices, finding substantial improvements whose optimal configurations vary by task, while verifier bias can induce verifier hacking.

  • Problem

    Performance gains from scaling diffusion inference through denoising steps tend to plateau, motivating evidence about scaling computation through other mechanisms.

  • Method

    The paper frames inference-time scaling as a search for better diffusion sampling noises, varying verifiers that provide feedback and algorithms that find candidates.

  • Results

    Scaling compute through search significantly improves performance across model sizes and generation tasks, with optimal verifier–algorithm configurations varying by task.

  • Takeaways & Limitations

    Different vision generation tasks require search setups aligned with their evaluation goals rather than a universal verifier–algorithm configuration.

  • Takeaways & Limitations

    Verifier-driven search can overfit verifier bias, reducing sample variance and causing mode collapse as compute increases.

Abstract

from arXiv · show

Generative models have made significant impacts across various domains, largely due to their ability to scale during training by increasing data, computational resources, and model size, a phenomenon characterized by the scaling laws. Recent research has begun to explore inference-time scaling behavior in Large Language Models (LLMs), revealing how performance can further improve with additional computation during inference. Unlike LLMs, diffusion models inherently possess the flexibility to adjust inference-time computation via the number of denoising steps, although the performance gains typically flatten after a few dozen. In this work, we explore the inference-time scaling behavior of diffusion models beyond increasing denoising steps and investigate how the generation performance can further improve with increased computation. Specifically, we consider a search problem aimed at identifying better noises for the diffusion sampling process. We structure the design space along two axes: the verifiers used to provide feedback, and the algorithms used to find better noise candidates. Through extensive experiments on class-conditioned and text-conditioned image generation benchmarks, our findings reveal that increasing inference-time compute leads to substantial improvements in the quality of samples generated by diffusion models, and with the complicated nature of images, combinations of the components in the framework can be specifically chosen to conform with different application scenario.

1. Introduction

Diffusion models offer inference-time flexibility, but increasing denoising steps alone often plateaus. This work extends inference scaling through search over better noises, organized by verifier and algorithm choices.

  • Evaluation: The framework compares inference scaling using FID and IS on ImageNet and CLIPScore and Aesthetic Score on DrawBench.Figure 1 reports substantial improvements over purely increasing denoising steps.
  • Motivation: Inference-time scaling can improve diffusion-model performance beyond increasing denoising steps, whose gains tend to plateau after a certain NFE budget.The framework instead allocates computation to search for better noise candidates during sampling.
  • Framework: The proposed search framework uses verifiers for feedback and algorithms for finding better noise candidates.The considered algorithms include Random Search, Zero-Order Search, and Search over Paths.
  • Evaluation: The framework is evaluated on class-conditioned ImageNet generation and larger-scale text-conditioned image generation.The text-conditioned setting requires multiple verifiers because image quality involves diverse aspects and text conditioning contains rich information.
  • Findings: No verifier–algorithm configuration is universally optimal; the best search setup varies across generation tasks.The paper analyzes combinations of verifiers and algorithms across tasks and model sizes.

2. Background and Motivation

Diffusion models reverse a noise-injection process through repeated denoising, with sampling governed by learned score functions and stochastic or deterministic dynamics. Because different noises map to different final samples, searching for preferable noises offers another inference-time scaling dimension.

  • Diffusion models: Diffusion models learn to reverse a reference process that adds Gaussian noise at different noise levels to clean data.Generation starts from nearly pure noise and progressively denoises toward a clean sample.
  • Diffusion models: Sampling uses an ODE or SDE whose dynamics depend on score functions at different noise levels.The model is trained to approximate the ground-truth score functions, while σ(t) specifies the noise schedule.
  • Randomness from noise: Diffusion sampling has explicit randomness from initial noise and, optionally, later noise injected during procedures such as SDE and Restart Sampling.Despite this randomness, deterministic model evaluations create a fixed mapping from each noise choice to its final sample.
  • Randomness from noise: Some noise choices produce better generations than others, making noise search an additional axis for scaling NFEs.This axis complements allocating computation to denoising steps alone, whose gains can plateau quickly.

3. How to Scale at Inference Time

The framework scales diffusion inference by searching over sampling noises, organizing the design space around verifier feedback and candidate-search algorithms. Experiments show that search can improve generation quality, while verifier biases and search locality make component choice important.

  • Framework: The framework formulates inference-time scaling as a search problem over sampling noises, with verifiers scoring candidates and algorithms finding better ones.Search cost is measured by additional model forward passes, or NFEs.
  • Search Algorithms: Random Search samples Gaussian noises, generates candidates, and selects the highest-scoring result as a one-time Best-of-N strategy.Its primary scaling axis is the number of candidate noises.
  • Search Verifiers: Oracle-verifier Random Search substantially improves both FID and IS as search NFEs increase across guidance weights.The oracle setting is a proof of concept because final evaluation procedures are generally unavailable in practice.
  • Search Verifiers: Supervised verifiers improve IS over scaling denoising steps alone, but point-wise classifier logits can reduce sample variance and cause mode collapse as compute increases.This verifier bias, accelerated by unconstrained Random Search, is termed Verifier Hacking.
  • Search Verifiers: Self-Supervised Verifiers use feature-space similarity as a surrogate for classification logits and exhibit effective scaling without extra conditioning information.The similarity score correlates strongly with classifier logits at low noise and in final clean samples.
  • Search Algorithms: Zero-Order Search and Search over Paths alleviate FID’s diversity issue while maintaining scaling Inception Score, with different compute advantages depending on N.For Search over Paths, small N is more efficient at small budgets, whereas large N gains an advantage at higher compute.

4. Inference-Time Scaling in Text-to-Image

Text-to-image experiments show that inference-time search generally improves sample quality, but the best verifier depends on the application’s evaluative requirements. Search benefits extend across models and budgets, while verifier biases can create metric trade-offs.

  • Experimental setup: The text-to-image evaluation uses FLUX.1-dev on DrawBench and T2I-CompBench, with multiple supervised verifiers and an additional LLM evaluator.DrawBench contains 200 prompts across 11 categories; T2I-CompBench uses 1,800 validation prompts and generates two images per prompt.
  • Scaling results: Search with all verifiers generally improves sample quality, although the improvement pattern varies across setups.The experiments support choosing search configurations according to different application scenarios.
  • DrawBench: ImageReward and Verifier Ensemble consistently improve all DrawBench evaluation metrics, making them suitable for generalized generation tasks.Aesthetic and CLIP verifiers are less desirable for broad multi-aspect evaluation because of verifier hacking.
  • Verifier-task alignment: Aesthetic and CLIP search overfit their respective biases, so improving one verifier can degrade the other’s evaluation metrics.Aesthetic Score favors visual quality, whereas CLIP prioritizes visual-text alignment; the search method preserves the model’s learned score function and avoids total sample collapse.
  • Scaling and generalization: Increasing search budgets produces similar scaling behavior across evaluation metrics and generalizes the search method across different models.The reported pattern appears on DrawBench and is described as similar to the ImageNet setting.
  • T2I-CompBench: On T2I-CompBench, ImageReward outperforms Verifier Ensemble across all evaluation categories, while Aesthetic search can minimally improve or degrade metrics.The benchmark emphasizes prompt correctness, including colors, object relationships, and compositions, rather than pure visual quality.

5. Axes of Inference Compute Investment

The framework scales diffusion inference by allocating compute across search iterations, computation per iteration, and final generation. Experiments show that search can improve smaller models and text-conditioned generation, while compute allocation exhibits diminishing returns and scenario-dependent behavior.

  • Increasing search iterations lets selected noises approach the verifier-optimal set and can substantially improve performance.
  • Compute per search iteration: Smaller NFEs/iter enable faster search convergence, whereas larger NFEs/iter converge more slowly but achieve higher final performance.
  • Compute per search iteration: When NFEs/iter ≥50, further increases provide minimal gains despite additional computation; the experiments therefore use 50 for ImageNet and 30 for text-to-image search.
  • Compute of final generation: Final generation uses 250 NFEs for ImageNet and a 30-step sampler for text-to-image because further denoising quickly reaches a performance plateau.
  • With fixed inference compute, searched small models can outperform larger models without search, although benefits are limited when the small model’s baseline lags significantly.
  • With one-tenth of the compute, PixArt-Σ outperforms FLUX-1.dev without search; with roughly double the compute, it surpasses it by a significant margin.

6. Related Work

Related work studies inference-time scaling, diffusion sample selection, and methods that learn preferable noises. Existing approaches either search over generated samples or shift computation into training, introducing curation, tuning, or performance concerns.

  • Table 4 compares searched PixArt-Σ with FLUX-1.dev without search using FLUX’s single-sample compute as the scaling unit, with estimates that may be imprecise.
  • Scaling test-time compute: Inference-time scaling in LLMs uses additional inference computation, often through search, to produce higher-quality and more contextually appropriate responses.
  • Sample selection and optimization in diffusion models: Diffusion sample-selection methods use Random Search with predefined metrics, including pretrained VQA and human-preference models, to guide selection.
  • Sample selection and optimization in diffusion models: Neural approaches approximate the distribution of good noises by transforming random noises through guided DDIM inversion and training predictors on the resulting pairs.
  • Sample selection and optimization in diffusion models: These learned-noise approaches shift computation from test time to one-time training but require additional dataset curation and parameter tuning, with unsatisfying performance in some scenarios.

7. Conclusion

The paper concludes that search-based inference-time scaling can improve diffusion generation across model sizes and tasks, but effective configurations depend on the task and compute budget. Verifier biases motivate designing verifiers aligned with specific vision-generation objectives.

  • Search-based compute scaling can significantly improve performance across various diffusion model sizes and generation tasks.
  • Different inference-time compute budgets can produce varied scaling behavior, so optimal verifier–algorithm configurations vary by task rather than following one universal solution.
  • Verifier–task alignment matters because different verifiers possess inherent biases, motivating more carefully designed verifiers for specific vision-generation tasks.

Appendices

The appendices specify model, sampling, search, hyperparameter, and verifier settings. They also document trade-offs in search behavior, including overfitting from large step sizes and metric-dependent effects from longer paths.

  • Experimental settings: The experiments use pretrained SiT-XL, FLUX.1-dev, and PixArt-Σ weights, while SiT-B and SiT-L are self-trained with matching architectures and configurations.
  • Sampling settings: Random Search samples i.i.d. Gaussian noise candidates, generates their samples with an ODE solver, and selects the candidate receiving the highest verifier score.
  • Sampling settings: Zero-Order Search scales primarily through iterations K, with neighbors N and step size λ providing additional tunable compute dimensions.
  • Search settings: Large λ can overfit the verifier’s high-scoring region, reducing diversity; FID increases beyond 10^3 NFEs even though this setting has the best Inception Score among three settings.
  • Search Over Paths: Search Over Paths uses M and N as primary and secondary scaling axes, while increasing path length can improve FID but has marginal effect on Inception Score.
  • Verifier settings: ImageNet verifiers include FID, IS, CLIP, and DINO, while text-to-image verifiers include Aesthetic and CLIPScore.

A.5. Evaluation Setting

The evaluation uses distinct protocols for ImageNet, DrawBench, and T2I-CompBench, with task-specific sample counts, prompts, evaluators, and metrics.

  • ImageNet evaluation computes FID and Inception Score from 50000 synthesized samples using randomly generated conditions and global batch size 256.
  • DrawBench searches for one noise per prompt and averages evaluator scores across 200 prompts, except for the LLM Grader protocol.
  • The LLM Grader rates Accuracy to Prompt, Originality, Visual Quality, Internal Consistency, and Emotional Resonance from 0 to 100, averaging them into the final metric.
  • T2I-CompBench searches for two noises per prompt, generates two samples, and evaluates six categories with BLIP-VQA, UniDet, and weighted combined scores.

B. Verifier Hacking Leads to Degeneracy in Evaluation Metrics

Searching against verifiers can improve their targeted metrics while causing overfitting, reduced diversity, or degeneration in other evaluation measures. Population-aware verification is identified as a potential future remedy.

  • Excessive optimization against a verifier can cause degeneration in other evaluation metrics, an overoptimization issue also observed during diffusion-model finetuning.
  • FID suddenly increases after enough search iterations against DINO or CLIP classification logits despite continuously improving Inception Score.
  • Precision increases while Recall decreases with search iterations, indicating improved sample quality alongside reduced sample-set diversity.
  • Per-noise selection concentrates samples in verifier-high-scoring regions, causing verifier overfitting and reduced variance that particularly affects diversity-sensitive FID.
  • Population-based verifiers that account for the global structure of selected noises are proposed as a more fundamental solution, but left for future work.
  • Over-searching Aesthetic Score degenerates CLIPScore, and over-searching CLIPScore produces the reverse cross-metric degradation.

C. Zero-Order and First-Order Search

The paper compares gradient-free and gradient-based search for verifier-guided noise optimization. First-Order Search converges faster, but offers little sustained advantage as compute increases and costs more memory.

  • First-Order Search initializes Gaussian noise, differentiates verifier output through the diffusion ODE solver, and repeatedly updates the noise by gradient descent.
  • Gradient checkpointing reduces memory use during ODE backpropagation by recomputing activations, at the cost of an extra model forward call.
  • Updated noise is rescaled to match the norm of i.i.d. Gaussian vectors, preserving consistency between training and sampling.
  • The comparison fixes First-Order Search at η=0.01 and estimates each iteration with backpropagation as 3× the cost of a forward-only iteration.
  • First-Order Search converges faster than Zero-Order Search, but shows no significant margin under continued compute scaling and has higher memory cost.
  • Its gradient-guided updates may still benefit fine-grained objectives such as image editing, inpainting, and inverse problems.

D. Self-Supervised Verifiers have Marginal Effect in Text-to-Image Setting

Self-supervised verifiers that work well for class-conditioned ImageNet generation have weaker alignment with text-to-image evaluation metrics. The results motivate task-specific verifiers and caution against optimizing a verifier’s particular bias.

  • On DrawBench, DINO, CLIP, and SigLIP do not show the expected performance as self-supervised verifiers, unlike their surrogate role on ImageNet.
  • CLIP, DINO, and SigLIP feature-similarity scores have much weaker correlations with CLIPScore in text-to-image generation than in ImageNet.
  • Text-to-image metrics assess nuanced properties such as visual-text alignment, composition correctness, and human preferences, while feature-space trajectory curvature may capture only sampling stability.
  • Rich text conditioning and extensive fine-tuning may change sampling dynamics, making low feature-space trajectory curvature less indicative of sample quality.
  • Verifier effectiveness is highly task-dependent, motivating task-specific verifier design and caution against hacking each verifier’s bias.
  • The visual examples compare increasing denoising steps with Zero-Order Search using DINO or a Verifier Ensemble across SiT-XL, FLUX.1-dev, and PixArt-Σ settings.
Loading 2501.09732v1…