PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Group Inference for Diverse and High-Quality Generation

Gaurav Parmar, Or Patashnik, Daniil Ostashev, Kuan-Chieh Wang, Kfir Aberman, Srinivasa Narasimhan, Jun-Yan Zhu

arXiv:2508.15773v1cs.CVcs.GRcs.LG

TL;DR

Independent sampling and individual-sample scaling do not directly address applications that present users with diverse groups of outputs. The paper formulates group inference as quality-and-diversity subset selection and accelerates it with progressive pruning, reporting stronger diversity-quality tradeoffs across multiple generative tasks. Its effectiveness remains constrained by the quality and diversity of the initial candidate pool.

  • Problem

    Independent sampling can produce redundant outputs even though real-world applications commonly present users with groups of four to eight images.

    Source: §1 · p. 1

  • Method

    The method selects K samples from M candidates using a quadratic integer program that combines unary quality and binary diversity, then progressively prunes candidates using intermediate predictions.

    Source: §1 · pp. 1–2, §3.1 · p. 4, §3.2 · p. 5

  • Results

    The method consistently achieves a superior diversity-quality tradeoff and outperforms independent sampling and recent single-sample inference algorithms across text-to-image, image-to-image, image prompting, and video generation.

    Source: §1 · p. 2, §4.2 · p. 9

  • Takeaways & Limitations

    The framework treats multiple generated outputs as cohesive groups and supports efficient generation of diverse, high-quality options across several modalities.

    Source: §1 · p. 2, §5 · p. 13

    Its efficacy is constrained when the base generative model produces poor-quality or mode-collapsed candidate pools because the method selects from, rather than intrinsically enhances, those candidates.

    Source: §5 · p. 13

Abstract

from arXiv · show

Generative models typically sample outputs independently, and recent inference-time guidance and scaling algorithms focus on improving the quality of individual samples. However, in real-world applications, users are often presented with a set of multiple images (e.g., 4-8) for each prompt, where independent sampling tends to lead to redundant results, limiting user choices and hindering idea exploration. In this work, we introduce a scalable group inference method that improves both the diversity and quality of a group of samples. We formulate group inference as a quadratic integer assignment problem: candidate outputs are modeled as graph nodes, and a subset is selected to optimize sample quality (unary term) while maximizing group diversity (binary term). To substantially improve runtime efficiency, we progressively prune the candidate set using intermediate predictions, allowing our method to scale up to large candidate sets. Extensive experiments show that our method significantly improves group diversity and quality compared to independent sampling baselines and recent inference algorithms. Our framework generalizes across a wide range of tasks, including text-to-image, image-to-image, image prompting, and video generation, enabling generative models to treat multiple outputs as cohesive groups rather than independent samples.

1 Introduction

The paper addresses the mismatch between independent-sample inference and applications that present users with diverse groups of outputs. It formulates group selection around sample quality and diversity, then scales it through progressive pruning and reports improvements across generative tasks.

1 Introduction

  • Platforms commonly show four to eight images per prompt, making diverse layouts, lighting, styles, and creative options practically important.Source: §1 · p. 1
  • The method jointly improves group diversity and sample quality by selecting K outputs from M candidates with a quadratic integer program.Individual quality is represented by a unary term, while group diversity is represented by a binary term.Source: §1 · pp. 1–2
  • Figure 1 contrasts repetitive I.I.D. samples with the proposed method’s more diverse and high-quality output collections.Source: Figure 1 · p. 2
  • Progressive pruning uses intermediate denoising predictions as previews, reducing complexity from O(MT) to O(M + KT) with K = 4.The strategy iteratively removes candidates before completing denoising for the selected group.Source: §1 · p. 2
  • Across text-to-image, image-to-image, image-prompting, and video-generation tasks, the method outperforms independent sampling and recent single-sample inference algorithms.The reported outputs are more diverse and realistic under similar cost budgets.Source: §1 · pp. 2–3

2 Related Works

Prior diffusion inference methods primarily improve individual-sample quality or controllability, often reducing diversity. This work instead optimizes collective quality and diversity while targeting the computational constraints of diffusion generation.

Diffusion Inference and Guidance

  • Diffusion guidance and conditioning methods improve sample quality or controllability but often sacrifice diversity, especially under strong guidance.Source: §2 · p. 3

Related Group Inference

  • Unlike individual-sample techniques, group inference explicitly balances single-sample quality with inter-sample diversity.Source: §2 · p. 3
  • Compared with particle guidance, the method is reported to preserve quality, scale to larger image sets through pruning, and support non-differentiable scoring functions.Particle guidance is described as limited to small sets because pairwise-gradient computation is memory-intensive.Source: §2 · p. 3

Inference-Time Scaling

  • Diffusion test-time scaling must balance quality gains against high computation, per-image cost, and users’ low-latency requirements.Source: §2 · p. 3

3 Method

Scalable Group Inference selects a diverse, high-quality subset from many generated candidates by combining quality and diversity scores in a QIP, then progressively pruning candidates using intermediate predictions to reduce computation.

3.1 Formulation

  • The method selects K outputs from M candidates by maximizing individual quality and pairwise diversity in a quadratic integer program.Unary scores reward strong individual outputs, while binary scores favor dissimilar pairs; the formulation is model-agnostic and supports nondifferentiable scoring functions.Source: §3 · p. 3, §3.1 · p. 4

3.2 Progressive Pruning for Efficient Selection

  • Progressive pruning evaluates intermediate predictions, scores candidates, and repeatedly solves the QIP until only K candidates remain for final denoising.The candidate sets form a nested sequence, and remaining denoising steps are completed only for the selected samples.Source: §3.2 · p. 5
  • Intermediate predictions reliably proxy final outputs, with score correlations exceeding 0.7 after five steps for multistep models and 0.95 after one step for distilled models.This reliability enables candidates to be ranked before full denoising.Source: §3.2 · p. 5, Figure 3 · p. 5

3.3 Computational Complexity Analysis

  • The implementation initializes candidate noises, generates intermediate previews during reversed denoising, scores them, and prunes according to the selected dropping ratio.Candidates are retained through SolveQIP until the target group size is reached.Source: §3.3 · pp. 5–6
  • For M=64, K=4, ρ=0.5, and T=20, progressive filtering uses 184 versus 1280 evaluations, an approximately 85% reduction over naive sampling.The resulting overall complexity is O(M + KT), compared with O(MT) without pruning.Source: §3.3 · p. 6

4 Experiments

Across five generative models and multiple tasks, the experiments evaluate group inference on diversity–quality tradeoffs, runtime scaling, and configurable diversity objectives. The method consistently improves the tradeoff and scales more efficiently than alternatives, while its benefits depend on the initial candidate pool.

4.2 Baselines and the Diversity-Quality Tradeoff

  • Users consistently preferred our generations for both diversity and quality across FLUX.1 Dev, FLUX.1 Schnell, and Stable Diffusion 3 Medium.Low-CFG and Interval Guidance comparisons are not applicable to distilled FLUX.1 Schnell.Source: Table 1 · p. 9
  • Our method dominates the quality–diversity Pareto fronts across text-to-image, depth-conditioned generation, and image prompting.It yields better diversity at a given quality or higher quality at comparable diversity than evaluated baselines.Source: Figure 5 · p. 8, §4.2 · p. 9, Figure 6 · p. 9
  • Increasing denoising steps or independently sampling more seeds provides limited group-objective gains, whereas Group Inference quickly achieves better performance–runtime tradeoffs.The denoising-step baseline plateaus, and Inference Diffusion Scaling requires substantially more runtime for marginal improvements.Source: §4.2 · p. 9, Figure 8 · p. 10, §4.3 · p. 11

4.3 Inference Scaling Analysis

  • Increasing the initial candidate count from 4 to 128 improves both sample quality and set diversity across three base models.The quality and diversity metrics are CLIP and DINO, respectively.Source: Figure 9 · p. 11, §4.3 · p. 11

4.2 Baselines and the Diversity-Quality Tradeoff

  • Particle Guidance increases diversity but sharply reduces sample quality and requires gradients, memory, and differentiable diversity potentials.Its altered sampling trajectories can push outputs off the learned data manifold.Source: §4.2 · p. 11

4.3 Inference Scaling Analysis

  • Progressive filtering achieves comparable group scores to full denoising while reducing runtime by up to 73% across architectures.The strategy uses intermediate predictions to prune candidates before full denoising.Source: §4.3 · p. 11, Figure 10 · p. 12

4.4 Different Diversity Objectives

  • Swapping the binary diversity objective targets different visual variations while keeping the unary CLIP quality term identical.Color dissimilarity produces varied color schemes, whereas DINO-based diversity produces greater pose and camera-angle variation.Source: §4.4 · p. 12, Figure 11 · p. 12
  • The method cannot create diversity absent from the initial candidate pool, such as color variation when the base model generates only red or black-and-white images.Failure cases show that candidate-pool diversity constrains the attainable output diversity.Source: Figure 12 · p. 13

5 Discussion, Broader Impacts, and Limitations

The paper reports that scalable group inference improves diverse, high-quality generation across tasks, while its effectiveness depends on the base model and the computational cost of score evaluation.

Discussion and broader impacts

  • Scalable group inference significantly improves group diversity and quality compared with existing baselines across various generative tasks.The method is presented as a user-centric approach for efficiently generating diverse, high-quality options, though increased efficiency may also facilitate varied misleading content.Source: §5 · p. 13

Limitations

  • Its effectiveness is constrained when the base model produces poor-quality or mode-collapsed candidate pools because the method selects from, rather than intrinsically enhances, those candidates.The limitation is illustrated in Figure 12.Source: §5 · p. 13
  • Runtime benefits are reduced when unary quality or pairwise diversity scores are computationally intensive to evaluate over large candidate sets.Source: §5 · p. 13

A Analysis

The analysis examines runtime, pruning, decoding, score functions, and model-dependent behavior to identify how scalable group inference balances quality, diversity, and cost.

Runtime breakdown

  • Across runtime settings, denoising dominates cost, while QIP solving and score computation are negligible compared with the denoising-transformer forward pass.The analysis varies initial candidate size and output set size using FLUX.1 Dev.Source: A Analysis · p. 17

Different ρ values

  • For FLUX.1 Dev, ρ = 0.5 balances higher scores and inference cost, whereas ρ = 0.1 or 0.25 is too aggressive and ρ = 1.0 is costly.The paper uses ρ = 0.5 for all FLUX.1 Dev experiments.Source: A Analysis · p. 17
  • FLUX.1 Schnell tolerates ρ = 0.1 without a noticeable score decrease, attributed to more reliable intermediate predictions in the distilled model.Source: A Analysis · p. 17

Efficient decoding

  • Using an efficient decoder improves runtime for both FLUX.1 Dev and FLUX.1 Schnell without sacrificing score.Source: A Analysis · p. 17

Evaluation with different score functions

  • Across models, the proposed group inference achieves a better quality–diversity trade-off with alternative score functions, although particle guidance produces artifacts despite a slightly better BLIP2 score for Stable Diffusion 3.The comparison covers FLUX.1 Dev, FLUX.1 Schnell, and Stable Diffusion 3 medium.Source: A Analysis · p. 17

B Additional Results

Additional results show that the method consistently produces more diverse outputs without quality degradation, and that intermediate-score correlations extend beyond the primary model.

Qualitative results

  • Across multiple models and tasks, qualitative examples show more diverse samples without degradation in quality compared with baselines.The comparisons include Low CFG, Interval Guidance, and particle guidance baselines.Source: B Additional Results · p. 17

Correlation analysis

  • FLUX.1 Depth and SynCD exhibit correlations between final-image and intermediate-image scores similar to those reported in the main analysis.The analysis uses CLIP text–image similarity as the unary score and DINO diversity as the binary score.Source: B Additional Results · p. 17

C Implementation Details

This section points to implementation and dataset details while the runtime analysis shows how computational cost changes with candidate-pool and output-set sizes.

Implementation details

  • Section C.1 provides implementation details and hyperparameters for the main figures, while Section C.2 lists datasets used for each task.Source: C Implementation Details · p. 17

Runtime

  • Runtime breakdowns vary the initial candidate count M and output set size K, with denoising dominating runtime across settings.The left plot fixes output size to 4, while the right fixes the initial candidate size to 200.Source: Figure 13 · p. 18

C.1 Baselines

The experiments compare multiple inference-time baselines and vary group-inference settings to assess quality–diversity trade-offs and runtime scaling.

Baselines

  • The study compares increased diffusion steps, classifier-free guidance, interval guidance, Particle Guidance, Inference Diffusion Scaling [2], and group inference.Group inference varies λ with fixed M to expose the diversity–quality trade-off, while other baselines use the specified search ranges.Source: §C.1 · p. 18
  • Runtime experiments fix λ = 1 and vary the candidate-set size M from 4 to 128.This setup studies performance across different inference budgets and runtimes.Source: §C.1 · p. 18
  • CLIP text-image similarity and DINO diversity are the default unary and binary scores, with SynCD using DINO target-image similarity as its unary score.Figure 11 varies the binary score while keeping the CLIP text-image similarity unary score fixed.Source: §C.1 · p. 18

C.2 Dataset

The supplementary evaluations span text-to-image, depth-to-image, and encoder-based image customization, with additional ablations, metric checks, runtime studies, and qualitative galleries.

Datasets

  • Text-to-image experiments use all 553 GenEval prompts, while depth-to-image uses 250 COCO 2017 validation images and customization uses 400 DreamBooth images.Depth maps are computed with Depth Anything Large for the depth-to-image experiments.Source: §C.2 · p. 18

Ablations

  • Supplementary ablations examine dropping ratios, progressive pruning, and efficient intermediate-prediction decoding for FLUX.1 Dev and FLUX.1 Schnell.Progressive-pruning evaluation reports function evaluations and wallclock runtime on one NVIDIA H100.Source: Figure 14 · p. 19, Figure 15 · p. 19, Figure 16 · p. 19

Additional evaluations

  • Additional evaluations test quality–diversity Pareto fronts with unseen Image Reward, Depth Diversity, BLIP2 [53], and CLIP Diversity metrics, alongside runtime and qualitative comparisons.The galleries report more diverse backgrounds, styles, textures, object poses, and styles than independent sampling in depth-to-image and customized generation.Source: Figure 17 · p. 20, Figure 19 · p. 21, Figure 20 · p. 21

Related papers

Loading 2508.15773v1…