Diffusion Models for Black-Box Optimization

1. Introduction

• Limited offline-dataset coverage is a central challenge, and forward surrogates can struggle to generalize outside the dataset.Forward methods learn input-to-value mappings and then optimize the learned surrogate or use it as a proxy oracle.Source: §1 · p. 1
• Inverse approaches map function values to inputs, but learning their one-to-many mappings is challenging in high-dimensional domains.Prior GAN-based approaches can also inherit training instability and mode collapse.Source: §1 · p. 1
• DDOM uses conditional diffusion models to learn the inverse mapping from observed function values to points in the domain.The model is trained on an offline dataset and conditioned on function values.Source: §1 · pp. 1–2
• DDOM reweights training toward higher-value data and uses classifier-free guidance at test time to strengthen conditioning.The guidance combines conditional and unconditional score components; Figure 1 depicts conditioning on the dataset maximum to sample candidate points.Source: §1 · p. 2, Figure 1 · p. 2
• DDOM outperforms all forward and inverse baselines on Design-Bench, achieving the highest average rank (2.8).The benchmark includes real-world tasks across discrete and continuous domains, with varied data sizes and dimensionalities.Source: §1 · p. 2

2.1. Problem Statement

• Offline BBO forbids function evaluations during training and uses an offline dataset, with a small evaluation budget for proposed candidates.The problem is to find an input maximizing the unknown black-box function.Source: §2.1 · p. 2

2.2. Diffusion Models

• Diffusion iteratively adds noise to data and learns to reverse the process, mapping noise back to data through a reverse-time SDE.A time-dependent neural network estimates the score function used in the reverse process.Source: §2.2 · p. 3

2.3. Classifier-free Guidance

• Classifier-free guidance combines conditional and unconditional score estimates, with its mixing parameter trading off coverage and fidelity during sampling.For nonnegative guidance weights, the score prioritizes samples that respect the conditioning information; one model can estimate both scores by randomly zeroing the conditioning value during training.Source: §2.3 · p. 3

3. Denoising Diffusion Optimization Models

• DDOM uses diffusion models to learn a conditional distribution over inputs given function values, addressing the one-to-many inverse mapping in offline BBO.The proposed approach is motivated by generative models’ ability to learn high-dimensional distributions and diffusion’s conditioning controls, unlike GAN approaches that can be hard to train and suffer mode collapse.Source: §3 · p. 3
• Training weights bins by their point counts and average function values, prioritizing bins with more data and higher-quality points.The bin weighting is a function of y and complements the original time-dependent weighting in the denoising objective.Source: §3 · pp. 3–4
• Inputs and function values are normalized to fit a standard normal; for discrete tasks, inputs are mapped to log probabilities rather than projected with a VAE.The paper reports that this discrete-input procedure performs reasonably well in its experiments.Source: §3 · p. 4
• At test time, DDOM conditions on the dataset’s maximum function value and uses classifier-free guidance to sample up to Q candidate points.The described evaluation procedure samples from the reverse SDE using a second-order Heun solver.Source: §3 · p. 4

4.1. Toy Branin Task

• DDOM reproduces both the high-value Gaussian-mixture and uniform Branin datasets, with denoising moving samples toward optima in the former and increasing diversity in the latter.The Gaussian-mixture samples’ mean function value rises and variance falls over denoising; uniform-data sample variance rises.Source: §4.1 · p. 5, Figure 3 · p. 5
• Conditioned DDOM learns Branin inverse contours close to grid-search contours and produces mean function values near the conditioning value.For a dataset with its top 10% of points removed, it proposes points above the dataset maximum; peak best and mean performance occurs when conditioning equals that maximum.Source: §4.1 · p. 6

4.2. Design-Bench

• The Design-Bench evaluation covers six high-dimensional real-world tasks, compares DDOM with canonical and learned baselines, and uses a query budget of 256.NAS is excluded for compute limits and Hopper for known inconsistencies; results are normalized using a larger unseen dataset that is not used for training.Source: §4.2 · pp. 6–7
• 2.8 average rank and 0.787 average score make DDOM the top-ranked method across six Design-Bench tasks; it places first on two tasks and second on two others.It beats the next-closest baseline by 11% on Superconductor; CMA-ES has five times DDOM’s standard deviation.Source: §4.2 · p. 7, Table 1 · p. 7

4.3. Ablations

• Across six tasks, reweighted DDOM outperforms its non-reweighted version, with significant score improvement on most Design-Bench tasks.Table 2 reports normalized scores averaged across five seeds.Source: §4.3 · p. 7, Table 2 · p. 7
• On Superconductor, predicted and conditioned values remain strongly correlated even at large conditioning values, though generalization beyond the dataset maximum is difficult.The plotted comparison uses the ideal y = x line and the dataset maximum as references.Source: §4.3 · p. 7, Figure 6 · p. 7
• On Superconductor, classifier-free guidance improves over no guidance, and higher guidance weights increase the rate at which predicted values rise.The no-guidance condition is γ = 0.Source: §4.3 · p. 8, Figure 7 · p. 8

Active BBO

• Active BBO methods commonly use Gaussian-process surrogates and uncertainty-aware acquisition, while DDOM operates without training-time function queries.The paper also contrasts DDOM with Neural Diffusion Processes, noting its offline setting, unrestricted model architecture, and classifier-free guidance.Source: §5 · p. 8

Offline BBO

• Offline BBO approaches include forward surrogates, inverse generative models such as GANs, and CbAS's iterative VAE-based density adaptation.Design-Bench is presented as a comprehensive real-world benchmark for offline BBO.Source: §5 · p. 8

Diffusion Models

• Diffusion models have been applied to images, video, audio, and reinforcement-learning planning, with related work exploring conditioning and sampling strategies.The paper frames these developments as relevant to DDOM's modeling and sampling choices.Source: §5 · p. 8

6. Summary

• DDOM achieves the highest average rank, 2.8, and outperforms forward and inverse baselines on Design-Bench.The paper also reports that DDOM is more stable to train and less susceptible to mode collapse than MINs.Source: §6 · p. 8
• DDOM uses loss reweighting to prioritize high-function-value offline points and classifier-guided sampling to strengthen test-time conditioning.The approach learns an inverse mapping from function values to points using conditional diffusion models.Source: §6 · p. 8
• Diffusion models sample relatively slowly, which may limit their use in some real-time applications.The study focuses on offline BBO and identifies warm-starting a subsequent diffusion-based online method as future work.Source: §6 · p. 9

A.2. Additional Experimental details

• The code builds on Huang et al. (2021)'s score-based diffusion implementation and is released under the MIT licence.The text says the code is provided through an anonymized link.Source: §A.2 · p. 12
• Training used one RTX A5000 GPU, five seeds, batch size 128, and 1,000 epochs for each Design-Bench task.The reported reweighting settings were K = 0.01 * N and τ = 0.1.Source: §A.2 · p. 12
• Evaluation used a budget of Q = 256 points, with the dataset maximum as the conditioning value.The dataset maximum was selected as a simple choice that does not require prior dataset knowledge.Source: §A.2 · p. 12

A.3. Unnormalized results

• Table 5 presents the unnormalized counterparts of the results shown in Table 1.Source: §A.3 · p. 12, Table 5 · p. 13

A.4. HopperController

• Hopper was omitted because offline dataset values disagreed with oracle values, a discrepancy associated with a known Design-Bench bug.The paper notes that Krishnamoorthy et al. (2023) observed a similar issue.Source: §A.4 · p. 13

• On Ant, DDOM outperforms MINs, COMs, and Grad. Ascent at various evaluation budgets.Source: §B.1 · p. 13
• The experiment varies the evaluation budget Q to assess sensitivity of DDOM's prediction quality.Source: §B.1 · p. 13
• The authors interpret the results across budgets as evidence that DDOM consistently finds good points.Source: §B.1 · p. 13

B.2. Ablation on reweighting parameters

• τ controls the relative weight assigned to good versus bad bins, with lower τ reducing weights for low-quality bins.K and τ are smoothing parameters in the loss-reweighting formula.Source: §B.2 · p. 13
• DDOM outperforms MINs, COMs, and gradient ascent on Ant across various evaluation budgets.The comparison is reported for most budget choices and is illustrated in Figure 10.Source: Figure 10 · p. 14
• Smaller K makes bin size irrelevant to reweighting, while larger K makes weights scale roughly linearly with bin size.Figure 11 plots Superconductor function values for varying K and τ.Source: §B.2 · p. 14, Figure 11 · p. 14
• On AntMorphology, one bin produces a significant performance difference, while the difference between 32 and 64 bins is much smaller.The ablation varies the number of bins; Table 6 presents the results.Source: §B.2 · p. 14, Table 6 · p. 14

B.3. Visualizing the Design-Bench datasets

• On D’Kitty, Ant, and Superconductor, predicted points do not simply follow dataset contours, indicating that DDOM is not merely memorizing the dataset.Figure 12 colors dataset points red and predicted points blue in t-SNE plots.Source: §B.3 · p. 15, Figure 12 · p. 15

B.4. Plotting the contours of the score function

• On Branin, the learned score-function contours vary across timesteps.Figure 13 plots the contours across different timesteps.Source: §B.4 · p. 15, Figure 13 · p. 15

B.5. Effect of adding randomness to data

• On Superconductor, even noise equal to 20% of the dataset maximum causes only a moderate performance decline.The experiment adds noise to function evaluations; Table 7 describes the noise levels as a percentage of the dataset maxima.Source: §B.5 · p. 15, Table 7 · p. 16

B.6. Effect of size of the offline dataset

• On Superconductor, DDOM performs better than the dataset optima even when parts of the offline dataset are withheld.The authors interpret this result as indicating a degree of generalizability; Table 8 covers the offline-data-size ablation.Source: §B.6 · p. 16, Table 8 · p. 16

C. Societal Impact

• The authors caution that optimizers such as DDOM could be used in harmful settings, including optimizing for drugs with adverse side effects.They identify real-world deployment as a setting requiring care.Source: C. Societal Impact · p. 16