PaperlayerView exact revision on arXiv

Source-linked AI summary

Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization

Brandon Trabucco, Xinyang Geng, Aviral Kumar, Sergey Levine

arXiv:2202.08450v1cs.LG

TL;DR

Offline MBO seeks designs from existing data when evaluating new proposals is costly, but progress is difficult to track without shared benchmarks. Design-Bench provides diverse tasks, a unified evaluation protocol, and reference implementations; its evaluation finds prior offline methods successful across varied tasks, with classical and gradient-ascent baselines also competitive in reported settings.

  • Problem

    Offline MBO lacks commonly adopted benchmarks and evaluation protocols, while its no-query setting presents challenges distinct from online optimization.

    Source: §1 · p. 2, What makes offline MBO especially… · p. 3

  • Method

    Design-Bench provides diverse offline MBO tasks, standardized evaluation, and reference implementations of prior methods and baselines.

    Source: §1 · p. 2, §8 · p. 9

  • Results

    Three prior offline MBO methods solve a wide range of tasks, while CMA-ES is competitive with several sophisticated methods on 4 of 8 tasks.

    Source: §7 · p. 8

  • Takeaways & Limitations

    The benchmark’s diverse tasks are intended to represent real-world MBO challenges and assess whether methods generalize across domains rather than specialize to one task.

    Source: §1 · p. 2

    Expert-model oracle evaluation makes benchmarking practical but diminishes realism because proxy true functions may not always be accurate.

    Source: §4 · p. 4

Abstract

from arXiv · show

Black-box model-based optimization (MBO) problems, where the goal is to find a design input that maximizes an unknown objective function, are ubiquitous in a wide range of domains, such as the design of proteins, DNA sequences, aircraft, and robots. Solving model-based optimization problems typically requires actively querying the unknown objective function on design proposals, which means physically building the candidate molecule, aircraft, or robot, testing it, and storing the result. This process can be expensive and time consuming, and one might instead prefer to optimize for the best design using only the data one already has. This setting -- called offline MBO -- poses substantial and different algorithmic challenges than more commonly studied online techniques. A number of recent works have demonstrated success with offline MBO for high-dimensional optimization problems using high-capacity deep neural networks. However, the lack of standardized benchmarks in this emerging field is making progress difficult to track. To address this, we present Design-Bench, a benchmark for offline MBO with a unified evaluation protocol and reference implementations of recent methods. Our benchmark includes a suite of diverse and realistic tasks derived from real-world optimization problems in biology, materials science, and robotics that present distinct challenges for offline MBO. Our benchmark and reference implementations are released at github.com/rail-berkeley/design-bench and github.com/rail-berkeley/design-baselines.

1. Introduction

Offline MBO seeks strong designs when evaluating new proposals is costly or impossible, using existing design–outcome data instead of active queries. Design-Bench addresses inconsistent evaluation with a diverse task suite and standardized protocol.

1. Introduction

  • Offline MBO uses existing design–objective records when evaluating new proposals through simulations or experiments is costly and active feedback is unavailable.Unlike online optimization, it must generate designs without querying the true objective for feedback.Source: §1 · p. 1
  • Online optimization often depends on iterative objective queries, while offline settings make distribution shift a major challenge.Online optimizers may query unseen designs regardless of their quality; without those queries, optimism is no longer desirable.Source: §1 · pp. 1–2
  • Design-Bench provides diverse tasks and a standardized evaluation protocol to address inconsistent comparisons across offline MBO methods.The suite spans domains from synthetic biology to robotics and includes reference implementations of existing methods.Source: §1 · p. 2

2. Offline Model-Based Optimization (Offline MBO) Problem Statement

Offline MBO maps a static dataset of designs and measurements to candidates evaluated against the true objective, without querying that objective during optimization. It can improve on observed designs, but must balance proxy exploitation against staying near the data.

2. Offline Model-Based Optimization (Offline MBO) Problem Statement

  • Offline MBO gives an algorithm a static dataset of designs and objective measurements, then evaluates its generated candidate against the true objective.The problem is commonly evaluated using the P percentile of the top K candidates rather than a single design.Source: §2 · p. 2
  • Compositional objectives can let offline MBO combine individually observed optimal variable partitions into a design better than the dataset’s best sample.The toy quadratic example illustrates a found design with a higher objective than observed dataset points.Source: §2 · pp. 2–3, Figure 2 · p. 2

What makes offline MBO especially challenging?

  • Optimizing a learned objective proxy can produce out-of-distribution designs that fool the model into predicting high values.The algorithm cannot query the true objective on its proposed candidates to check those predictions.Source: What makes offline MBO especially… · p. 3
  • Staying close to training data can avoid out-of-distribution inputs, but may miss high-value designs that lie in the dataset’s tail.A good method must balance proximity to data against the need to deviate from it; high dimensionality and sparse data exacerbate this conflict.Source: What makes offline MBO especially… · p. 3

3. Related Work

Offline MBO differs from online optimization because it uses existing data without queries to the true objective. Design-Bench responds to inconsistent task-specific evaluations with a standardized suite spanning varied domains and design spaces.

3. Related Work

  • Online and active optimization methods rely on querying the ground-truth function, whereas offline MBO uses an existing database and presents different challenges.The cited online approaches include Bayesian optimization, direct search, evolutionary methods, cross-entropy, and simulated annealing.Source: §3 · p. 3
  • Prior offline MBO studies used different tasks and domain-specific protocols, limiting comparisons across methods.Design-Bench incorporates modified variants of existing datasets alongside its own tasks and standardizes evaluation.Source: §3 · p. 3
  • Recent offline MBO methods typically learn an objective model and sometimes a generative model of the design manifold for optimization.The benchmark overview describes tasks from different domains and both discrete and continuous design spaces.Source: §3 · p. 3, Table 1 · p. 4

4. Design-Bench Benchmark Tasks

Design-Bench pairs task datasets with ground-truth oracles for evaluation, while prohibiting oracle queries during training. Its tasks span robotics, neural architecture search, molecules, materials, DNA sequences, and controller weights.

4. Design-Bench Benchmark Tasks

  • Design-Bench tasks pair a dataset of designs and measurements with an oracle objective for evaluation, which algorithms must not query during training.The benchmark includes expert-built oracle models for tasks whose true values would otherwise require expensive physical experiments; this proxy evaluation may be less realistic if those functions are inaccurate.Source: §4 · p. 4
  • The Superconductor task optimizes chemical formulas for critical temperature using an 86-component real-valued design vector.Its dataset contains 21,263 superconductors, and its oracle’s held-out Spearman rank correlation is 0.9210.Source: §4 · p. 4
  • Ant and D’Kitty morphology tasks optimize 60 and 56 continuous limb-parameter values, respectively, using simulated robot performance as the objective.Evaluation averages 16 trials of 100 simulation time steps.Source: §4 · p. 5
  • NAS searches 64 discrete dimensions of kernel sizes and activation functions, while ChEMBL optimizes assay activity over short categorical molecule sequences.The NAS task uses five kernel sizes and five activation choices per layer; ChEMBL’s training set has 1,093 samples and sequences of length 31.Source: §4 · p. 5
  • Hopper Controller optimizes 5,126 continuous neural-network weights using return values, with no trajectory data in its dataset.The task evaluates designs through 1,000-step MuJoCo simulations.Source: §4 · p. 5

5. Task Properties, Challenges and Considerations

Design-Bench selects tasks to represent varied, realistic offline MBO challenges rather than directly solve real-world design problems. The suite highlights sparse high-dimensional design manifolds, sensitive objectives, and heavy-tailed data.

Diversity and realistically challenging

  • The tasks span discrete and continuous spaces and are intended to sample realistic challenges, not directly identify optimal real-world designs.Continuous spaces offer metric and ordering structure, while discrete spaces are finite and may have better dataset coverage.Source: §5 · p. 6

High-dimensional design spaces

  • High-dimensional tasks challenge methods to capture thin manifolds of valid, high-performing designs that uninformed random sampling is unlikely to hit.The ChEMBL-derived task has up to 31 dimensions and 591 categories per dimension; uniformly sampled designs generally score below the best dataset sample.Source: §5 · p. 6

Highly sensitive objective function

  • In DKitty and Ant Morphology, small changes in one design variable can sharply reduce objective values, making optimal designs difficult to produce.This reflects the broader challenge that nearby designs need not have similar objective values.Source: §5 · p. 6

Heavy-tailed data distributions

  • Heavy-tailed objective-value distributions can impair learning by generative models and learned objective models.The benchmark includes tasks exhibiting this distributional structure.Source: §5 · p. 6

6. Algorithm Implementations

The benchmark implements recent offline MBO methods and baselines that can use static data and high-dimensional inputs. The implementations include model-based optimization, generative approaches, and search over learned objectives.

6. Algorithm Implementations

  • The benchmark compares MINs, CbAS, autofocusing CbAS, REINFORCE/CMA-ES, and gradient ascent, selecting methods that handle offline data and high-dimensional inputs.The implementations are open sourced in design-baselines.Source: §6 · p. 6
  • Gradient ascent (Grad) optimizes a learned objective model for 200 steps, but can produce invalid designs because it does not constrain solutions to the valid-design manifold.Grad. Min and Grad. Mean use, respectively, the minimum or mean prediction from an ensemble of five learned objective models.Source: §6 · pp. 6–7
  • Figure 3 contrasts dataset objective values with uniform resampling, while Figure 4 plots DKitty performance against one varied leg-orientation dimension.The figures illustrate skew toward lower values under resampling and sensitivity to small changes in orientation.Source: Figure 3 · p. 7, Figure 4 · p. 7
  • CMA-ES adapts a Gaussian belief distribution toward high-scoring samples, while REINFORCE updates a design distribution using a policy-gradient estimator.REINFORCE is described as suited to non-smooth and highly stochastic objectives; it uses an ensemble of objective models filtered by a validation-loss threshold.Source: §6 · p. 7
  • CbAS adapts a learned density model toward optimized designs, using a VAE and a separately trained objective model to score generated samples.Autofocusing CbAS retrains the objective model under the current design distribution via importance sampling before feeding it into CbAS.Source: §6 · pp. 7–8

Model inversion networks (MINs) (Kumar & Levine,

  • MIN learns an inverse map from objective values to designs and samples from it conditioned on an optimized objective value.Offline BO-qEI instead fits a Gaussian Process to a learned objective, proposes candidates, and labels them using that model.Source: Model inversion networks (MINs) (Kumar… · p. 8
  • COMs penalize overestimation on off-manifold designs, then optimize the conservative learned objective from a design in the observed training set.The method uses gradient-ascent steps on the conservative model's predictions.Source: Model inversion networks (MINs) (Kumar… · p. 8

7. Benchmarking Prior Methods

The benchmark evaluates prior offline MBO algorithms under a shared protocol, with tuning kept independent of the ground-truth objective. Its results show that sophisticated methods can perform well, while classical baselines are competitive on several tasks.

Algorithm setup and hyperparameter tuning

  • Hyperparameter tuning must remain offline and agnostic to the ground-truth objective, preserving the benchmark’s evaluation boundary.Recommended tuning procedures are provided for the benchmarked methods and as guidance for future methods.Source: §7 · p. 8
  • CbAS tuning uses a β-VAE for stability, MINs benefits from objective normalization, and gradient ascent performs reasonably across tasks with normalized inputs and scores.For CbAS, β > 1 is especially important in high-dimensional spaces; gradient ascent also scales its learning rate by design dimension.Source: §7 · p. 8
  • The gradient-ascent baseline uses 200 steps from each dataset’s top-scoring 128 samples; discrete tasks optimize over design log-probabilities.The reported baseline is sensitive to input normalization and the number of optimization steps.Source: §7 · p. 8

Results

  • MINs, CbAS, and Autofocused CbAS solve varied tasks; CMA-ES and naïve gradient ascent each match complex methods on 4 of 8 tasks.The table reports 100th-percentile performance averaged over eight trials, with standard deviations.Source: §7 · p. 8, Table 2 · p. 9
  • The authors argue that gradient ascent could become a building block for offline MBO, while noting that benchmark results differ from prior reports because dataset generation was standardized.They caution that gradient ascent is not a full-fledged offline MBO method.Source: §7 · p. 9

8. Discussion and Conclusion

Design-Bench addresses the difficulty of tracking offline MBO progress by providing diverse tasks and a shared evaluation protocol. Its results motivate careful standardization and further work on evaluating produced solutions offline.

Discussion and Conclusion

  • Design-Bench covers diverse domains and continuous and discrete, low- and high-dimensional design spaces, and evaluates existing methods under identical assumptions.The authors note that even CMA-ES and naïve gradient ascent can be comparatively effective, underscoring the need for careful tuning and standardization.Source: §8 · p. 9
  • A key future direction is developing offline methods for model and hyperparameter selection, including offline evaluation of produced solutions.The authors hope the benchmark will become a standard metric for evaluating offline MBO algorithms.Source: §8 · p. 9
  • Design-Bench is intended to support insight into future offline MBO algorithm development through standardized evaluation.Source: §8 · p. 9

A. Data Collection

The benchmark combines task datasets from biology, chemistry, materials science, robotics, and machine learning with task-specific preprocessing and evaluation oracles. Its experimental details describe how training subsets and normalized scores are defined.

A.1. TF Bind 8 and TF Bind 10

  • TF Bind 8 uses all 65,792 measured length-8 DNA sequences for SIX6 REF R1, but exposes only the bottom 50% as offline training data.The task derives from a transcription-factor binding survey; the sequences represent every possible length-8 nucleotide combination.Source: §A.1 · p. 13

A.2. ChEMBL; A.3. Superconductor

  • ChEMBL optimizes MCHC for assay CHEMBL3885882 over tokenized SMILES, while the superconductivity task encodes chemical formulas as 86-dimensional element-count vectors.The ChEMBL dataset has 1,093 samples and a selected random-forest validation rank correlation of 0.7141; superconductors use 21,263 samples, with 17,010 in training.Source: §A.2 · p. 13, §A.3 · p. 13

A.4. Ant & D’Kitty Morphology

  • Ant and D’Kitty tasks map robot morphologies to returns from morphology-conditioned controllers, using CMA-ES-collected datasets of 25,009 samples per robot.The design spaces have 60 and 56 dimensions, respectively, and the top 40% of samples are removed for offline training.Source: §A.4 · pp. 13–14

A.5. NAS

  • NAS searches 64 discrete architecture dimensions with five categories each, using 1,771 of 2,440 randomly sampled CIFAR-10 designs as training data.The sampled architectures’ test accuracies range from 59.3% to 63.8%.Source: §A.5 · p. 14

A.6. Hopper Controller

  • Hopper Controller optimizes 5,126 neural-network weights using 3,200 policy snapshots from PPO training and scores each design with one environment rollout.The rollout and evaluation horizon is 1,000 simulation steps.Source: §A.6 · p. 14

B. Oracle Functions; B.1. TF Bind 8 and TF Bind 10; B.2. ChEMBL; B.3. Superconductor

  • The TF Bind oracle is a lookup table; ChEMBL and Superconductor use random-forest regression models trained on their full hidden datasets.The reported held-out rank correlations are 0.7141 for ChEMBL and 0.9155 for Superconductor.Source: §B.1 · p. 14, §B.2 · p. 14, §B.3 · p. 14

B.4. Ant & D’Kitty Morphology; B.5. NAS; B.6. Hopper Controller

  • Morphology, NAS, and Hopper use exact task oracles: simulator rollouts for robots and Hopper, and CIFAR-10 training and test accuracy for NAS.Morphology rollouts are limited to 100 steps; NAS architectures are trained for 20 epochs, while Hopper scores sum rewards over 1,000 steps.Source: §B.4 · pp. 14–15, §B.5 · p. 15, §B.6 · p. 15

C. Experimental Details; C.1. Objective Normalization; C.2. 50th Percentile Experiment Results

  • Normalized scores use the minimum and maximum objective values in the full unobserved dataset; the appendix also reports 50th-percentile results.Scores above one exceed the full dataset’s maximum, and the 50th-percentile appendix results use the dataset’s objective range.Source: §C.1 · p. 15, §C.2 · p. 15

C.3. Unnormalized Experimental Results

The appendix reports raw objective values for the benchmark experiments and summarizes their computational requirements. NAS is the exception to the otherwise modest resource demands.

C.3. Unnormalized Experimental Results

  • Raw 100th-percentile results are reported as mean objective values and standard deviations for the evaluated offline MBO methods.The results are presented in Table 4.Source: §C.3 · p. 15

C.4. Computation Resources

  • Experiments ran on one server with two Intel Xeon E5-2698 v4 CPUs and eight Nvidia Tesla V100 GPUs, and completed within 96 hours; NAS required comparatively more computation.Source: §C.4 · p. 15

D. Additional MBO Tasks That Were Discarded From Our Benchmark

The appendix describes additional candidate tasks and reports why they were excluded from the benchmark. It also details the GFP and UTR task data and oracle setups.

D. Additional MBO Tasks That Were Discarded From Our Benchmark

D.1. GFP

  • GFP uses a Transformer oracle trained on the full hidden dataset and a visible training set of 5,000 proteins from the 50th–60th fluorescence percentiles.The oracle’s Spearman rank correlation on a held-out validation set is 0.8497.Source: §D.1 · p. 16

D.2. UTR

  • UTR optimizes length-50 DNA sequences for expression, using a Transformer oracle and a visible dataset of 140,000 sequences.The 140,000 visible sequences are the lower half by ribosome load of the 280,000 selected sequences; the Transformer oracle has held-out Spearman rank correlation 0.6424.Source: §D.2 · pp. 16–17

D.3. Additional Experimental Results

  • Baseline performance was not distinguishable on GFP and UTR, and the authors considered these tasks unsuitable for benchmarking offline MBO methods.Two other candidate tasks were also excluded because results were indistinguishable across methods, suggesting they may not support better algorithm development.Source: D. Additional MBO Tasks That Were… · p. 16, §D.3 · p. 17

E. Normalization Of Inputs and Outputs Is Important for Gradient Ascent

The appendix reports that normalization improves gradient-ascent performance and outlines offline hyperparameter-selection strategies for benchmarked methods. These strategies use task data and validation signals to guide tuning.

E. Normalization Of Inputs and Outputs Is Important for Gradient Ascent

  • Gradient ascent performed 1.4x worse on Hopper Controller without normalization; the workflow normalizes designs and objective values to unit Gaussian statistics.Continuous designs are normalized directly, while discrete designs are first mapped to categorical logits; optimized designs are then de-normalized.Source: E. Normalization Of Inputs and Outputs… · p. 17

F. Hyperparameter Selection Workflow

  • Offline hyperparameter selection must use task data rather than performance feedback, with hyperparameters conditionally independent of the performance metric given the offline dataset.The appendix recommends validation-based tuning and presents general principles for methods with shared components.Source: F. Hyperparameter Selection Workflow · pp. 17–18

F.1. Strategy For Autofocused CbAS; F.2. Strategy For CbAS

  • Autofocused methods tune the learned objective and generative model using validation generalization signals; CbAS shares this approach, with task-specific β-VAE β controlling generalization.Autofocused tuning also tracks effective sample size of importance weights; CbAS objective models are not affected by those weights.Source: §F.1 · p. 18, §F.2 · p. 18

F.3. Strategy For MINs

  • MINs tune the learned objective with validation loss and assess whether their conditional generative model depends on the target objective value.The appendix proposes comparing conditional and unconditional predictions or tracking mutual information I(x; z).Source: §F.3 · pp. 18–19

F.4. Strategy For Gradient Ascent

  • Gradient-ascent methods tune the learned objective and optimizer; a learning rate should keep steps increasing predicted objective, while choosing step count offline remains future work.The reported workflow uses a fixed number of steps.Source: §F.4 · p. 19

F.5. Strategy For REINFORCE

  • REINFORCE methods tune the learned objective and policy-gradient estimator, using Gaussian policies for continuous tasks and categorical policies for discrete tasks.The learning rate and optimizer should make policy updates improve the model-predicted score.Source: §F.5 · p. 19

F.6. Strategy For Bayesian Optimization; F.7. Strategy For Covariance Matrix Adaptation (CMA-ES)

  • Bayesian optimization uses a Gaussian Process with quasi-Monte Carlo Expected Improvement; CMA-ES uses its open-source implementation’s default settings with σ = 0.5.The reported acquisition function is selected for scaling to high-dimensional optimization problems.Source: §F.6 · p. 19, §F.7 · p. 19

F.8. Strategy For Conservative Objective Models (COMs)

  • COM tuning balances off-manifold overestimation against fitting the dataset, with the threshold or penalty set by design-space type and gradient ascent run for 50 steps.The overestimation parameter is 2 for discrete tasks and 0.5 for continuous tasks.Source: §F.8 · p. 19

Related papers

Loading 2202.08450v1…