Source-linked AI summary
A Tutorial on Bayesian Optimization
Peter I. Frazier
TL;DR
Bayesian optimization addresses expensive objective evaluations in low-dimensional continuous domains where the objective is black-box and derivative-free. The tutorial explains a Bayesian surrogate-and-acquisition framework, develops Gaussian-process and acquisition-function methods, and extends them to noisy and exotic settings. It highlights a decision-theoretically justified treatment of expected improvement for noisy measurements while identifying limited theory and deployability for multi-step methods.
Problem
Bayesian optimization targets expensive, black-box, derivative-free global optimization problems with limited evaluations, often in low-dimensional domains.
Method
The tutorial develops Gaussian-process Bayesian modeling, acquisition functions, and extensions for noisy, parallel, constrained, multi-fidelity, multi-source, environmental, multi-task, and derivative-observation settings.
Results
The tutorial provides a novel analysis of expected improvement for noisy measurements and argues that Scott et al.'s acquisition function is the most natural application in that setting.
Takeaways & Limitations
Bayesian optimization offers a versatile framework for expensive black-box optimization, with potential impact in chemistry, chemical engineering, materials design, and drug discovery.
Takeaways & Limitations
Finite-time bounds for practical acquisition functions are unavailable, and approximate multi-step methods are not yet broadly deployable because their error and extra cost can overwhelm their benefits.
Abstract
from arXiv · showhide
Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient. We then discuss more advanced techniques, including running multiple function evaluations in parallel, multi-fidelity and multi-information source optimization, expensive-to-evaluate constraints, random environmental conditions, multi-task Bayesian optimization, and the inclusion of derivative information. We conclude with a discussion of Bayesian optimization software and future research directions in the field. Within our tutorial material we provide a generalization of expected improvement to noisy evaluations, beyond the noise-free setting where it is more commonly applied. This generalization is justified by a formal decision-theoretic argument, standing in contrast to previous ad hoc modifications.
1 Introduction
Bayesian optimization targets expensive, black-box, derivative-free global optimization under limited evaluations, typically in low-dimensional continuous domains. This tutorial presents its Bayesian surrogate-and-acquisition framework, applications, extensions, and analysis of noisy expected improvement.
- Problem setting: The typical input dimension is low, with d ≤20 in most successful applications, and feasible sets are usually simple hyper-rectangles or simplices.
- Problem setting: Evaluations are expensive because they may take hours or incur monetary or opportunity costs, limiting experiments typically to a few hundred.
- Problem setting: Bayesian optimization is designed for black-box derivative-free global optimization.The objective is continuous, lacks exploitable special structure, and is observed without derivatives; the focus is global rather than local optimization.
- Applications: Bayesian optimization has been applied to machine-learning hyperparameter tuning, engineering design, laboratory experiments, materials, and drug design.
- Distinction: Bayesian optimization distinguishes itself from surrogate methods by using Bayesian statistical models and Bayesian interpretations to select evaluations.
- Tutorial scope: The tutorial emphasizes exotic Bayesian optimization problems, acquisition functions, and a decision-theoretic treatment of expected improvement for noisy measurements.
2 Overview of BayesOpt
Bayesian optimization iteratively combines a probabilistic model of the objective with an acquisition function to allocate a limited evaluation budget. Gaussian-process posteriors quantify uncertainty, while acquisition values guide the next sample and final solution selection.
- Core components: BayesOpt combines a Bayesian statistical model of the objective with an acquisition function that selects where to sample next.
- Iterative procedure: After an initial space-filling design, the algorithm repeatedly updates the posterior, maximizes the acquisition function, evaluates the objective, and increments the budget.
- Gaussian-process model: Gaussian-process regression provides a posterior distribution for potential objective values at candidate points and updates it after each new observation.
- Gaussian-process model: GP regression represents uncertainty with a posterior mean and variance, including a 95% Bayesian credible interval around the mean.
- Acquisition function: Expected improvement is larger in regions with greater credible intervals and equals zero at previously evaluated points when evaluations are noise-free.
- Extensions: The tutorial extends the basic framework to measurement noise, parallel evaluations, constraints, multi-fidelity settings, multiple information sources, and other exotic problems.
3 Gaussian Process (GP) Regression
GP regression models function values with a Gaussian-process prior and updates the resulting posterior after observations. Its mean and kernel encode prior beliefs, while hyperparameter choices determine how those beliefs are fitted or integrated.
- Gaussian-process model: GP regression places a multivariate normal prior over function values at selected points, defined by a mean vector and covariance matrix.The covariance matrix comes from a kernel evaluated at pairs of input points.
- Choosing a kernel: The kernel assigns stronger positive correlation to nearby inputs, encoding the belief that nearby points have more similar function values.Kernels must also be positive semi-definite so they produce valid covariance matrices.
- Posterior inference: Conditioning on observed function values produces a posterior mean and variance for unevaluated points.The posterior mean combines prior and data-based estimates, while the posterior variance reflects uncertainty removed by observations.
- Choosing a kernel: Power exponential and Matérn kernels provide alternative covariance structures, with power-exponential parameters controlling beliefs about how quickly the function changes.Different α1 values generate different prior function behaviors in the one-dimensional illustration.
- Choosing hyperparameters: Hyperparameters can be selected by maximum likelihood, maximum a posteriori estimation, or fully Bayesian marginalization.MAP estimation incorporates a prior over hyperparameters, whereas fully Bayesian inference integrates over their posterior distribution.
- Choosing hyperparameters: Fully Bayesian hyperparameter integration is typically intractable and therefore can be approximated by sampling with MCMC.MAP inference corresponds to approximating the hyperparameter posterior by a point mass at its maximizer.
4 Acquisition Functions
Acquisition functions use the Gaussian-process posterior to choose informative evaluations, balancing expected improvement against uncertainty and extending this logic to more complex settings. Expected improvement is simple and effective, while knowledge gradient and entropy-based methods address cases where sampling benefits the solution indirectly, though multi-step and entropy-based optimization remain computationally difficult.
- Expected Improvement: Expected improvement chooses the next evaluation by maximizing the posterior expected increase over the best previously observed value.The improvement is the positive part of f(x) − f*_n, averaged under the posterior distribution.
- Expected Improvement: The expected improvement algorithm evaluates the point with the largest EI and can optimize this inexpensive acquisition function using first- or second-order methods.This contrasts with optimizing the original objective, which is expensive and derivative-free.
- Expected Improvement: EI increases with both expected quality relative to the previous best and posterior uncertainty, creating an exploration–exploitation tradeoff.Equal-EI contours trade off high expected quality against uncertainty, while uncertainty is valuable because it can reveal better points in poorly observed regions.
- Computational Considerations: Optimizing KG and entropy search can be computationally challenging, especially in higher dimensions or when entropy and stochastic objectives require difficult numerical optimization.KG discretization is burdensome in higher dimensions, while entropy search lacks closed-form maximizer entropy and known stochastic-gradient optimization.
- Alternative Acquisition Functions: Knowledge gradient and entropy-search methods are useful when sampling improves the inferred solution rather than the function value at the sampled point.This includes noise, multi-fidelity, derivative, and environmental-condition settings; KG can substantially outperform EI when such effects are first-order.
- Multi-Step Optimal Acquisition Functions: 98%: KG is within 98% of the multi-step optimal algorithm in the cited Bayesian feasibility problems, although approximate multi-step methods are not yet broadly practical.Approximation error and extra computational cost can overwhelm the benefit of considering multiple steps.
5 Exotic Bayesian Optimization
The tutorial extends Bayesian optimization beyond its standard noise-free setting to noisy evaluations, parallel sampling, constraints, and multiple fidelities or information sources. It also discusses acquisition-function adaptations for these settings, including a decision-theoretically motivated noisy-EI generalization.
- Noisy Evaluations: Knowledge gradient, entropy search, and predictive entropy search retain their one-step optimality properties under noisy observations.The noisy posterior mean is used in these acquisition functions.
- Noisy Evaluations: Noisy evaluations require modifying Gaussian-process inference and acquisition functions because the observed value and improvement are no longer directly defined.KG, ES, and PES apply directly with a noise-inclusive posterior, while EI requires additional choices.
- Noisy Evaluations: The tutorial argues that the Scott et al. noisy-EI acquisition function is the most natural generalization of EI’s assumptions to noisy measurements.Its derivation restricts the reported final solution to a previously evaluated point and selects the one-step optimal sampling location under that restriction.
- Parallel Evaluations: Parallel EI jointly selects multiple evaluation points, while asynchronous versions hold currently evaluated points fixed and optimize over available resources.Constant Liar sequentially chooses points by assigning assumed constant values to earlier pending evaluations to speed computation.
- Constraints: In constrained optimization, improvement requires both feasibility and a better objective value than the best previously evaluated feasible point.This extension is described for noise-free evaluations when the objective and constraints are equally expensive to evaluate.
- Multi-Fidelity and Multi-Information Source Evaluations: Multi-fidelity optimization trades evaluation cost against accuracy, while multi-information-source optimization allows sources with different biases, noise levels, and costs.The overarching multi-fidelity goal is to optimize the highest-fidelity objective under a total cost budget.
- Multi-Fidelity and Multi-Information Source Evaluations: Direct EI fails for lower-fidelity sources because they never directly improve the best observed objective value, so separate source-selection procedures or other acquisition functions are needed.KG, ES, and PES can be applied directly to the more general multi-information-source setting.
6 Software
The tutorial surveys software for Gaussian-process regression and Bayesian optimization, covering packages that combine both capabilities as well as standalone libraries. The listed implementations span R, Python, C++, MATLAB, TensorFlow, and PyTorch ecosystems.
- Software: Bayesian optimization software includes integrated packages and standalone libraries for either Gaussian-process regression or Bayesian optimization.The tutorial lists prominent packages and their URLs.
- R Packages: DiceKriging and DiceOptim provide Gaussian-process regression and Bayesian optimization in R.They are available through CRAN.
- Python Packages: GPyOpt is a Python Bayesian optimization library built on the GPy Gaussian-process regression library.Both are maintained by the machine learning group at Sheffield University.
- C++ and Python Packages: MOE is a C++ Bayesian optimization library with a Python wrapper supporting GPU-based computations for improved speed.Cornell MOE adds support for parallel and derivative-enabled knowledge-gradient algorithms.
- Other Libraries: Spearmint is a Python Bayesian optimization library, while DACE is a MATLAB Gaussian-process regression library that remains widely used despite its last update in 2002.DACE is described as a library for design and analysis of computer experiments.
- Other Libraries: GPFlow and GPyTorch are Python Gaussian-process regression libraries built on TensorFlow and PyTorch, respectively, while laGP supports regression, Bayesian optimization, and inequality constraints in R.The packages target different computational ecosystems and capabilities.
7 Conclusion and Research Directions
The tutorial concludes that Bayesian optimization has broad methodological and application opportunities, alongside important gaps in theory, modeling, high-dimensional optimization, and exotic problem structures. It highlights chemistry, chemical engineering, materials design, and drug discovery as fields where repeated costly experiments may provide substantial impact.
- Conclusion: The tutorial identifies noisy measurements, parallel evaluations, constraints, multiple fidelities and information sources, environmental conditions, multitask optimization, and derivative observations as advanced problem settings.These settings extend the standard Bayesian optimization framework.
- Theoretical Research Directions: Finite-time bounds and practical benefits of multi-step optimal algorithms remain poorly understood, and convergence rates are known only in limited settings.For expected improvement, it remains unknown whether removing periodic uniform sampling changes the convergence rate.
- Statistical Research Directions: Bayesian optimization research could benefit from statistical models beyond Gaussian processes that better represent particular classes of objectives or applications.The tutorial calls for both broadly useful and application-specific models.
- High-Dimensional Optimization: Developing methods for high-dimensional objectives is a major practical and theoretical research direction, including methods that identify and exploit structure in such objectives.The tutorial also suggests that new acquisition functions may provide substantial value.
- Exotic Problem Structure: Methods that exploit exotic problem structure may be especially fruitful when combined with real-world applications, where unanticipated difficulties can emerge.The tutorial connects methodological development with practical deployment.
- Applications: Chemistry, chemical engineering, materials design, and drug discovery are highlighted as application areas with potential impact because they involve repeated physical experiments requiring substantial time and money.The tutorial notes early work in these areas.