Source-linked AI summary
BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
Ruben Martinez-Cantin
TL;DR
Bayesian optimization requires sample-efficient methods for nonlinear optimization, bandits, and sequential experimental design. BayesOpt provides a flexible, efficient library with multiple models, algorithms, optimization domains, and language interfaces for these tasks.
Problem
Bayesian optimization seeks sample-efficient nonlinear optimization by updating surrogate-function posteriors from observations to guide subsequent exploration.
Method
BayesOpt implements Bayesian optimization in C++ with selectable surrogate processes, posteriors, kernels, acquisition components, and optimization algorithms through common interfaces.
Results
The library supports continuous, discrete, categorical, and high-dimensional optimization across C, C++, Python, Matlab, and Octave interfaces.
Takeaways & Limitations
BayesOpt offers a portable framework for combining Bayesian optimization components across problem types and programming environments.
Abstract
from arXiv · showhide
BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by building a posterior distribution to capture the evidence and prior knowledge for the target function. Built in standard C++, the library is extremely efficient while being portable and flexible. It includes a common interface for C, C++, Python, Matlab and Octave.
1 Introduction
Bayesian optimization selects the next point by analyzing a distribution over functions and an acquisition function, while retaining all observations in the surrogate model’s posterior.
- Bayesian optimization: Bayesian optimization chooses the next point using an acquisition function applied to a distribution over functions, such as a Gaussian process or another surrogate model.The distribution may represent prior knowledge and is updated using observed data.
- Bayesian optimization: Its posterior surrogate model P(f|D) encodes a memory of all observations.This posterior provides the basis for subsequent exploration decisions.
2 BayesOpt library
BayesOpt is a portable C++ Bayesian optimization library with flexible surrogate models, posterior inference, and runtime-configurable components. It emphasizes efficiency, correctness, parallel safety, and common interfaces across C, Python, Matlab, and Octave.
- Surrogate models: BayesOpt models f(x) as φ(x)T w plus nonparametric perturbation ǫ(x), supporting Gaussian, Student-t, and Mixture of Gaussians processes.The formulation can represent linear regression with heteroscedastic perturbation, a nonparametric process with nonzero mean, or a semiparametric model.
- Posterior inference: Closed-form marginal posteriors are available except for kernel parameters θ, which BayesOpt can estimate using empirical Bayes or MCMC.The library also allows hyperpriors on model parameters.
- Efficiency: Efficiency relies on derivative-free DIRECT and BOBYQA methods for empirical-Bayes kernel optimization, while avoiding unnecessary kernel-parameter updates.The authors report that combining global and local derivative-free methods marginally outperforms gradient-based optimization in CPU time by avoiding marginal-likelihood derivative overhead.
- Architecture: A factorylike design lets users select and combine optimization components at runtime, including sampling methods and extensible kernels.New components can be created by inheriting abstract or existing component classes and are then automatically integrated into the library.
- Correctness: BayesOpt is thread- and exception-safe for parallelized calls, uses numerically stable GP-Hedge variations, and relies on NLOPT for inner optimization loops.Inner loops include optimizing criteria and learning kernel parameters.
- Portability and API: The C++ core has been tested across Linux, Mac OS, and Windows and provides interfaces for C, Python, and Matlab/Octave through a common API.The API supports both function-template and object-oriented usage, with nonlinear constraints available through checkReachability in C++ and Python.