BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization

• BoTorch introduces a modular and scalable Monte Carlo framework for Bayesian optimization that leverages modern computational paradigms and novel convergence results.Source: §1 · p. 1
• Its optimization approach combines sample average approximation, deterministic higher-order optimization, and variance reduction for MC acquisition functions.Source: §1 · pp. 1–2
• BoTorch provides the first convergence results for MC-acquisition-function SAA, including general results for randomized quasi-Monte Carlo.Source: §1 · p. 2
• The framework introduces an SAA-based one-shot Knowledge Gradient formulation with improved performance over the state of the art.Source: §1 · p. 2
• Composable model-agnostic abstractions use auto-differentiation and scalable parallel computation on CPUs and GPUs.Source: §1 · p. 2

• Bayesian optimization maximizes an expensive function using a probabilistic surrogate model and an acquisition function balancing exploration and exploitation.Source: §2 · p. 2
• BoTorch differs from existing libraries through modularity, gradient-based optimization, hardware acceleration, and natural support for Monte-Carlo acquisition functions.Source: §2 · p. 2
• The paper applies sample average approximation to Monte-Carlo acquisition optimization, combining sampling, deterministic optimization, and variance reduction with new theoretical analysis and systematic implementation.Source: §2 · p. 2

• The framework permits multi-output, noisy observations and arbitrary posterior forms rather than restricting the surrogate to Gaussian processes or Gaussian noise.Source: §3 · p. 2
• Many acquisition functions are represented as expectations of a utility applied to a composite objective, with parameters independent of the candidate set.Source: §3 · p. 2
• Posterior samples are evaluated in parallel and averaged, while all operations remain fully differentiable for acquisition optimization.Source: Figure 1 · p. 3
• Expected Improvement is an analytic special case for a single-output, single-candidate identity objective, maximizing the expected improvement over the current best noiseless value [46].Source: §3 · pp. 2–3
• Monte-Carlo integration approximates acquisition expectations when arbitrary objectives, utilities, non-Gaussian posteriors, or parallel and asynchronous candidate sets prevent analytic expressions.Source: §3 · p. 3
• Randomized quasi-Monte Carlo reduces variance in both the acquisition estimate and its gradient relative to straightforward independent sampling.Source: §3 · p. 3

4.1 Optimizing General MC Acquisition Functions

• Fixed base samples make the Monte Carlo acquisition estimate deterministic, enabling auto-differentiation and deterministic optimization instead of re-sampled stochastic optimization.The gradient is computed as an average of sample-level gradients, regardless of whether that average is an unbiased or consistent estimator of the true gradient.Source: §4.1 · p. 3
• Under stated compactness, smoothness, and Lipschitz conditions, SAA optimizers converge almost surely to true acquisition optimizers and have exponential convergence in probability under additional regularity conditions.The results are derived for GP surrogates with i.i.d. base samples in the simplified presentation, with extensions to RQMC in the appendix.Source: §4.1 · p. 4
• RQMC extends the convergence analysis beyond i.i.d. samples and shows remarkably strong empirical behavior, although theoretical improvements over MC are not characterized.Figures 2 and 3 compare MC and RQMC acquisition functions and optimizer convergence under fixed-sample SAA and re-sampled stochastic optimization.Source: §4.1 · p. 4, Figure 2 · p. 4, Figure 3 · p. 4
• SAA permits quasi-Newton multi-start optimization with L-BFGS-B, often improving over stochastic approaches while requiring less learning-rate tuning.The authors report that SAA bias has only a minor effect relative to analytic ground truth in practice.Source: §4.1 · p. 4

4.2 One-Shot Formulation of the Knowledge Gradient using SAA

• The Knowledge Gradient addresses future observation effects but is traditionally computationally expensive because maximizing it requires nested optimization.The paper focuses on KG while noting that the approach extends to other look-ahead acquisitions such as two-step EI.Source: §4.2 · pp. 4–5
• The one-shot formulation has convergence results, applies with RQMC, and is asymptotically optimal under a matched GP-prior setting with affine identity utility.Theorem 2 gives almost-sure convergence and exponential convergence in probability under its stated assumptions.Source: §4.2 · p. 5
• The one-shot Knowledge Gradient jointly optimizes the current candidate and fantasy points, moving maximization outside the sample average so deterministic optimization can solve one higher-dimensional problem.For affine utility functions, the expectation has an analytical expression; otherwise, an additional fixed-sample MC approximation is used.Source: §4.2 · p. 5
• The main practical limitation of one-shot Knowledge Gradient is that optimization dimension grows linearly with the number of base samples, though moderate sample counts perform well empirically.Auto-differentiation through posterior updates enables joint optimization over the candidate and fantasy points.Source: §4.2 · p. 5

5 Programmable Bayesian Optimization with BOTORCH

• BOTORCH implements Monte Carlo acquisition functions with sample average approximation, enabling deterministic gradient-based optimization and variance reduction.Its operations are PyTorch modules that support automatic differentiation and parallel execution.Source: §5 · pp. 5–6

5.1 Composing BOTORCH Modules for Multi-Objective Optimization

• BOTORCH composes models, objectives, acquisition functions, and optimization utilities so complex acquisition functions can be implemented and optimized through common APIs.Models may be any PyTorch module conforming to the BOTORCH Model API, while optimize_acqf() selects candidates by maximizing acquisition utility.Source: §5 · p. 6, §5.1 · p. 6
• qParEGO uses a GenericMCObjective for augmented Chebyshev scalarization and qExpectedImprovement for parallel multi-objective candidate selection.MCObjective transforms posterior samples, and auto-differentiation supports gradient-based optimization of the resulting acquisition function.Source: §5.1 · p. 6

5.2 Implementing Parallel, Asynchronous Noisy Expected Improvement

• BOTORCH’s full Monte Carlo noisy expected improvement averages improvements over joint posterior samples from candidate, observed, and pending points.qNEI supports asynchronous optimization by concatenating pending points and optimizing only the new candidates; sequential greedy generation can exploit acquisition-function submodularity.Source: §5.2 · pp. 6–7

5.3 Look-ahead Bayesian Optimization with One-Shot KG

• One-shot Knowledge Gradient concatenates candidate and fantasy points, then uses fantasized models to represent future observations for one-step look-ahead optimization.Automatic differentiation propagates through fantasy-model generation and posterior evaluation, allowing the formulation to use the same optimization APIs as other acquisition functions.Source: §5.3 · p. 7

6.1 Exploiting Parallelism and Hardware Acceleration

• 10–40X speedups arise from fast predictive covariance estimates over standard posterior inference, while GPU execution scales nearly linearly with batch size except at very large batches and sample counts.Batch evaluation and fast predictive distributions enable acquisition evaluation over tens of thousands of points.Source: §6.1 · p. 8

6.2 Bayesian Optimization Performance Comparisons

• GPFlowOpt was excluded because it was incompatible with current versions of GPFlow and TensorFlow.This limits the set of external library comparisons reported in the experiments.Source: §6.2 · p. 8
• BOTORCH’s algorithms tend to achieve greater sample efficiency than other packages using their default models and settings.The comparison includes GPyOpt, Cornell MOE, and Dragonfly.Source: §6.2 · pp. 8–9
• OKG often outperforms other acquisition functions and reduces wall time by up to 6X versus MOE KG while improving optimization performance.MOE KG is identified as the gold-standard implementation of Knowledge Gradient; Figure 6 reports the corresponding Hartmann comparison.Source: Figure 6 · p. 8, §6.2 · p. 9
• The evaluation covers noisy parallel optimization on four synthetic functions and three real-world hyperparameter-optimization tasks.The synthetic experiments use q = 4 and 100 trials; the HPO tasks include DQN Cartpole, UCI Adult, and SWA on VGG-16 for CIFAR-10.Source: §6.2 · p. 9

7 Discussion and Outlook

• BoTorch’s SAA-based OKG improves scalability and optimization performance while supporting generic composite objectives and outcome constraints.The approach can also extend naturally to multi-step and other look-ahead methods.Source: §7 · p. 9
• The framework provides modular abstractions, flexible APIs, auto-differentiation, parallel computation, and support for batch, asynchronous, RQMC, composite-objective, and constrained optimization.These abstractions are intended to help researchers rapidly assemble novel Bayesian optimization techniques.Source: §7 · p. 9
• Empirical results show significantly faster and more accurate closed-loop optimization across a range of standard problems.The paper states that high-dimensional, multi-fidelity, and multi-objective BO are outside its scope, although the software includes or can support them.Source: §7 · p. 9
• BoTorch offers a robust, extensible basis for new algorithms, theoretically backed by novel SAA convergence results.The methodological and theoretical contributions are distributed through the open-source library.Source: §7 · p. 9

Broader Impact

• Bayesian optimization is domain-general and has been applied in areas including drug discovery, energy-efficient ML tuning, malaria-survey design, and materials research.The broader-impact discussion also notes that general-purpose BO tools could be misused.Source: Broader Impact · p. 10

A Brief Overview of Other Software Packages for BO

• Existing BO libraries differ in feature coverage and extensibility, with some packages supporting parallel, constrained, multi-fidelity, or high-dimensional optimization but remaining harder to modify.The comparison includes Spearmint, GPyOpt, Cornell-MOE, RoBO, Emukit, and Dragonfly.Source: A Brief Overview of Other Software… · p. 16

B.1 Batch Evaluation

• GPU batch evaluation scales essentially linearly with batch size except at very large b and N, while CPU evaluation is faster for small batches and sample sizes.Batching also makes larger MC sample sizes have little impact on wall time.Source: §B.1 · pp. 16–17

C.1 Synthetic Functions

• Under noise generated from a N(0, .25) distribution, BOTORCH’s NEI and OKG acquisition functions provide highly competitive performance across all synthetic functions reported in Figures 9–11.Figures 9–11 present results for all synthetic functions from Section 6.Source: §C.1 · p. 17

C.2 One-Shot KG Computational Scaling

• OKG is significantly faster than MOE KG while achieving much better optimization performance on the reported one-shot KG experiments.Wall-time comparisons cover Cholesky and Linear CG inference on CPU and GPU as data size grows.Source: §C.2 · p. 17, Figure 12 · p. 18, Figure 13 · p. 18

C.3 Constrained Bayesian Optimization

• BOTORCH evaluates constrained BO with feasibility-weighted EI, NEI, and OKG objectives under noisy multi-output synthetic functions.The constrained KG outcome is presented as a contribution not previously shown in the literature, but its results are not directly comparable to because conventions differ.Source: §C.3 · p. 18
• The constrained Hartmann6 experiments report BOTORCH algorithms because the other packages do not natively support optimization under unknown constraints.Regret uses a feasibility-weighted objective, assigning infeasible points an objective value of zero.Source: §C.3 · p. 18

C.4 Hyperparameter Optimization Details

• BOTORCH OKG, EI, and NEI learn faster early in DQN Cartpole tuning, although all algorithms reach reasonable performance in later iterations.The comparison uses 15 rounds of q = 4 parallel evaluations over 100 trials.Source: §C.4 · p. 19, Figure 16 · p. 19
• All BOTORCH algorithms achieve significantly better classification accuracy than the other algorithms on the UCI Adult neural-network surrogate benchmark.OKG performs slightly better in earlier iterations, with results averaged over 200 trials.Source: §C.4 · p. 19, Figure 17 · p. 19
• 93.84 ± 0.03 average accuracy is obtained for SWA hyperparameter tuning on CIFAR-10, compared with the reported 93.64 ± 0.20 reference interval.The experiment tunes learning rate, update frequency, and starting iteration using OKG.Source: §C.4 · p. 19

D.1 General SAA Results

• Under compactness, smooth GP priors, and Lipschitz conditions, SAA optimizers converge almost surely to true acquisition-function optimizers, with exponential convergence in probability under additional moment conditions.The formal theorem and its supporting propositions impose the stated regularity and moment assumptions.Source: §D.2 · p. 20, §D.1 · p. 20

D.3 Randomized Quasi-Monte Carlo Sampling for Sample Average Approximation

• The convergence results extend to randomized quasi-Monte Carlo samples generated from suitably randomized (t, d)-sequences.The paper uses scrambled Sobol sequences and reports improved empirical convergence rates, while not characterizing a theoretical RQMC-over-MC rate improvement.Source: §D.3 · p. 20
• The RQMC extension does not provide a convergence rate for the optimizer, which the authors leave for future work.The stated theorem preserves convergence conclusions but omits this rate characterization.Source: §D.3 · p. 20

D.4 Asymptotic Optimality of OKG

• OKG is asymptotically optimal when the number of fantasies grows asymptotically with iteration, with explicit sufficient sample-size conditions in finite one-point design spaces.The result assumes the relevant conditions of the preceding theorems and an analytic inner expectation, or corresponding inner MC growth.Source: §D.4 · p. 21

Generalization

• The analysis extends to multi-output Gaussian processes by treating outputs jointly and requiring differentiability of each cross-output covariance function.The resulting posterior mean and covariance objects are handled in higher-dimensional stacked representations.Source: §D.5 · p. 21

MC convergence

• Theorem 1 uses Lipschitzness, integrability, and the strong law of large numbers to establish almost-sure convergence of MC acquisition approximations and their optimized values.The argument relies on continuously differentiable GP posterior quantities and Lipschitz objective and utility functions.Source: §D.5 · p. 22

RQMC convergence

• Randomized quasi-Monte Carlo approximations converge almost surely under square-integrability, using nested uniform scrambling and the Owen–Rudolf convergence result.The proof maps scrambled uniform samples through the Gaussian inverse CDF and establishes finite second moments.Source: §D.5 · pp. 22, 24

Knowledge Gradient

• For the one-shot Knowledge Gradient, Gaussian-process structure and affine objectives yield finite moments, enabling almost-sure convergence of MC and RQMC approximations.The proof represents the conditional expected objective as a Gaussian process and applies strong-law or randomized-QMC arguments.Source: §D.5 · pp. 22–24

Limitations

• The proofs assume a fixed heteroskedastic noise function; if σ^2(x) changes with observations, the argument no longer holds and that case is excluded.This scope boundary is stated explicitly for observation-dependent noise models.Source: §D.5 · p. 23

Knowledge Gradient

• The Knowledge Gradient convergence results support asymptotic optimality of the MC-approximated policy when the true function is drawn from the same GP prior and g(f) ≡ f.The asymptotic argument shows the limiting exact Knowledge Gradient vanishes across the feasible space under the stated policy.Source: §D.5 · p. 26

SAA and QMC

• Fixed base samples make the MC approximation deterministic and differentiable but introduce consistent conditional bias, whose variance is smaller for QMC than MC.Across base-sample realizations, QMC substantially reduces estimate variance relative to MC.Source: E Illustration of Sample Average… · pp. 26–27, Figure 18 · p. 27

Optimization

• The optimizer’s bias is typically small, and deterministic optimization of the QMC approximation is excellent even at relatively small sample sizes and generally better than stochastic optimization.Figure 19 compares relative objective gaps and distances to the analytic EI optimizer across 250 runs.Source: E Illustration of Sample Average… · p. 27, Figure 19 · p. 27

SAA and QMC

• QMC requires only 64 samples to match the error of MC with 4096 samples for bias, variance, and distance to the true optimizer.The comparison concerns the setting evaluated in Figure 20 and its empirical convergence-rate analysis.Source: E Illustration of Sample Average… · p. 27, Figure 20 · p. 28, Table 1 · p. 28

Scope and caveat

• Improved acquisition optimization does not necessarily improve closed-loop BO because the effect depends on the acquisition function and underlying problem.For noisy problems, imperfect EI optimization can add exploration and improve closed-loop performance.Source: E Illustration of Sample Average… · p. 28

Optimization

• Deterministic optimization is essentially hyperparameter-free, whereas stochastic methods depend materially on optimizer choice and learning rate.The rank ordering of SGD and ADAM changes with learning rate, making optimizer tuning non-trivial.Source: E Illustration of Sample Average… · p. 28

F.1 Batch Initialization for Multi-Start Optimization

• BoTorch uses Sobol candidate sets, batch acquisition evaluation, and softmax sampling to generate multi-start initial conditions for non-convex acquisition optimization.The sampling temperature η controls how strongly initialization favors high acquisition values.Source: §F.1 · p. 29
• As η approaches zero initialization becomes Sobol sampling, whereas large η becomes greedy and can cluster starts, undermining multi-start coverage.Large batch evaluations make tens of thousands of initialization candidates feasible for moderately sized models.Source: §F.1 · p. 29

F.2 Sequential Greedy Batch Optimization

• Sequential greedy batch optimization selects candidates one at a time while conditioning on pending points and integrating over their uncertain outcomes.The full MC formulation avoids constructing a separate fantasy model for each sampled outcome.Source: §F.2 · p. 29
• Sequential greedy optimization can outperform joint optimization because it decomposes a large difficult problem into a sequence of smaller optimization problems.The approach is also theoretically justified for acquisition functions with submodularity properties.Source: §F.2 · p. 29

Active Learning Example

• NIPV is implemented by integrating posterior variance over configurable MC points, allowing non-uniform weighting and supporting posterior-variance-reduction sampling.The acquisition function supports parallel and asynchronous point selection.Source: G Active Learning Example · p. 29
• The NIPV implementation benefits from fast predictive variance computations because MC integration evaluates posterior variance at many points.The implementation uses standard BoTorch components and can be applied with scalable probabilistic models.Source: G Active Learning Example · pp. 29–30
• The active-learning task selects geographic locations around Nigeria to minimize 2017 prediction error across the country using data from 2011–2016.The model is fit to a spatio-temporal malaria dataset at 4.5km2 spatial resolution.Source: G Active Learning Example · p. 30
• NIPV-selected 2017 locations reduce MSE by 16.7% on average, versus 12.4% for a regularly spaced grid.The results average across 60 subsampled datasets, with SEMs of 0.96% and 0.99%, respectively.Source: G Active Learning Example · p. 30, Figure 21 · p. 30

Comparing Implementation Complexity

• <30 LoC implements a more efficient Knowledge Gradient, while composite-function support takes 7 LoC and Noisy EI takes 14 LoC.The comparison uses Lines of Code as an approximate indicator of implementation effort, despite limitations across design philosophies and languages.Source: Comparing Implementation Complexity · p. 31

H.1 Composite Objectives

• KG-CF reaches lower regret significantly faster at q = 1 than q = 3, while EI-CF performs similarly across both settings.The results average log regret at the posterior-mean maximizer over 250 trials; the setting extends composite objectives to Knowledge Gradient and parallel BO.Source: §H.1 · p. 31

H.2 Generalized UCB

• Generalized parallel UCB supports pending candidates, generic objectives, and QMC sampling, with default QMC sampling and the identity objective when unspecified.These defaults are exposed through the code example’s optional sampler and objective arguments.Source: §H.2 · p. 31

H.3 Full Code Examples

• The full examples implement parallel Noisy EI, one-shot Knowledge Gradient, and negative Integrated Posterior Variance using BoTorch’s model, sampler, objective, and pending-point abstractions.The examples cover acquisition evaluation through posterior sampling, fantasized models, objectives, and variance calculations.Source: §H.3 · pp. 31–34