PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial

Zhongwei Yu, Rasul Tutunov, Alexandre Max Maraval, Zikai Xie, Zhenzhi Tan, Jiankang Wang, Bin Cao, Zijing Li, Liangliang Xu, Qi Yang, Jun Jiang, Sanzhong Luo, Zhenxiao Guo, Tongyi Zhang, Haitham Bou-Ammar, Jun Wang

arXiv:2604.01328v3cs.LG

TL;DR

Scientific discovery often involves costly evaluations, unknown mechanisms, and enormous constrained search spaces. This tutorial frames discovery as Bayesian optimisation, combining surrogate models with acquisition functions and iterative experimentation, and reports broad performance advantages across scientific tasks. It also identifies software-coverage and scalability boundaries that limit the tutorial’s scope.

  • Problem

    Scientific discovery involves combinatorially large search spaces, expensive evaluations, and black-box objectives and constraints whose mechanisms must be inferred from data.

  • Method

    The tutorial presents Bayesian optimisation through Gaussian-process surrogates, acquisition functions, iterative workflows, scientific case studies, coding guidance, and application-specific extensions.

  • Results

    The tutorial reports consistent BO performance advantages over blind random search across catalysis, materials science, organic synthesis, and molecular design tasks.

  • Takeaways & Limitations

    BO provides a principled framework for selecting experiments in resource-constrained scientific optimisation and is illustrated across diverse natural-science applications.

  • Takeaways & Limitations

    Exact Gaussian-process inference scales as O(n^3) in time and O(n^2) in memory, restricting standard GP use to small-to-medium datasets and limiting coverage of scalable alternatives.

Abstract

from arXiv · show

Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often wastes resources, yields inefficient designs, and misses critical insights. This tutorial presents Bayesian Optimisation (BO), a principled probability-driven framework that formalises and automates this core scientific cycle. BO uses surrogate models (e.g., Gaussian processes) to model empirical observations as evolving hypotheses, and acquisition functions to guide experiment selection, balancing exploitation of known knowledge and exploration of uncharted domains to eliminate guesswork and manual trial-and-error. We first frame scientific discovery as an optimisation problem, then unpack BO's core components, end-to-end workflows, and real-world efficacy via case studies in catalysis, materials science, organic synthesis, and molecule discovery. We also cover critical technical extensions for scientific applications, including batched experimentation, heteroscedasticity, contextual optimisation, and human-in-the-loop integration. Tailored for a broad audience, this tutorial bridges AI advances in BO with practical natural science applications, offering tiered content to empower cross-disciplinary researchers to design more efficient experiments and accelerate principled scientific discovery.

4 Coding an Optimisation Problem

Scientific discovery can be organized as an optimization problem in which Bayesian methods connect evolving hypotheses, empirical evidence, and decisions about experiments. The tutorial presents this framework through historical discovery paradigms, practical workflows, and scientific case studies.

  • Scientific discovery paradigms: Scientific discovery has progressed from demonstrative and inductive approaches to hypothetico-deductive methods that test and refine tentative hypotheses against evidence.These paradigms provide the conceptual background for Bayesian approaches to discovery.
  • Bayesian scientific discovery: Bayesian discovery represents uncertainty across competing hypotheses and updates beliefs with empirical data, producing posteriors that summarize knowledge and guide decisions.This distributional perspective makes belief dynamics across large hypothesis spaces computationally actionable.
  • Optimisation formulation: Bayesian optimisation frames objective-oriented scientific inquiry as decision-making under unknown mechanisms, where experiments jointly learn the model and determine feasibility.The objective and constraints may both depend on an unknown system mechanism, requiring real-world experimentation.
  • BO workflow: The BO workflow alternates surrogate-model fitting, acquisition-guided experiment selection, evaluation, and model updating to navigate large scientific search spaces.The acquisition function balances exploitation of promising hypotheses with exploration of uncertain regions.
  • Scientific case studies: In antibody design, AntBO uses a Gaussian process, a trust region, and oracle evaluations to avoid biologically unsatisfiable sequences while refining sequence-affinity predictions.The trust region restricts proposals toward similar, developable antibodies.
  • Scientific case studies: Fewer than fifty sequence evaluations yielded high-affinity antibody candidates, while BO catalyst discovery reduced experimental cost by orders of magnitude versus trial-and-error.The catalyst strategy identified a Cu-doped RuO2 catalyst combining low overpotential and excellent durability.
  • Tutorial focus: The tutorial addresses gaps in optimization framing and code-level guidance through structured case studies, hands-on instructions, and content stratified by technical complexity.Its stated aim is to make BO accessible and applicable to scientific practitioners with different backgrounds.

2 Optimisation Problems and Algorithms

Scientific discovery can be formalised as constrained optimisation over experimental designs, with unknown mechanisms, noisy observations, and costly evaluations making sample-efficient sequential methods necessary. Bayesian optimisation addresses this setting by combining probabilistic modelling with decisions that balance exploration and exploitation.

  • Scientific discovery as optimisation: Scientific discovery seeks actions that optimise reward over outcomes generated by an unknown mechanism, subject to potentially mechanism-dependent constraints.This formulation extends discovery from identifying and explaining hypotheses to determining how to act.
  • Challenges: The objective and constraints are often black boxes, so their values and feasibility must be learned through real-world experiments rather than analytic expressions.Experimental outcomes may also be noisy because of measurement error and latent variables.
  • Challenges: Expensive experiments make sample efficiency paramount: useful solutions should be identified with as few trials as possible.Examples include chemical synthesis, clinical trials, and high-energy physics experiments.
  • Bayesian optimisation: Bayesian optimisation provides a sequential decision-making framework that combines Bayesian inference with modelling of the unknown objective.It is presented as a computational analogue of Bayesian hypothetico-deductive scientific discovery.
  • Bayesian optimisation: Acquisition-driven optimisation balances exploitation of promising hypotheses with exploration of unknown regions to guide informative future experiments.This supports iterative use of probabilistic reasoning when every trial counts.
  • Theoretical evaluation: Sublinear cumulative regret is an ideal target because it supports convergence in average performance and efficient identification of near-optimal designs.The tutorial links vanishing average regret and convergence, providing a basis for analysing sequential optimisation algorithms.

4 Coding an Optimisation Problem

Coding an optimisation problem requires representing heterogeneous scientific design spaces, connecting batches of candidate designs to experiments or simulations, and returning objective values for model updates. The tutorial illustrates these tasks with HEBO and supports both automated and human-in-the-loop evaluation.

  • Problem setup: Unconstrained BO defines the objective over the entire design space, while constrained BO is treated as an advanced topic beyond this coding section.The implementation examples focus on unconstrained optimisation.
  • Design-space representation: Scientific design spaces may combine continuous, integer, logarithmic, stepped, exponent, Boolean, and categorical variables.HEBO is used because it supports multiple variable types encountered in chemistry and materials science.
  • Design-space representation: Logarithmic transformations balance exploration when parameters span orders of magnitude, while logarithmic integer types preserve this treatment for discrete values.The tutorial gives reaction-rate constants and molecule counts as examples.
  • Design-space representation: Categorical variables encode unordered options, unlike integers whose values imply an ordered magnitude.Solvent selection is categorical, whereas the number of cooling stages is naturally integer-valued.
  • Objective evaluation: An evaluation function receives a batch of design points and returns one scalar objective value per point, enabling parallel experiment or simulation execution.The workflow translates designs into equipment or simulator commands, executes evaluations, and preprocesses results.
  • Objective evaluation: When automatic acquisition is unavailable, users can manually perform each experiment and enter the observed results while preserving HEBO’s batch interface.This provides a human-in-the-loop workaround for real-world evaluations.

5 Case Studies

The case studies formulate diverse scientific problems as optimisation tasks spanning continuous, simplex, mixed-variable, and high-dimensional spaces. Across the previewed studies, Bayesian optimisation outperforms random trials under the same fixed experimental budget, while reparameterisation makes bounded-simplex optimisation compatible with standard hyper-rectangular tools.

  • Case-study formulation: The case studies cover high-dimensional continuous, simplex, mixed-variable, and high-dimensional feature spaces, using mainly database-fitted mock oracles for demonstration.The formulation process translates scientific questions into well-posed optimisation problems.
  • Performance preview: BO consistently outperforms random trials across all case studies, achieving superior final objective values under the same fixed experimental budget.The passage describes this as a preliminary preview, with detailed trajectories and analysis deferred to Section 19.
  • Photocatalytic catalyst design: Photocatalytic water-splitting catalyst design maximises hydrogen evolution rate by selecting material amounts in a ten-dimensional composition space.Even coarse unit discretisation yields 6^10 candidate compositions, making exhaustive search impractical.
  • High-entropy alloy nanozymes: High-entropy alloy nanozyme optimisation adjusts five elemental ratios to maximise Vmax and minimise KM, subject to bounded proportions summing to one.The combined catalytic-efficiency objective depends nonlinearly on composition and other physicochemical parameters.
  • Simplex reparameterisation: A bounded simplex can be bijectively transformed into the hypercube [0, 1]^(n−1), enabling Bayesian optimisation with standard hyper-rectangular software.The forward and inverse maps preserve feasibility through an invertible reparameterisation.
  • Electrolytic water splitting: Green-hydrogen electrocatalyst discovery is motivated by production costs 2–3 times those of conventional methods and search spaces exceeding 30 dimensions.The passage identifies catalyst expense and suboptimal efficiency as principal economic bottlenecks.
  • Multi-objective optimisation: Multi-objective BO approximates the Pareto frontier when objectives compete and assigning scalarisation weights is difficult.A weighted sum reduces the problem to a single objective but may inadequately represent competing goals.
  • Molecule discovery: Molecule discovery requires sample-efficient optimisation over enormous discrete spaces while respecting structural constraints such as valence and ring closures.Effective optimisation also needs to incorporate chemical domain knowledge.

Understanding the Black Box

Scientific discovery can be framed as sequential optimisation of an unknown, costly and noisy objective. Surrogate modelling addresses this black-box setting by updating probabilistic beliefs from observations and guiding future designs.

  • Black-box optimisation: The objective f is an unknown black box whose real-world evaluations consume substantial time and resources.Sequential model-based optimisation uses prior trial data to approximate f rather than querying it directly at every step.
  • Surrogate modelling: A surrogate model provides an efficient proxy for the true objective, supporting prediction, experimental design, and uncertainty quantification.Observed data are noisy, so the surrogate predicts objectives while representing uncertainty about those predictions.
  • Hypothesis spaces: A hypothesis space H defines candidate models for approximating f and must account for observation noise.Examples include linear, parametric, and function-space hypotheses.
  • Hypothesis spaces: The choice of H balances representational power and computational efficiency while incorporating available prior knowledge.Fitting and evaluating the surrogate should remain substantially cheaper than evaluating the true objective.
  • Probabilistic modelling: Probabilistic surrogates represent beliefs over possible objectives instead of selecting one hypothesis that best fits finite data.This avoids assuming that the true objective lies within the representational capacity of H.
  • Bayesian inference: Bayesian inference updates a prior p(h) to a posterior p(h|D), which is then marginalised to make predictions for new designs.For complex hypothesis spaces, inference may require MCMC or variational methods because exact integrals and sums can be intractable.

7 Gaussian Process

Gaussian processes are widely used Bayesian-optimisation surrogates because they flexibly model unknown functions while supporting analytic Bayesian inference. Their posterior mean predicts objective values, and posterior covariance quantifies uncertainty for exploration.

  • GP foundations: A Gaussian process places a prior directly over functions and encodes structural beliefs such as smoothness and periodicity.Its kernel represents similarity between inputs, while its mean function represents prior expected behaviour.
  • GP foundations: For any finite input set, a Gaussian process defines a multivariate Gaussian distribution over the corresponding function values.The mean vector and covariance matrix are determined by the mean and kernel functions.
  • GP regression: Conditioning a GP on jointly Gaussian observations produces a posterior GP, including the standard regression case with noisy measurements.In standard regression, observations satisfy yi = f(xi) + εi with i.i.d. Gaussian noise.
  • GP regression: The posterior mean estimates the objective at new points, while posterior covariance quantifies uncertainty that supports exploration in Bayesian optimisation.The usual i.i.d. additive-noise assumption is strong but often reasonable in practice.
  • Kernel design: Kernel choice encodes assumptions about function structure, including smoothness, linearity, periodicity, and controllable degrees of smoothness.RBF, Matérn, linear, and periodic kernels provide different structural assumptions through their hyperparameters.
  • Structured inputs: Kernels for strings, graphs, sequences, and other structured objects allow GPs to exploit domain-specific similarity beyond numeric inputs.String and Fisher kernels support applications involving biological sequences and other non-vectorial design spaces.
  • Kernel design: Kernel sums, products, and scaling construct composite kernels that capture additive effects, interactions, and heterogeneous design spaces.These operations support combinations such as periodicity with smoothness or categorical with continuous variables.
  • Computational considerations: GP covariance-matrix inversion scales as O(n^3) in time and O(n^2) in memory, motivating incremental, sparse, or inducing-point approximations for larger datasets.Cholesky-based updates can accelerate sequential computation.

8 Implementation

The implementation section translates surrogate-modelling concepts into practical BO software workflows centered on Gaussian processes. It emphasizes configurable libraries, alternative surrogate models, and the distinction between introductory examples and scalable custom deployments.

  • Model configuration: Surrogate predictive accuracy and uncertainty quantification depend strongly on feature representations and hyperparameter choices.Relevant representations include molecular graphs, descriptors, fingerprints, sequence embeddings, and structural features.
  • Hyperparameter inference: Maximising marginal likelihood provides a principled way to select surrogate hyperparameters from observed data.A fully Bayesian alternative places priors over hyperparameters and integrates them out.
  • Hyperparameter inference: Hierarchical Bayes can provide more robust uncertainty estimates when data are scarce or hyperparameters are poorly identified.The required integration is often approximated using MCMC or variational inference.
  • Software workflows: The tutorial demonstrates Gaussian-process surrogate implementation in BO frameworks and from scratch using Python libraries.Its coding patterns are presented as invariant across applications and problem settings rather than as case-specific studies.
  • Software workflows: BO software commonly infers GP hyperparameters from data, but interface design can limit users’ ability to customise the surrogate.HEBO exposes some configuration through model_config and supports multiple surrogate families.
  • Software workflows: HEBO’s default GP uses a Matérn kernel for numeric variables and combines it with suitable kernels for categorical variables.Kernel hyperparameters are initialized from the dataset, providing a practical default for heterogeneous variables.
  • Custom implementations: Scikit-Learn prioritizes accessible prototyping for small to moderate datasets, whereas GPyTorch emphasizes scalability and customizability.The tutorial uses both libraries to expose Gaussian-process modelling workflows.
  • Scope of implementation guidance: The code examples are deliberately minimal and introductory rather than exhaustive guidance for production or large-scale GP deployment.Readers seeking comprehensive technical coverage are directed to official documentation and community resources.

Acting and Decision Making

In scientific discovery, decision making maps current knowledge into the next experiment. BO formalises this mapping with a policy that selects the next design from accumulated data and the posterior model.

  • Decision making: Scientific decision making translates beliefs about hypotheses into subsequent experiments aimed at learning about the unknown and achieving higher objective values.
  • Decision making: A policy π maps accumulated data D_t to the next design x_t+1 ∈ X using the posterior model M(f̂|x̂, D_t).This policy determines the next action in both sequential model-based optimisation and Bayesian optimisation.

10 Acquisition Function

Acquisition functions turn the surrogate model’s predictions and uncertainty into a tractable criterion for selecting the next experiment. They formalise exploration–exploitation trade-offs and can be interpreted through expected utility, although common choices are typically myopic.

  • Acquisition-function role: The acquisition function converts the surrogate model into a tractable optimisation problem for choosing the next design.Its optimisation is substantially cheaper than evaluating the costly black-box objective.
  • Exploration–exploitation: Acquisition functions balance exploitation of promising predictions against exploration of uncertain regions.They typically incorporate both the surrogate’s predicted mean and uncertainty.
  • Common acquisition functions: PI targets the probability of improving on the current best, EI also accounts for improvement magnitude, and UCB trades off posterior mean against uncertainty.UCB uses β to control the exploration–exploitation balance; larger β encourages exploration.
  • Common acquisition functions: Thompson Sampling samples a function from the posterior surrogate and selects its maximiser, inherently balancing exploration and exploitation through uncertainty.Its simplicity and theoretical guarantees make it attractive in parallel or distributed settings.
  • Utility and decision making: Information gain is another possible utility, but the tutorial does not cover advanced utility-function design.This scope boundary distinguishes the mathematical exposition from broader Bayesian experimental-design objectives.
  • Utility and decision making: Acquisition functions can be grounded in expected posterior utility, with utility defined over outcomes after future experiments.The tutorial notes that standard acquisition functions usually use myopic decisions rather than explicitly modelling long-term effects.

12 Coding the Acquisition Function

The coding section shows how acquisition functions are selected and implemented in BO software. Examples cover HEBO, BO, Bgolearn, Scikit-Learn, and GPyTorch workflows, with task direction and exploration parameters made explicit.

  • Software interfaces: BO software exposes acquisition functions through optimiser interfaces, allowing defaults or explicit user customisation.HEBO selects an acquisition according to the optimiser class but also accepts a configured acquisition class.
  • HEBO acquisition classes: HEBO provides LCB, Mean, Sigma, EI, logEI, weighted EI, MES, and multi-objective acquisition classes.These classes encode pure exploitation, pure exploration, confidence bounds, expected improvement, entropy search, and multi-objective strategies.
  • HEBO acquisition classes: In HEBO, acquisition functions are implemented for minimisation, so LCB replaces the maximisation-oriented UCB convention.LCB is defined as µ(x) − κσ(x), where κ controls the trade-off between exploration and exploitation.
  • Configuration examples: The examples demonstrate explicit selection of LCB, Mean, Sigma, EI, and MACE, including a custom exploration parameter κ=3.0 for LCB.Mean and Sigma represent pure exploitation and pure exploration, respectively.
  • Bgolearn: Bgolearn exposes acquisition methods such as EI, Knowledge Gradient, and Probability of Improvement through corresponding method calls.The section directs readers to the package documentation for further acquisition-function details.
  • Numerical implementations: Scikit-Learn and GPyTorch implementations obtain posterior means and standard deviations from fitted models before evaluating UCB, EI, and PI.GPyTorch examples use tensor candidate inputs and evaluation-mode predictive computations.

Algorithmic Workflow and Implementation

The basic BO workflow repeatedly updates a surrogate with observations, optimises an acquisition function, evaluates the proposed design, and augments the dataset. Practical implementations also require initialization, inner optimisation, stopping, and final-recommendation choices.

  • Iterative workflow: The workflow accepts a design space and objective evaluator and returns the best solution found.The evaluator may be a manual experiment or an external interface.
  • Iterative workflow: BO starts with initial observations, fits a surrogate, constructs an acquisition function, selects its maximiser, evaluates the objective, and repeats.The loop terminates according to a stopping criterion after each new observation is added to the dataset.
  • Extensions: Extensions include batched acquisition, human-in-the-loop interaction, and more sophisticated surrogate models.These extensions modify the basic workflow to increase its flexibility across applications.
  • Workflow sub-modules: Initial observations are normally collected because the surrogate requires data to construct its initial estimate of the objective.Starting from an empty dataset is possible with a well-defined prior but uncommon in practice.
  • Workflow sub-modules: Acquisition maximisation is an inner optimisation problem whose solution method depends on the design space.Exhaustive search suits small finite spaces, while evolutionary or random search can handle more complex or mixed-variable spaces.
  • Workflow sub-modules: Stopping can use a fixed evaluation budget or criteria based on observed improvement, the surrogate, or the acquisition function.A maximum number of iterations or function evaluations is the simplest widely used choice.
  • Workflow sub-modules: The final recommendation may be the best observed point or a surrogate-predicted optimum when observations are noisy or sparse.The choice depends on whether observed values or predicted means provide the preferred basis for recommendation.

14 The GP-UCB Algorithm and Theoretical Guarantees

GP-UCB combines a Gaussian-process surrogate with an upper-confidence-bound acquisition function and provides a theoretically analysed BO baseline. Under stated assumptions, its cumulative regret grows sublinearly, so average regret vanishes for commonly used kernels.

  • GP-UCB: GP-UCB uses a Gaussian process surrogate and the UCB acquisition function, making it a practical baseline with strong regret analysis.The algorithm is widely studied as a canonical BO instantiation.
  • Assumptions: The GP-UCB formulation assumes a Gaussian-process prior and independent, identically distributed Gaussian observation noise.Observations follow y_t = f(x_t) + ε_t with ε_t ∼ N(0, σ^2).
  • GP-UCB: At each iteration, GP-UCB selects the point maximising an upper confidence bound built from posterior mean and standard deviation.The exploration coefficient β_t controls the balance between high predicted values and uncertain regions.
  • Theoretical guarantees: Under mild conditions, GP-UCB has sublinear cumulative regret, implying that average regret per step vanishes as evaluations increase.This supports increasingly efficient identification of near-optimal solutions.
  • Theoretical guarantees: With high probability, the regret bound scales as O(√Tβ_Tγ_T), where γ_T is the maximum information gain and depends on the kernel and function complexity.The bound is stated for finite and compact continuous design spaces under their respective assumptions.
  • Theoretical guarantees: For commonly used kernels such as the squared exponential, sublinear information gain yields no-regret behaviour and eventual identification of near-optimal solutions.The result is expressed as R_T/T → 0 as T → ∞.
  • Scientific extensions: Scientific BO extensions address parallel experimentation, heteroscedasticity and non-stationarity, and human expertise within the optimisation loop.HEBO is described as addressing heteroscedasticity and non-stationarity, while human intervention can preserve trust and use domain expertise.

16 Coding BO Workflows

This section presents code-level Bayesian optimisation workflows using HEBO, Bgolearn, and custom Gaussian-process implementations. The examples cover standard, human-in-the-loop, and molecularly adaptable workflows while noting that production use requires additional engineering.

  • HEBO workflow: HEBO implements the standard BO loop by suggesting candidates, evaluating the original objective, observing its negative for minimisation, and tracking the best original-scale result.HEBO initializes random samples, fits its surrogate and acquisition function during suggestion, and selects the best observed design after a fixed iteration budget.
  • Human-in-the-loop BO: Human-in-the-loop BO generates a batch of candidates, lets an expert select one, evaluates that point, and feeds the observation back to the optimiser.Input validation ensures that the selected candidate index is valid.
  • Bgolearn workflow: Bgolearn demonstrates a complete low-dimensional workflow that fits a surrogate, maximises Expected Improvement over a dense candidate grid, evaluates the selected point, and updates the dataset.The example uses a synthetic two-dimensional black-box objective and a 20-iteration loop.
  • Custom BO workflow: A custom implementation combines a Matérn-kernel Gaussian process with Upper Confidence Bound acquisition in a two-dimensional design space, while the loop structure also applies to EI and PI.The working example sets the exploration parameter to β = 2.0 and maximises the acquisition function by dense-grid search.
  • Practical use: The minimal custom examples are illustrative rather than production-ready because real applications may require batch evaluation, mixed-variable support, advanced acquisition optimisation, and robust noise handling.The section recommends established libraries such as HEBO, BoTorch, or BayesOpt/GPy for these complexities.

17 Integrating BO with Scientific Tools

This section adapts Bayesian optimisation to molecular search spaces by mapping SMILES strings into chemically meaningful descriptor vectors. Gaussian-process surrogates and acquisition functions then guide batched selection from a large discrete candidate set.

  • Molecular representation: Molecular BO addresses the mismatch between SMILES-based molecular spaces and vector-oriented surrogate models by using domain-specific feature extraction.The molecular search space is a finite set of candidate molecules with black-box performance scores.
  • Molecular representation: RDKit converts each SMILES string into descriptors such as molecular weight, logP, hydrogen-bond counts, topological polar surface area, and ring-related features.These descriptors provide the continuous vector input used by the Gaussian process.
  • Surrogate and acquisition: The surrogate uses a Gaussian process over descriptor vectors, while UCB combines its posterior mean and variance to score candidate molecules.The implementation uses a Matérn kernel and a constant mean function.
  • Robust handling: Invalid SMILES receive a fixed low score and zero uncertainty, allowing the descriptor-aware surrogate to handle malformed inputs robustly.Valid descriptor vectors are added to the training data and used to fit the Gaussian process.
  • Candidate selection: Because exhaustive search over all ZINC molecules is infeasible, each iteration samples a large candidate subset and selects the top-k molecules according to UCB.This procedure supports batched acquisition before evaluation and dataset updating.

Demonstrative Experiments and Conclusion

This part empirically evaluates Bayesian optimisation across diverse scientific discovery tasks using systematic experiments and presents practical insights into its utility and limitations. It also discusses implementation barriers in real laboratory settings.

  • Demonstrative experiments: The experiments evaluate BO across catalysis, materials science, organic synthesis, and molecular design using a consistent framework.The stated purpose is to provide evidence-based insight into BO performance across real-world scientific discovery tasks.
  • Demonstrative experiments: The discussion examines how constrained spaces, mixed variables, high dimensionality, and discrete structured search spaces shape optimisation behaviour.It also compares standard BO implementations with frameworks such as HEBO.
  • Conclusion: The concluding discussion highlights practical limitations and implementation barriers to deploying BO in real laboratory settings.It places these constraints alongside the alignment between BO and the scientific method.

18 Experimental Settings

The experimental settings cover mathematical and structured scientific optimisation problems under a consistent evaluation protocol. Performance is assessed with task-appropriate objectives and regret or overpotential curves, including a discrete molecular benchmark.

  • Experimental design: The study evaluates BO across numerical or categorical design spaces and non-conventional structured spaces such as molecules.The protocol is designed to ensure comparability across cases.
  • Mathematical design spaces: The mathematical cases include HER catalyst design, HEA nanozyme formulation, OER electrocatalyst design, and Buchwald-Hartwig synthesis yield optimisation.These spaces mix bounded real-valued and categorical variables.
  • Evaluation protocol: Experiments use mock oracles based on predictive models trained on empirical datasets to evaluate optimisation strategies without the cost of real experiments.The protocol compares three distinct optimisation strategies.
  • Evaluation metrics: For HER, HEA, and BH, performance is measured by regret for the best solution found, whereas OER is evaluated by overpotential in millivolts.Faster declines in the curves indicate better optimisation performance.
  • Molecular optimisation: The molecular benchmark maximises QED over 250,000 discrete drug-like molecules using RDKit as the oracle and descriptor-based BO with a total budget of 100 evaluations.The configuration includes 20 initial points, batch size 10, and 20,000 acquisition candidates.

19 Experimental Results

Across the evaluated scientific design tasks, advanced Bayesian optimisation consistently outperforms random search, while standard BO is less reliable in heterogeneous mixed-variable settings. HEBO combines strong final performance with faster convergence, and BO remains effective when paired with suitable representations and constraints.

  • Advanced BO frameworks consistently outperform random search across the evaluated tasks, whereas standard BO performance depends on the design space.Standard BO improves over random search in continuous, constrained, and high-dimensional continuous tasks but underperforms in the mixed-variable OER task.
  • HEBO outperforms random search across all four mathematical design problems and converges faster than BO-LCB.HEBO achieves lower regret or overpotential after equivalent iterations and identifies near-optimal solutions more rapidly in early iterations.
  • 19.1 Photocatalytic HER Catalyst Design: 85% reduction in required experimental effort is reported for HEBO in photocatalytic HER catalyst design.HEBO reaches random search’s final performance within the first 30 iterations and finishes with regret 9.2 versus random search’s 21.8 after 200 iterations.
  • 19.2 High-Entropy Alloy Nanozyme Formulation: HEBO reduces regret from around 290 to 153 in constrained HEA nanozyme formulation, compared with around 238 for BO-LCB and 244 for random search.The bounded simplex is mapped to a unit box, while HEBO’s non-stationary modelling and multi-objective acquisition support navigation of the constrained region.
  • 19.3 OER Electrocatalyst Design: HEBO reduces OER overpotential from around 320 mV to 219 mV, while BO-LCB ends at approximately 252 mV and random search at 245 mV.This mixed-variable task shows that standard BO-LCB can fail to outperform random search, whereas HEBO models shifts across categorical and continuous variables.
  • 19.4 Buchwald-Hartwig Reaction Yield Optimisation: HEBO reduces Buchwald-Hartwig reaction-yield regret from around 70 to 9.9, compared with around 25 for BO-LCB and 47 for random search.The task uses the top 20 features selected from 530 DFT-derived features, showing that BO can remain effective after conventional dimensionality reduction.
  • 19.5 Molecular QED Optimisation: Descriptor-based GP-UCB reaches a final mean QED score of 0.918 after 100 iterations, versus 0.881 for random search.RDKit descriptors map discrete SMILES strings into a continuous space so the Gaussian-process surrogate can exploit chemical similarity.

20 Discussion

The discussion finds Bayesian optimisation broadly applicable across scientific discovery tasks, while showing that variant robustness and practical software requirements shape real-world performance and adoption.

  • Applicability and value: Bayesian optimisation showed consistent advantages across all tasks, supporting its broad applicability to scientific discovery.The tutorial links this consistency to BO’s iterative realisation of the Bayesian hypothetico-deductive scientific method.
  • Applicability and value: BO operationalises scientific inquiry through surrogate-model fitting, acquisition-function maximisation, experimental evaluation, and model updating.The surrogate represents current scientific belief, the acquisition function selects the next experiment, and new data updates that belief.
  • Limitations and future directions: Practical adoption remains constrained by manual design-space transformations, dimensionality reduction, and domain-specific feature engineering that require optimisation expertise.The tutorial calls for user-friendly, domain-specialised tools that automate these steps for experimental scientists.
  • BO variants: HEBO combines mixed-variable kernels, heteroscedastic and non-stationary modelling, and the MACE acquisition function for complex scientific optimisation.These components address variable types, varying measurement noise, non-smooth objectives, and exploration–exploitation balance.
  • BO variants: HEBO consistently outperformed random search and standard BO across the reported tasks, indicating robust performance under real-world scientific complexities.The discussion attributes this robustness to HEBO’s support for mixed variables, non-ideal noise, non-stationarity, and multi-objective acquisition synthesis.
  • Empirical efficacy: The tutorial reports that BO reduced the number of experiments required by 60%–85% across case studies while outperforming random search and traditional trial-and-error.It presents this result across catalysis, materials science, organic synthesis, and molecular design tasks.
Loading 2604.01328v3…