PaperlayerView exact revision on arXiv

Source-linked AI summary

Modern Bayesian Experimental Design

Tom Rainforth, Adam Foster, Desi R Ivanova, Freddie Bickford Smith

arXiv:2302.14545v2stat.MLcs.AIcs.LGstat.CO

TL;DR

Optimizing data acquisition remains less explored than using data, despite experiment design’s importance across science and industry. This review surveys recent advances in Bayesian experimental design and areas for future development, reporting that recent developments have overcome some key computational bottlenecks.

  • Problem

    Quantitative methods for optimizing data acquisition have been less explored than methods for using data, despite experiment design’s fundamental importance across science and industry.

    Source: §1 · p. 1

  • Method

    The review surveys recent developments in Bayesian experimental design and discusses areas for future development.

    Source: §1 · p. 1

  • Results

    Recent developments have systematically addressed some key computational bottlenecks in Bayesian experimental design.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The review offers a guide to deploying Bayesian experimental design in practice with modern techniques.

    Source: §1 · p. 1

    More work is needed to understand how BED-collected data affects downstream model selection, checking, and non-Bayesian uses, and to develop approaches ensuring useful properties beyond inference.

    Source: §5.3 · p. 9

Abstract

from arXiv · show

Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments. However, its deployment often poses substantial computational challenges that can undermine its practical use. In this review, we outline how recent advances have transformed our ability to overcome these challenges and thus utilize BED effectively, before discussing some key areas for future development in the field.

1. INTRODUCTION

Bayesian experimental design formalizes information-maximizing experiment choice, but computational bottlenecks have limited practical uptake; this review surveys recent advances, practical deployment, and future development.

1. INTRODUCTION

  • BED formalizes information-maximizing experiment design as an optimal, model-based framework grounded in information theory.Experiment design matters across science and industry, yet quantitative methods for optimizing data acquisition have received less attention than methods for using collected data.Source: §1 · p. 1
  • When an appropriate model or simulator is available, BED can help automate difficult manual design and adapt decisions to outcomes as experiments proceed [42] [70].Sequential BED also appears as Bayesian adaptive design, adaptive design optimization, Bayesian active learning, and Bayesian optimization.Source: §1 · p. 1
  • BED uptake has remained modest partly because of computational bottlenecks that impede practical deployment, especially in adaptive settings.Other historical factors include limited application-field expertise, past gaps in regulatory guidance, and philosophical disputes over Bayesian statistics.Source: §1 · p. 1
  • This review surveys advances that reduce BED’s computational barriers, explains how to deploy modern techniques in practice, and identifies future development needs.It also describes how the field has changed since earlier reviews.Source: §1 · p. 1

2. INFORMATION-THEORETIC DESIGN

Information-theoretic design chooses experiments to maximize expected utility from data, here measured by information gained about a quantity of interest. BED formalizes this with expected information gain and extends it to sequential design, while the section contrasts its Bayesian consistency with limitations of Fisher-information-based approaches.

2. INFORMATION-THEORETIC DESIGN

  • Experimental design chooses controllable settings ξ to maximize expected utility from outcomes y, with this review focusing on information gained about a quantity θ.θ may represent real-world quantities, model parameters, or future predictions.Source: §2 · p. 2

2.1 Bayesian Experimental Design

  • BED selects designs maximizing expected information gain (EIG), the expected reduction in Shannon entropy from prior to posterior, equivalently mutual information or expected posterior-to-prior KL divergence.The EIG can concern parameters, function optima, algorithm outputs, or downstream predictions; BED more generally permits posterior-based utilities, though the review focuses on EIG.Source: §2.1 · p. 2

2.2 Bayesian Adaptive Design

  • Bayesian adaptive design (BAD) maximizes incremental EIG at each step using the history, updating all underlying model parameters so incremental gains sum to total EIG.This iterative process incorporates previous observations and is usually more effective than choosing all designs upfront.Source: §2.2 · pp. 2–3, Figure 1 · p. 3

2.3 Why Take a Bayesian Approach?

  • Fisher-information methods depend on unknown true parameters, so approximations, point estimates, or prior averaging can introduce errors, ignore uncertainty, or undermine sequential consistency.Bayesian design instead provides a self-consistent framework incorporating available information without asymptotic approximations or model restrictions; prior averaging already specifies a Bayesian model.Source: §2.3 · p. 3
  • Classical Fisher-information design must reduce its matrix to summary criteria such as trace or determinant, which may miss joint-distribution gains and vary with parameterization.Certain formulations are invariant to parameterization.Source: §2.3 · p. 3
  • EIG-based BED remains computationally challenging and depends on model fit, so heuristic or non-Bayesian updates may be needed when accurate modeling is infeasible.Such updates may arise even when EIG remains the design objective in BAD.Source: §2.3 · pp. 3–4

3. A COMPUTATIONAL REVOLUTION

BED’s information-theoretic foundations do not remove the computational barriers to estimating and optimizing expected information gain (EIG), especially when decisions must be made quickly. Recent methods address these challenges through debiasing, functional and variational approximations, and stochastic-gradient optimization, including for implicit models.

3.1 Nested Estimation

  • Nested estimation is difficult because EIG is generally doubly intractable, and conventional Monte Carlo cannot directly estimate it [47].For a small discrete outcome space, enumeration and Rao–Blackwellization can help; traditional nested Laplace approximations may be biased, whereas NMC is asymptotically consistent [6].Source: §3.1 · p. 4
  • For continuous or large outcome spaces, estimating EIG requires nested estimates of p(y|ξ) or p(θ|y,ξ), while NMC is biased at finite M and its cost scales as C = NM.Basic NMC has asymptotic mean squared error O(a/N + b/M^2); importance sampling can reduce its constants and finite-sample bias and variance [43], but typical problems cannot beat O(C^-2/3).Source: §3.1 · p. 4

3.2 Debiasing Schemes

  • Multi-level Monte Carlo can yield unbiased EIG and gradient estimates with finite expected variance and cost, recovering the O(C^-1/2) rate under stated conditions.Antithetic coupling and an importance sampler enable this result when r(ℓ) ∝ 2^-τℓ for 1 < τ < 2; each increment can still require many likelihood evaluations.Source: §3.2 · p. 5

3.3 Functional and Variational Approaches

  • Functional approximations replace the intractable p(y|ξ) term with a learned q(y|ξ), making approximation and subsequent Monte Carlo costs additive rather than multiplicative [43].This can theoretically recover O(C^-1/2) convergence in total cost, though practical performance depends on how accurately the functional approximation can be learned.Source: §3.3 · p. 5

3.3.1 Variational Approaches

  • Variational approximations yield EIG lower bounds from approximating p(y|ξ) or the posterior, while the expected NMC estimator gives an upper bound that tightens as M increases [43].These bounds are exact when their approximations are exact, and a learned posterior approximation can also serve as a proposal for NMC [43].Source: §3.3.1 · p. 5

3.3.2 Estimation for Implicit Models

  • Implicit likelihoods require approximating an additional intractable density; some variational bounds accommodate them, while certain bounds also avoid evaluating an implicit prior [43] [44] [73] [87].Other approaches approximate the likelihood density or estimate the likelihood-to-marginal ratio using logistic regression [86].Source: §3.3.2 · p. 6

3.4 Optimization

  • Common optimization wrappers, including enumeration and Bayesian optimization, can be slow because they require many EIG estimates and often scale poorly to high-dimensional designs [43] [86].Sampling-based approaches can improve efficiency and scaling but remain expensive and may require challenging custom proposals.Source: §3.4 · p. 6

3.4.1 Stochastic Gradient Schemes

  • Stochastic gradients can optimize EIG without repeatedly estimating it, and variational bounds or debiased gradients can make this optimization consistent [44].The multi-level Monte Carlo gradients avoid a variational family but have higher per-sample cost and may have greater variance; gradient methods also remain challenging for discrete designs.Source: §3.4.1 · pp. 6–7

4. FROM DESIGNS TO POLICIES

Traditional Bayesian adaptive design (BAD) can be costly because it requires Bayesian inference at each step, and its greedy decisions ignore effects on future information. Recent policy-based methods learn design decisions upfront to address both limitations.

4. FROM DESIGNS TO POLICIES

  • Traditional BAD is often impractical because it requires costly Bayesian inference at each step, and its greedy decisions can miss information gains in future steps [56].Methods from Bayesian optimization have sought to address myopia, but add further cost [57] [75].Source: §4 · p. 7
  • Rather than optimize designs directly, policy-based methods train upfront to map previously gathered data to design decisions [13] [42] [70] [73].The earlier approach of Huan and Marzouk (2016) [70] mapped explicit posterior representations to designs, so posterior updates were still required at each iteration.Source: §4 · p. 7

4.1 Deep Adaptive Design

  • DAD learns a network upfront that maps experiment history directly to the next design, avoiding inference during the experiment and requiring only a forward evaluation.The network supports real-time adaptation and can be reused across experiment realizations, amortizing the cost of adaptation.Source: §4.1 · p. 7, Figure 2 · p. 7
  • DAD has been found to improve decision quality over traditional greedy strategies by learning non-myopic policies and avoiding errors from approximate inference [13] [42] [73].Its network can make decisions almost instantly during experiments, and its adaptation cost can be amortized across multiple experiment realizations.Source: §4.1 · p. 7

4.2 Learning Policies

  • Policy learning targets the policy maximizing total EIG over all T experiment steps, optimizing its parameters φ while marginalizing designs that depend on prior outcomes [42] [70] [73].DAD uses variational bounds and stochastic gradients; iDAD extends training to implicit likelihood models, while other work uses reinforcement learning [13] [94].Source: §4.2 · p. 8

5. FUTURE DIRECTIONS

The review identifies opportunities to extend policy-based BAD and link BED with related fields, while emphasizing the need to manage model misspecification and use more flexible models on larger problems.

5.1 Policy-Based BAD

  • Policy-based BAD has improved deployment speed and design quality, but scaling to complex problems remains a major challenge.Future work could improve network architectures, objectives and estimators, and training mechanisms to address model and design-space dimensionality and smoothness, long experiment sequences, and discrete design components.Source: §5.1 · p. 8

5.2 Linking with Related Areas

  • BED research can benefit from renewed exchange with Bayesian active learning, a BAD special case, while addressing limits in active-learning applications and BALD’s focus on model parameters.Active learning has experience with large, high-dimensional datasets, but is often limited to classification with modest class counts and can face computational difficulty in large-batch acquisition; policy-BAD may also help. BALD may be suboptimal when the goal is information about downstream predictions.Source: §5.2 · pp. 8–9
  • BAD’s policy-learning connection to Bayesian reinforcement learning is grounded in a shared Bayes-adaptive decision-process formulation and information-gain rewards.The fields differ in how policies are trained and used, but further exchange of ideas may benefit both.Source: §5.2 · p. 9

5.3 Model Misspecification and Downstream Analysis

  • Model misspecification can cause BAD to repeatedly query similar designs and produce poor datasets, making its failure mechanisms and mitigations important research needs.For linear regression, coefficient EIG is maximized at input extrema regardless of prior, which may prevent exploration of interior designs when the true relationship is not linear. Misspecification also matters when gathered data supports downstream Bayesian or non-Bayesian analyses, model checking, or machine-learning training; future work should study these effects and ensure data quality beyond inference in the chosen model.Source: §5.3 · p. 9

5.4 Models and Applications

  • BED advances should pair more flexible, accurate models with implicit-likelihood methods that can use existing simulators and enable study of larger, more complex problems.Because BED performance depends on its underlying model, practitioners should use models that reflect their beliefs rather than defaulting to simple models for ease of estimation; finding the limits of newer approaches is future work.Source: §5.4 · pp. 9–10

Related papers

Loading 2302.14545v2…