PaperlayerView exact revision on arXiv

Source-linked AI summary

Risk-Aware Goal-Oriented Bayesian Optimal Experimental Design

John D. Jakeman, Rebekah White, Bart van Bloemen Waanders, Drew P. Kouri, Alen Alexanderian

arXiv:2609.08023v1stat.MEcs.CEstat.CO

TL;DR

Parameter-focused designs can be suboptimal for downstream predictions, and existing goal-oriented criteria do not prioritize rare, high-consequence outcomes. This paper builds a risk-aware Bayesian design framework that composes risk across predictive deviation, QoI aggregation, and datasets. Its examples show that risk preferences can substantially change sensor placements and that parameter-focused designs can underperform for prediction.

  • Problem

    Existing goal-oriented criteria do not encode asymmetric predictive risk preferences or distinguish adverse datasets from typical ones.

    Source: §1 · pp. 2–3

  • Method

    The framework uses risk-quadrangle regret functions to compose risk over posterior predictive deviation, QoI components, and plausible datasets.

    Source: §1 · p. 3, §6 · p. 23

  • Results

    Risk preferences substantially impact the resulting design, with risk-averse and risk-neutral objectives placing their primary measurements at physically distinct locations.

    Source: §6 · p. 23

  • Takeaways & Limitations

    Parameter-oriented approaches can underperform when the objective is predicting a downstream quantity of interest.

    Source: §1 · p. 3, §6 · p. 23

    The diagonal noise-weighting parameterization assumes a Gaussian observation model and scales poorly with extremely large sensor noise beyond approximately 100 candidate locations.

    Source: §6 · pp. 23, 25

Abstract

from arXiv · show

Traditional Bayesian optimal experimental design (OED) selects measurements that best inform a model's parameters. However, such measurements can be suboptimal for downstream predictions. Goal-oriented OED targets the prediction directly. However, the existing goal-oriented criteria value all reductions in predictive uncertainty equally, with no way to prioritize rare, high-consequence outcomes. In this article, we develop a risk-aware framework that composes risk at three levels, each generalizing an ingredient of classical $I$- and $G$-optimal design: a deviation measure of the posterior predictive uncertainty (generalizing the predictive variance), a risk measure across the prediction domain (interpolating $I$-optimal averaging and $G$-optimal worst-case selection), and a risk measure over datasets (generalizing the expectation). We generate each level from a regret function in the risk quadrangle, so that one triple specifies a practitioner's risk preference. We relax the design to continuous weights on the unit simplex and construct a nested-quadrature estimator that is differentiable in the design variable. This enables solving the optimal design problem with gradient-based methods, avoiding a combinatorial search over candidate designs. For a linear-Gaussian lognormal model and a nonlinear extension, we derive closed-form objectives. These give exact references against which we verify that the estimator converges. We demonstrate this framework for finding optimal sensor placements in an inverse problem governed by an advection-diffusion equation. We find that the risk-aware designs substantially outperform the expected-information-gain baseline, which is statistically indistinguishable from a random allocation.

1 Introduction

Parameter-focused designs can leave downstream predictions largely uncertain, while existing goal-oriented criteria do not prioritize rare, high-consequence outcomes. The paper develops a risk-aware Bayesian design framework that targets predictive uncertainty and accommodates risk preferences.

1 Introduction

  • Parameter-focused OED can spend effort on directions weakly coupled to quantities of interest, producing only marginal gains in predictive performance.Goal-oriented OED instead targets the posterior push-forward of the parameter posterior through the parameter-to-QoI map.Source: §1 · p. 2
  • Existing goal-oriented criteria cannot encode asymmetric predictive risk preferences or prioritize adverse datasets over typical ones.Covariance-based criteria penalize deviations on both sides of the mean, while information-gain criteria cannot favor one region of the predictive distribution.Source: §1 · pp. 2–3
  • The framework composes risk at three levels: posterior predictive deviation, aggregation across QoI components, and aggregation across plausible datasets.A regret function in the risk quadrangle maps to matched deviation and risk measures at each level.Source: §1 · p. 3
  • A nested estimator evaluates risk-aware utilities of posterior push-forwards without covariance-criterion linearization or information-gain kernel density estimation.The avoided kernel density estimation degrades beyond a few QoIs and breaks down for continuous prediction fields.Source: §1 · p. 3
  • Continuous simplex weights make the design objective differentiable, and closed-form objectives for a linear-Gaussian lognormal model provide exact references for estimator convergence.In the advection–diffusion example, different risk preferences yield distinct sensor placements, and parameter-focused designs can be suboptimal for goal-oriented objectives.Source: §1 · p. 3

2 Problem Formulation

This section formulates Bayesian inference, goal-oriented prediction, and OED over candidate experimental conditions. It introduces a linear-Gaussian example in which informative parameter measurements can still leave the quantity of interest uncertain.

2.1 Bayesian Inference

  • A parameter posterior is formed by combining the prior with the likelihood of data under a design; evaluating the likelihood requires the parameter-to-observable map.For an additive observation model with Gaussian measurement noise, the likelihood is Gaussian, and a PDE-governed map can make its evaluation expensive.Source: §2.1 · p. 4

2.2 Goal-Oriented Bayesian Inference

  • A vector-valued quantity-of-interest map pushes a parameter density forward to a predictive distribution, whose uncertainty depends on the design through the posterior.Monte Carlo propagation approximates the push-forward by mapping parameter samples through the QoI map.Source: §2.2 · p. 4

2.3 Optimal Experimental Design

  • OED minimizes a design objective over feasible designs, and the paper adopts a convex relaxation over fixed candidate conditions that supports gradient-based optimization.The relaxed design space avoids fixing the number of measurements in advance.Source: §2.3 · p. 5
  • The design is represented by nonnegative weights on a simplex, with each weight denoting the fraction of the measurement budget assigned to a candidate condition.A fixed number of measurements can be recovered by rounding optimized weights to integer allocations.Source: §2.3 · p. 5
  • The expected-information-gain baseline targets parameter uncertainty and does not distinguish parameter directions by their influence on the quantity of interest.The objective minimizes the negated expected KL divergence from posterior to prior.Source: §2.3 · p. 5

2.4 Linear Gaussian Example

  • The running linear-Gaussian example uses a design-dependent noise covariance, with larger candidate weights shrinking effective measurement variance.A Gaussian prior gives a Gaussian posterior with design-dependent covariance and data-dependent mean.Source: §2.4 · pp. 5–6
  • An exponential QoI map yields a lognormal push-forward even though the parameter posterior is Gaussian, providing a non-Gaussian predictive benchmark with closed-form risk-aware objectives.The prediction directions correspond to locations in the prediction space.Source: §2.4 · p. 6
  • A design that concentrates the parameter posterior can leave predictive uncertainty nearly unchanged, while a design aligned with the QoI direction can reduce it sharply.The two designs rank differently under posterior covariance determinant and push-forward standard deviation.Source: §2.4 · p. 6

3 Risk-Aware Goal-Oriented Design Objective

The objective composes three risk-aware summaries of posterior predictive uncertainty: across each QoI component, across the prediction domain, and across datasets. Risk preferences enter through deviation and risk measures generated from regret functions.

3 Risk-Aware Goal-Oriented Design Objective

  • The framework composes three stages: posterior push-forward deviation for each QoI component, risk aggregation across QoI components, and risk aggregation across unobserved datasets.The stages summarize uncertainty conditional on a dataset, over the prediction domain, and across the data marginal, respectively.Source: §3 · p. 6, §3.2 · pp. 8–9

3.1 Deviation and Risk Measures

  • A regret function encodes displeasure with outcomes and links deviation and risk measures, allowing a matched set of operators to reflect practitioner preferences.Symmetric regret penalizes departures on either side; asymmetric regret can emphasize one tail, such as concentrations above a threshold.Source: §3.1 · p. 7

3.2 The Design Objective

  • For fixed data, level 1 measures posterior push-forward deviation for each QoI component; level 2 aggregates these under a prediction-domain measure, and level 3 aggregates r(y, w) across p(y | w).The paper’s mean-plus-standard-deviation safety-margin deviation cannot be written as an expectation of regret, while the mean-plus-variance variant can.Source: §3.2 · pp. 8–9

3.3 Level 1: Deviation of the Posterior Push-forward

  • For each scalar QoI and dataset, level 1 applies a deviation measure to the posterior push-forward, capturing its spread but not its mean.Standard deviation treats both directions symmetrically, whereas AVaR or entropic deviation can emphasize the upper tail when high values carry the consequence.Source: §3.3 · pp. 9, 11

3.4 Level 2: Aggregation Across QoI Components

  • Level 2 aggregates component deviations with a risk measure, interpolating from I-optimal spatial averaging at α = 0 toward G-optimal worst-component selection as α approaches 1.AVaR averages the worst-performing fraction of prediction locations.Source: §3.4 · p. 11

3.5 Level 3: Aggregation Across Data Realizations

  • Level 3 applies a risk measure to the level-2 outcome across plausible datasets; AVaRβ focuses on the worst (1 − β)-fraction rather than average performance.In the running example, the distribution of level-2 outcomes is right-skewed, with a small fraction of adverse datasets producing disproportionately large deviations.Source: §3.5 · p. 12

4 Estimation and Optimization

The section constructs a nested estimator for three-level risk-aware objectives and makes its design dependence differentiable, enabling gradient-based optimization on the simplex.

4 Estimation and Optimization

  • The three levels combine posterior deviation for each QoI, risk across QoI components, and risk over plausible datasets; the estimators specialize to weighted Gaussian likelihoods and simplex designs.Level 1 and level 3 require sampling for general nonlinear models and priors; level 2 aggregates sampled deviations without additional forward-model evaluations.Source: §4 · p. 13

4.3 Gradient Computation and Optimization

  • Holding reparameterized samples fixed makes the sample-average objective differentiable in design weights, allowing gradient-based simplex optimization instead of combinatorial sensor-set search.The implementation enforces weights above 10^-6 because the observation model is singular at zero; experiments use a closed-form gradient that reuses cached quantities without further forward-model evaluations.Source: §4.1 · p. 13, §4.3 · p. 16

4.1 Double-Loop Estimator

  • Reusing prior samples with self-normalized importance weights reduces nominal sampling cost from N × M to N + M, with bias that vanishes as N grows and asymptotic consistency for the discussed utilities.The numerator and evidence estimates are correlated, making the finite-sample posterior-variance ratio biased.Source: §4.1 · p. 14

4.2 The Estimation Algorithm

  • The algorithm reuses shared inner weights and cached QoI values, while its risk operators can be swapped to reflect different preferences.AVaR is the exception: the estimator uses a smoothed surrogate whose bias vanishes as its smoothing parameter tends to infinity.Source: §4.2 · pp. 14–15

5 Numerical Experiments

The experiments show that prediction-focused designs can depend strongly on risk preference, while parameter-focused KL designs can be nearly indistinguishable from random allocations on prediction objectives. The study also demonstrates continuous I–G interpolation and examines sensor placement in an advection–diffusion inverse problem.

5.1 I–G Interpolation

  • Increasing the QoI-level AVaR parameter continuously interpolates from I-optimal averaging to an approximation of G-optimal worst-case selection, although the optimal sensor weights can change discontinuously.The weight jumps arise as the minimizer switches between local minima with nearly identical objective values; the optimal objective itself varies continuously.Source: §5.1 · pp. 16–17

5.2 The Advection–Diffusion Test Problem

  • The test problem estimates an uncertain upstream contaminant source and predicts final-time concentrations at downstream receptors using sensor measurements in an advection-dominated channel with obstructions.The experiment optimizes measurement weights over candidate sensor locations for the downstream prediction.Source: §5.2 · pp. 18–19

5.3 Risk Preference Shapes the Design

  • Across the design objectives, weight concentrates on at most five of 50 candidate sensors, but objectives assign different primary weights; U5's choice between sensors 26 and 38 remains uncertain at this sample size.The gap between the most and least weight assigned to sensor 26 by goal-oriented objectives exceeds 0.4; U5's bootstrap weight at sensor 38 overlaps U1's, while its sensor 26 weight is only marginally larger.Source: §5.3 · p. 21
  • Strongly tail-averse U3 and U4 concentrate weight most narrowly and leave sensor 6 at or below uniform allocation, unlike U1, U2, and U5; bootstrap boxes are narrow relative to these inter-objective gaps.The KL design instead spreads median weights across sensors 38, 8, 6, and 34, while assigning essentially zero to sensor 26, which goal-oriented objectives weight up to approximately 0.72.Source: §5.3 · p. 21

5.4 The Cost of Optimizing for the Wrong Objective

  • The KL-optimal design performs nearly indistinguishably from random allocation on goal-oriented objectives, while goal-oriented designs perform no better than random allocation under KL.On goal-oriented objectives, the KL design lies near the favorable edge of random-design performance but remains roughly one histogram-width from the corresponding goal-oriented optimum.Source: §5.4 · p. 22
  • Goal-oriented designs outperform random designs under their own objectives, and cross-evaluation shows that changing risk preference matters less here than choosing prediction-focused rather than parameter-focused objectives.The paper proposes comparing optimized objectives on a shared cached dataset to assess sensitivity to risk preference and parameter-versus-prediction mismatch without additional forward simulations.Source: §5.4 · p. 22

6 Conclusion

The paper introduces a risk-aware framework for goal-oriented Bayesian experimental design and demonstrates it on illustrative and advection-diffusion examples. Its results show that risk preferences can change sensor placement, while the estimator has limitations tied to its design encoding.

6 Conclusion

  • The framework composes risk across posterior deviation, prediction-domain aggregation, and aggregation over likely datasets.It accommodates choices including standard deviation, AVaR, entropic risk, and safety-margin design.Source: §6 · p. 23
  • Risk-averse and risk-neutral objectives place their primary measurement at physically distinct locations in the advection-diffusion example.The paper also compares performance across six goal-oriented objectives and a parameter-oriented objective.Source: §6 · p. 23
  • The continuous-simplex design encoding uses a diagonal noise-weighting matrix, relying on a Gaussian observation model.At extremely large sensor noise, the parameterization scales poorly beyond approximately 100 candidate sensor locations.Source: §6 · pp. 23, 25
  • Future work is proposed on sequential OED and replacing importance sampling with amortized generative models to reduce computational cost.These extensions are intended to broaden the estimator's reach to larger candidate sets and non-Gaussian likelihoods.Source: §6 · p. 25

A Linear Gaussian Model: Posterior and Push-Forward Statistics

A.1 Model specification

The appendix specifies a linear-Gaussian observation model in which design weights set the effective noise at candidate locations. Its posterior is Gaussian, and its covariance depends on the design but not on the observed data.

A.1 Model specification

  • The model has a Gaussian prior and linear observations, with design weights scaling precision through a diagonal effective noise covariance.A larger weight at a candidate location reduces its effective noise.Source: §A.1 · p. 27
  • The observation matrix is evaluated at candidate locations, and the polynomial-basis experiments use degree 4, 10 locations, common noise 0.5, and an isotropic prior.Source: §A.1 · p. 27

A.2 Posterior distribution

  • For the linear-Gaussian model, the parameter posterior is Gaussian and its covariance depends on design w, not observed data y.Source: §A.2 · p. 27

A.3 Distribution of the posterior mean over data

The appendix characterizes how the posterior mean varies across datasets and how posterior uncertainty maps to scalar lognormal quantities of interest. It uses these results to derive an expected-standard-deviation design objective.

A.3 Distribution of the posterior mean over data

  • The posterior mean is unbiased for the prior mean in expectation over data.The appendix also derives its covariance using the marginal data distribution.Source: §A.3 · p. 28

A.4 Lognormal push-forward

  • Each quantity-of-interest component follows a lognormal distribution, with its conditional standard deviation depending on design through the posterior covariance.The log-mean is Gaussian over data.Source: §A.4 · p. 28

B Analytical Design-Objective Expressions: Scalar QoI

B.1 U1: Expected standard deviation

  • The expected-standard-deviation objective U1 is derived using the mean of a lognormal random variable.Source: §B.1 · p. 28

B.2 U3: AVaRβ of the standard deviation over data

The appendix derives scalar-QoI objective expressions for AVaR, a safety margin, and the expected-information-gain baseline. The baseline is based on information about the full parameter posterior and has a closed form for the linear-Gaussian model.

B.2 U3: AVaRβ of the standard deviation over data

  • The AVaR objective for standard deviation over data uses the lognormal quantile and conditional tail expectation.Source: §B.2 · p. 29

B.3 U4: Safety margin

  • The safety-margin objective U4 combines expected deviation d with c times its standard deviation over data.Source: §B.3 · p. 29

B.4 EIG baseline

  • The EIG baseline measures information about the full parameter posterior, not its projection onto the prediction direction, and is data-independent in the linear-Gaussian model.Its closed form uses the design-weighted Fisher information.Source: §B.4 · p. 29

C Convergence of the Design-Objective Estimator

The nested-quadrature estimator converges to exact linear-Gaussian references, with different loops limiting prediction-objective and KL accuracy. The appendices derive gradients through design-dependent observations, normalized importance weights, and the three-level objective.

C Convergence of the Design-Objective Estimator

  • The convergence test uses a scalar QoI at xpred = 0 in a linear-Gaussian model, comparing U1, U3, U6, and KL estimates with analytical values.It evaluates uniform design weights and uses R = 1000 independent data realizations.Source: C Convergence of the Design-Objective… · p. 30
  • For prediction objectives, inner-loop variance limits accuracy; for KL, outer-loop variance sets the floor after N ≳200, while QMC lowers outer variance versus MC at matched counts.The sweeps show convergence to analytical references; refining the inner loop helps prediction objectives, while refining the outer loop helps KL.Source: C Convergence of the Design-Objective… · p. 30, Figure 13 · p. 31

D.1 Reparameterized observations and log-likelihood

  • Fixed latent-noise draws make artificial observations differentiable in design weights; reducing a sensor weight toward zero inflates its effective noise toward infinity.The observation reparameterization separates across sensors.Source: §D.1 · p. 31

D.2 Evidence and normalized importance weights

  • Self-normalized importance weights represent each dataset’s posterior, and their quotient-rule gradients include a weighted mean log-likelihood gradient to account for normalization.The inner samples are reweighted for each outer dataset rather than resampled from scratch.Source: §D.2 · p. 32

D.3 Chain rule through the three levels

  • The objective gradient applies the chain rule through deviation, QoI risk, and data risk; its required quantities are already computed in the forward pass.The gradient costs a constant multiple of a function evaluation.Source: §D.3 · pp. 32–33

D.4 Smoothed AVaR evaluation and gradient

The AVaR gradient procedure smooths the nondifferentiable risk functional using a regularized dual representation. The appendix also specializes gradient formulas to the Std/Mean/Mean objective U1.

D.4 Smoothed AVaR evaluation and gradient

  • Exact AVaR is nondifferentiable at the positive-part function, so the method regularizes its dual representation with a Moreau-Yosida approximation whose error is O(δ^-1).The smoothed AVaR converges to exact AVaR as δ →∞.Source: §D.4 · p. 33
  • The smoothed AVaR maximizer is computed by weighted projection and clipping; design-dependent importance weights require reverse-mode automatic differentiation through the smoothed projection.No closed-form gradient is available in that case because the binary-search active set is discontinuous.Source: §D.4 · p. 34

D.5 Specialization: Std/Mean/Mean (design objective U1)

  • For scalar QoI objective U1, the specialization uses standard deviation, mean QoI aggregation, and mean data aggregation; other deviation gradients follow the same structure.The other deviation measures replace the standard deviation with their corresponding functional of importance-weighted moments.Source: §D.5 · pp. 34–35

E Advection–Diffusion Experiment: Regime Replications

As diffusion increases, goal-oriented and KL sensor placements become less separated, and the penalty for using a mismatched design shrinks. Integer allocations preserve the continuous optimum’s weight hierarchy, concentrating first on dominant sensors.

E Advection–Diffusion Experiment: Regime Replications

  • Bootstrap boxes remain narrow relative to inter-objective gaps, while goal-oriented objectives disagree on primary and secondary sensor selections.The spread at the dominant sensor approaches 0.4 in the advection-dominated regime, is closer to 0.2 in the moderate regime, and falls below 0.1 across shared sensors in the diffusion-dominated regime.Source: E Advection–Diffusion Experiment:… · p. 35
  • As diffusion increases, the primary goal-oriented sensor shifts upstream and KL’s dominant sensors increasingly overlap with goal-oriented selections.The dominant sensor groups are disjoint in the advection-dominated regime, partially overlapping in the moderate regime, and aligned on the primary selection in the diffusion-dominated regime.Source: E Advection–Diffusion Experiment:… · p. 35
  • The cross-evaluation penalty for mismatched designs shrinks from the advection-dominated to the moderate and diffusion-dominated regimes.In the diffusion-dominated regime, the KL and six goal-oriented optima sit within approximately one histogram-width of their matched optima under any design objective.Source: E Advection–Diffusion Experiment:… · pp. 35–36
  • For U3 in the advection-dominated regime, the rounded allocation shifts from concentrated to more spatially diverse as the measurement budget grows from S = 5 to 20.Hamilton’s largest-remainder rule rounds continuous weights; dominant sensors receive added measurements before secondary sensors activate.Source: Figure 14 · p. 36, E Advection–Diffusion Experiment:… · p. 36

F Continuity of the Smoothed AVaR Value Function

The optimal smoothed AVaR value is continuous and increasing in confidence parameter α on [0, 1). The proof establishes continuity of an auxiliary value function and composes it with a continuous map.

F Continuity of the Smoothed AVaR Value Function

Loading 2609.08023v1…