Source-linked AI summary
Modern Monte Carlo Methods for Efficient Uncertainty Quantification and Propagation: A Survey
Jiaxin Zhang
TL;DR
Standard Monte Carlo is often time-consuming for uncertainty quantification, motivating an overview of efficient modern Monte Carlo methods. These methods address UQ challenges beyond forward problems, including uncertainty propagation and settings with limited data.
Problem
Standard Monte Carlo is often very time-consuming for uncertainty quantification and propagation.
Method
The article surveys several efficient Monte Carlo methods, including control-variate approaches that combine different model fidelities and sampling reweighting schemes.
Results
The surveyed methods address many UQ challenges, including more general problems beyond forward UQ.
Takeaways & Limitations
These modern Monte Carlo methods can be employed for uncertainty quantification and propagation, including challenges involving limited data.
Takeaways & Limitations
Related stochastic methods such as collocation and stochastic Galerkin suffer from the curse of dimensionality.
Abstract
from arXiv · showhide
Uncertainty quantification (UQ) includes the characterization, integration, and propagation of uncertainties that result from stochastic variations and a lack of knowledge or data in the natural world. Monte Carlo (MC) method is a sampling-based approach that has widely used for quantification and propagation of uncertainties. However, the standard MC method is often time-consuming if the simulation-based model is computationally intensive. This article gives an overview of modern MC methods to address the existing challenges of the standard MC in the context of UQ. Specifically, multilevel Monte Carlo (MLMC) extending the concept of control variates achieves a significant reduction of the computational cost by performing most evaluations with low accuracy and corresponding low cost, and relatively few evaluations at high accuracy and correspondingly high cost. Multifidelity Monte Carlo (MFMC) accelerates the convergence of standard Monte Carlo by generalizing the control variates with different models having varying fidelities and varying computational costs. Multimodel Monte Carlo method (MMMC), having a different setting of MLMC and MFMC, aims to address the issue of uncertainty quantification and propagation when data for characterizing probability distributions are limited. Multimodel inference combined with importance sampling is proposed for quantifying and efficiently propagating the uncertainties resulting from small datasets. All of these three modern MC methods achieve a significant improvement of computational efficiency for probabilistic UQ, particularly uncertainty propagation. An algorithm summary and the corresponding code implementation are provided for each of the modern Monte Carlo methods. The extension and application of these methods are discussed in detail.
INTRODUCTION
UQ characterizes how variability, randomness, and limited knowledge affect response outputs, while standard Monte Carlo methods can become costly for computationally intensive models. The paper reviews modern Monte Carlo methods that reduce computational cost or improve estimator convergence for probabilistic UQ and uncertainty propagation.
- UQ context: Uncertainties are commonly categorized as aleatory variability or epistemic lack of knowledge, although classifying a particular uncertainty can be challenging.Many engineering and science problems involve both types of uncertainty.
- UQ context: UQ quantifies the effects of variability, randomness, and limited knowledge on quantities of interest across science and engineering applications.Probabilistic UQ converts sources such as input, boundary, physical-model, and parameter uncertainty into random variables or fields.
- Alternative UQ methods: Non-sampling methods can be accurate but intrusive, whereas stochastic collocation is non-intrusive yet shares the curse of dimensionality with stochastic Galerkin methods.Stochastic Galerkin uses polynomial expansions; stochastic collocation samples at selected stochastic-space points and interpolates the results.
- Modern Monte Carlo methods: MLMC reduces cost by allocating most simulations to low-accuracy, low-cost levels and few evaluations to high-accuracy, high-cost levels.Its allocation uses control variates and distributes the computational budget according to variance-reduction needs across hierarchical levels.
- Modern Monte Carlo methods: MFMC combines cheap low-fidelity and expensive high-fidelity outputs for significant time-cost reduction and unbiased high-fidelity statistical estimators.MMMC instead combines multimodel inference with importance sampling to propagate uncertainties when data for probability characterization are limited.
STANDARD MONTE CARLO METHOD FOR UQ
Standard Monte Carlo propagates input uncertainty through repeated model evaluations and statistically summarizes the outputs. It is unbiased and converges with more samples, but can become impractical for complex high-fidelity simulations because accurate estimates require extensive evaluations.
- Forward and inverse UQ: Forward UQ propagates uncertainty in input random variable X through model M to estimate output statistics such as pY(y) and E[Y].
- Sampling and propagation: Standard MC samples inputs from pX(x), evaluates the computational model for each sample, and combines outputs to estimate statistics of Y.
- Statistical properties: The MC estimator is unbiased, and its average converges almost surely to the expected value as the sample count increases.
- Statistical properties: RMSE = O(n^-1/2) when the individual-term variance is bounded, while confidence intervals can be estimated from the sampling distribution.
- Strengths and challenges: Standard MC is non-intrusive because an available computational model can be rerun with independently drawn inputs, but complex high-fidelity simulations make the method time-consuming.
- Extensions: Variance reduction, quasi-Monte Carlo, importance sampling, and MCMC are among approaches proposed to reduce cost or address difficult sampling distributions.
MODERN MONTE CARLO METHODS FOR UQ
The paper reviews three modern Monte Carlo methods for uncertainty quantification and propagation. It covers each method's procedure, generalizations, extensions, and applications.
- The review focuses on multilevel, multifidelity, and multimodel Monte Carlo methods.
- For each method, the paper describes the algorithm procedure, generalizations, extensions, and applications to uncertainty quantification.
Multilevel Monte Carlo Methods
Multilevel Monte Carlo (MLMC) extends control variates across hierarchical simulation levels, reallocating samples toward inexpensive levels while retaining high-accuracy corrections. This reduces computational cost for uncertainty propagation and supports several extensions and applications.
- Core idea: MLMC uses control variates across hierarchical simulation levels to reduce computational cost.Most simulations use low-fidelity, low-cost levels, while relatively few high-fidelity simulations provide accurate corrections.
- Core idea: Samples are allocated across levels according to each level’s variance reduction and simulation cost.The allocation is selected to minimize estimator variance under a fixed computational budget.
- Estimator construction: The multilevel estimator is constructed from corrections between successive model levels and remains unbiased for the finest-level expectation.Independent samples are used at each correction level.
- Computational efficiency: MLMC cost can be lower than standard MC cost by factors determined by level variances and costs.The cited comparison identifies reductions involving V(L)/V(0) and C(0)/C(L), depending on how these quantities change with level.
- Extensions and applications: Extensions include randomized MLMC, multi-index MC, multilevel QMC, multilevel MCMC, and combinations with data assimilation.Applications discussed include forward and inverse UQ, failure probabilities, aerodynamics, and uncertain systems.
Multifidelity Monte Carlo Methods
Multifidelity Monte Carlo (MFMC) combines high- and low-fidelity model outputs as control variates to estimate response statistics under computational constraints. Its efficiency depends on model costs and correlations, while estimated variance and correlation inputs add expense and can increase estimator variance.
- Method: MFMC combines high-fidelity outputs with many low-fidelity model outputs to accelerate statistical estimation.It retains high-fidelity evaluations to establish convergence and accuracy guarantees rather than replacing the high-fidelity model entirely.
- Estimator properties: The multifidelity estimator is unbiased for the high-fidelity model’s expectation.Low-fidelity models are constructed as auxiliary random variables and used through control variate coefficients.
- Optimization: Optimal control variate coefficients and model evaluation counts are obtained by minimizing estimator variance under a computational budget.The optimization uses model costs, variances, correlations, and the constraints 0 < n0 ≤ n1 ≤ ··· ≤ nk.
- Computational efficiency: χ < 1 indicates that MFMC is more computationally efficient than standard Monte Carlo using only the high-fidelity model.The ratio reflects both model costs and correlations.
- Caveat: Estimating variances and correlations from model samples adds computational expense and can increase the multifidelity estimator’s variance.These estimation costs must be included in the total MFMC budget.
Multimodel Monte Carlo Methods
Multimodel Monte Carlo (MMMC) addresses uncertainty quantification with limited data by retaining an ensemble of candidate probability distributions and their parameters. Combined with importance sampling, it propagates epistemic uncertainty through a computational model using a single Monte Carlo loop.
- Method: MMMC uses multimodel inference to assign candidate probability distributions associated model probabilities to small datasets.The procedure draws on information-theoretic and Bayesian perspectives.
- Uncertainty representation: The methodology identifies both model-form uncertainty and model-parameter uncertainty.Unlike single-model selection, it retains uncertainty from all candidate distributions and their parameters during propagation.
- Propagation: Importance sampling identifies a surrogate density, propagates it through the computational model, and reweights samples from candidate distributions.This produces a probabilistic description of the response output.
- Results: The proposed method provides a complete probabilistic description of epistemic uncertainty with several orders of magnitude reduction in Monte Carlo-based computational costs.It collapses multiple Monte Carlo loops into a single loop on an optimized surrogate distribution while propagating the full set of probability distributions.
- Scope: MMMC targets input uncertainty propagated through a deterministic computational model; incorporating computational-model stochasticity is beyond this article’s scope.The paper distinguishes this setting from the MLMC and MFMC methods discussed earlier.
- Motivation: Sparse datasets make it difficult to identify a unique probability distribution without significant assumptions.The paper motivates comparing multiple candidate distributions when input data are very limited.
CONCLUSIONS
The survey presents modern Monte Carlo methods that reduce computational costs for uncertainty quantification by exploiting low-cost simulations, multiple fidelities, and multimodel inference. These methods address standard Monte Carlo's time demands and support broader UQ applications, including uncertainty propagation.
- Motivation: Standard Monte Carlo is time-consuming because estimating response statistics requires many random-sample model evaluations, while high-fidelity models are computationally expensive.The computational burden motivates more efficient methods for complex mathematical, physical, and engineering systems.
- Modern Monte Carlo methods: MLMC reduces computational cost by using control variates and allocating most simulations to low-accuracy, low-cost levels.Only relatively few simulations are performed at high cost and accuracy.
- Modern Monte Carlo methods: MFMC combines high-fidelity and low-fidelity models through control variates to speed statistical estimation.The models differ in fidelity and computational cost, enabling a cost-conscious estimation strategy.
- Modern Monte Carlo methods: MMMC quantifies uncertainty under imprecise probabilities caused by limited data using multimodel inference that combines model-form and model-parameter uncertainties.It then propagates an ensemble of probability models through an optimal importance-sampling reweighting scheme.
- Applications and scope: These modern Monte Carlo methods improve computational efficiency for probabilistic UQ and address forward UQ as well as optimization and robust design under uncertainty.The survey also identifies applications involving uncertainty in artificial intelligence and machine learning.