Source-linked AI summary
Global Sensitivity Analysis of Stochastic Computer Models with joint metamodels
Bertrand Iooss, Mathieu Ribatet, Amandine Marrel
TL;DR
Global sensitivity analysis is established for deterministic computer codes, but stochastic codes can vary even under the same controllable inputs. This paper uses joint mean-dispersion modeling, including nonparametric GAM and Gaussian-process approaches, to estimate sensitivity indices. The reported results show accurate estimates in heteroscedastic cases, with limits on resolving individual contributions from uncontrollable variation.
Problem
Classical sensitivity methods and metamodels address deterministic codes, whereas stochastic codes can vary for the same controllable inputs and may exhibit input-dependent dispersion.
Source: §1 · pp. 1–2Method
The paper models mean and dispersion jointly, developing nonparametric GAMs and describing a joint Gaussian-process approach for complex stochastic-code outputs.
Source: §2.2 · p. 4, §2.3 · p. 4, §5 · pp. 13–14Results
In the industrial application, modeling dispersion provides a robust estimate of the uncontrollable-variable total sensitivity index and correct estimates of controllable-variable first-order indices.
Source: §5 · p. 14Takeaways & Limitations
Joint GAM and Gaussian-process models are adapted to complex heteroscedastic cases where classical metamodels are inadequate, and provide a basis for computing Sobol indices.
Source: §5 · p. 14The separate sensitivity contributions involving uncontrollable variation cannot be quantitatively distinguished, and the proposed total-index calculation from metamodel predictivity relies on a strong fit assumption.
Source: §3.2 · p. 7, §4.1.2 · p. 10
Abstract
from arXiv · showhide
The global sensitivity analysis method, used to quantify the influence of uncertain input variables on the response variability of a numerical model, is applicable to deterministic computer code (for which the same set of input variables gives always the same output value). This paper proposes a global sensitivity analysis methodology for stochastic computer code (having a variability induced by some uncontrollable variables). The framework of the joint modeling of the mean and dispersion of heteroscedastic data is used. To deal with the complexity of computer experiment outputs, non parametric joint models (based on Generalized Additive Models and Gaussian processes) are discussed. The relevance of these new models is analyzed in terms of the obtained variance-based sensitivity indices with two case studies. Results show that the joint modeling approach leads accurate sensitivity index estimations even when clear heteroscedasticity is present.
1 Introduction
Global sensitivity analysis apportions output uncertainty to uncertain inputs, but established methods and metamodels chiefly address deterministic computer codes. This paper extends the joint modeling approach to stochastic codes, using nonparametric models to handle complex outputs and examining their sensitivity-index estimates.
1 Introduction
- Global sensitivity analysis apportions output uncertainty to input uncertainty across input ranges and supports calibration, validation, and decision making.Probabilistic and Monte Carlo approaches can require many simulations.Source: §1 · p. 1
- Deterministic-code metamodels approximate costly simulations while aiming for good predictive capability and negligible computation time.Common examples include polynomials, splines, neural networks, and Gaussian processes.Source: §1 · pp. 1–2
- Stochastic codes can produce different outputs for the same controllable inputs because of uncontrollable variables, including complex random fields.These variables may be known to exist but be unobservable, inaccessible, or too complex to describe with a reasonable number of scalar parameters.Source: §1 · p. 2
- Classical metamodels for deterministic codes are not pertinent to stochastic codes, while a Gaussian-process nugget assumes input-independent error.That homoscedasticity assumption limits its use, although prior work has demonstrated Gaussian-process usefulness in heteroscedastic cases.Source: §1 · p. 2
- The paper extends stochastic-code sensitivity analysis with nonparametric joint models because parametric GLMs can be limited for complex outputs.It considers GAMs and notes that Gaussian processes or other nonparametric models may also be relevant.Source: §1 · p. 2
- Joint modeling of the mean and dispersion has been used for stochastic-code sensitivity analysis, including computation of a total index for uncontrollable variables.The paper compares proposed approaches on an analytic function and applies them to groundwater radionuclide migration modeling.Source: §1 · p. 2
2 Joint modeling of mean and dispersion
The paper builds joint models of response mean and dispersion, first in a GLM framework and then with more flexible GAMs. It also describes a joint Gaussian-process alternative for heteroscedastic computer experiments.
2.1 Using the Generalized Linear Models
- Joint GLMs model the response mean and dispersion as linked components, with the dispersion allowed to vary with explanatory variables.The model is fitted iteratively: estimate the mean, fit dispersion from a dispersion statistic, then update mean weights using the estimated dispersion.Source: §2.1 · p. 3
2.2 Extension to the Generalized Additive Models
- Joint GAMs replace linear predictor terms with smooth functions, enabling flexible nonlinear effects and multivariate smooths for input interactions.They maximize penalized extended quasi-likelihood; smoothing parameters are jointly updated during fitting, with GCV scores evaluated at convergence.Source: §2.2 · pp. 3–4
- The joint GAM extends MADAM beyond Gaussian observations to quasi-distributions and allows multivariate smoothers such as thin plate regression splines.The paper also notes that other model combinations, including Gaussian processes, can be used within the joint-model framework.Source: §2.2 · p. 4
2.3 Joint modeling with other models
- Joint modeling is not the only option: a heteroscedastic Gaussian process can use input-varying nugget estimates and its MSE as a dispersion statistic.The paper focuses instead on joint Gaussian processes to remain coherent with its other joint models.Source: §2.3 · p. 4
- The joint Gaussian-process approach fits a mean metamodel with a nugget, then fits a second Gaussian process to squared residuals.The fitting can be iterated; residuals may also be derived by cross-validation.Source: §2.3 · pp. 4–5
3 Global sensitivity analysis
The paper reviews variance-based Sobol indices for deterministic models and extends the decomposition to stochastic outputs with uncontrollable variation. Joint mean-dispersion metamodels support index estimation, but the separate contributions of uncontrollable-variable interactions cannot be quantitatively resolved.
3.1 Deterministic models
- Sobol indices decompose output variance into input and interaction contributions, with larger indices indicating greater importance.The total sensitivity index aggregates all terms containing an input, avoiding the intractable count of 2^p − 1 separate indices as p grows.Source: §3.1 · p. 5
- For deterministic models, sensitivity indices can be estimated by Monte Carlo on metamodels that approximate costly computer codes.The input vector is treated as independent random variables with known distributions.Source: §3.1 · p. 5
3.2 Stochastic models
- For stochastic models, output variance combines variation in the mean response over controllable inputs and expected conditional dispersion.The joint model's mean component supports Sobol indices for controllable inputs, while its dispersion component supports the total sensitivity index of uncontrollable variation.Source: §3.2 · p. 6
- The method cannot quantitatively distinguish the separate sensitivity contributions involving the uncontrollable variable.The authors leave finding composite indices as an open problem, while suggesting dispersion-model terms and t-values for qualitative information.Source: §3.2 · p. 7
- The method is useful when stochastic codes need metamodels, uncontrollable variables interact with controllable inputs, and those interactions matter.The framework treats uncontrollable variation as a stochastic input ε and controllable inputs X as independent random variables.Source: §3.2 · p. 7
4 Applications
The applications test joint metamodels on the Ishigami function and a radioactive-pollutant transport model, showing their usefulness for estimating sensitivity indices when outputs are heteroscedastic. On Ishigami, the joint GAM and Gp approximate the indices well; for MARTHE, the authors judge the joint GAM more reliable than the joint GLM.
4.1 An analytic test case: the Ishigami function
- The Ishigami test treats X1 and X2 as controllable inputs and X3 as inaccessible, uncontrollable noise, while retaining the standard case’s theoretical sensitivity indices.The fitting sample contains 1,000 observations without replicates in the controllable-input plane.Source: §4.1 · p. 7, §4.1.1 · p. 7
4.1.1 Metamodeling
- For Ishigami, the joint GAM explains 93% of mean-component deviance, versus 77% for simple GAM and 61% for joint GLM, and identifies X1 as explanatory for dispersion.The joint GLM’s dispersion component is constant because no explanatory variable was found; the joint Gp estimates dispersion sensitivity as SYd(X1) = 0.996 and SYd(X2) = 0.001.Source: §4.1.1 · pp. 7–8
4.1.2 Sobol indices
- On Ishigami, joint GAM and joint Gp approximate all Sobol indices well, with their largest ST3 errors at 7% and 16%; the joint GLM’s errors exceed 30%.Some indices remain unknown when dispersion-component effects are non-separable, although the joint-model formulas correctly deduce ST2 = S2 and S12 = S23 = S123 = 0.Source: §4.1.2 · p. 10, Table 2 · p. 11
4.1.3 Convergence study
- For Ishigami, a learning sample of 100 is sufficient for fitting the joint GAM and obtaining precise indices; Q2 is recommended over the dispersion component for estimating ST3.Q2 converges rapidly, while the dispersion-component mean converges more slowly.Source: §4.1.3 · p. 11
4.2 Application to an hydrogeologic transport code
- The MARTHE study analyzes 90Sr concentration with 20 uncertain scalar inputs and an uncontrollable spatial field, using metamodels because code runs are costly.Its 300-run design uses independent spatial-field realizations; alternative designs with replicates were outside the paper’s scope.Source: §4.2 · pp. 11–12
- In MARTHE, joint GAM fits the mean and dispersion competitively, yielding reliable confidence intervals, while simple GAM residuals retain heteroscedastic bias.The joint GAM’s mean deviance is 98.1%, with dispersion deviance of 29.7%.Source: §4.2 · p. 13
- In MARTHE, the joint GAM is judged more reliable than the joint GLM: kd2 explains more than 52% of output variance, per2 more than 5%, and ε’s total influence is 27.7% versus 38.2%.The authors report that the second hydrogeological layer’s geometry can influence predicted 90Sr concentration by up to 28%.Source: §4.2 · p. 13
5 Conclusion
The conclusion favors joint modeling of output mean and dispersion, with nonparametric models for complex stochastic codes. The industrial application supports robust sensitivity estimates and reveals influential interactions, though their quantitative values remain open.
5 Conclusion
- Rather than separately fitting components, the method jointly models code-output mean and dispersion, extending parametric joint models with GAMs and quasi distributions for complex nonlinear behavior.Source: §5 · pp. 13–14
- The joint GAM fits mean and dispersion components as well as the powerful Gaussian-process model, while flexibly fitting complex data.Source: §5 · p. 14
- On Ishigami, joint GAM and joint GP suit complex heteroscedastic cases where classical metamodels are inadequate, while joint GAM formulas aid sensitivity analysis and model understanding.The paper also describes a theoretical basis for efficient Sobol-index computation.Source: §5 · p. 14
- On MARTHE, modeling dispersion robustly estimates the uncontrollable variable’s total sensitivity index, improves first-order estimates for controllable inputs, and reveals influential interactions.The comparison is with other methods.Source: §5 · p. 14
- Quantifying interaction effects between the uncontrollable variable and other inputs remains an open issue.Source: §5 · p. 15
- Joint models may also support uncertainty-propagation studies seeking the full output distribution.Source: §5 · p. 15