Source-linked AI summary
An efficient methodology for modeling complex computer codes with Gaussian processes
Amandine Marrel, Bertrand Iooss, Francois Van Dorpe, Elena Volkova
TL;DR
The paper addresses the need to replace computationally expensive computer codes for uncertainty propagation and global sensitivity analysis. It develops a Gaussian-process estimation methodology for complex data and applies it to analytical and hydrogeological cases. Results on the MARTHE code are described as very encouraging, positioning Gaussian processes as an alternative to efficient but non-explicit methods.
Problem
Complex computer codes are too time expensive for direct uncertainty propagation and global sensitivity studies, while fitting Gaussian processes is difficult with many inputs and inadequate sampling designs.
Method
The paper develops a Gaussian-process modeling methodology using progressive input-parameter selection and covariance-function parameter estimation, including adaptation to previously collected non-optimal data.
Results
Results on the MARTHE computer code are described as very encouraging, placing Gaussian processes as an alternative to efficient but non-explicit methods such as boosting trees or neural networks.
Takeaways & Limitations
The methodology supports implementing and adapting Gaussian-process models to complex data while studying their prediction capabilities.
Takeaways & Limitations
The covariance function requires estimating 2d correlation parameters, and the paper reports instability and a high number of parameters in estimation procedures.
Abstract
from arXiv · showhide
Complex computer codes are often too time expensive to be directly used to perform uncertainty propagation studies, global sensitivity analysis or to solve optimization problems. A well known and widely used method to circumvent this inconvenience consists in replacing the complex computer code by a reduced model, called a metamodel, or a response surface that represents the computer code and requires acceptable calculation time. One particular class of metamodels is studied: the Gaussian process model that is characterized by its mean and covariance functions. A specific estimation procedure is developed to adjust a Gaussian process model in complex cases (non linear relations, highly dispersed or discontinuous output, high dimensional input, inadequate sampling designs, ...). The efficiency of this algorithm is compared to the efficiency of other existing algorithms on an analytical test case. The proposed methodology is also illustrated for the case of a complex hydrogeological computer code, simulating radionuclide transport in groundwater.
1 INTRODUCTION
Complex computer codes are often too costly for uncertainty and sensitivity studies, motivating reduced metamodels. The paper develops and evaluates a Gaussian-process methodology for fitting complex data and studying prediction capabilities.
- Motivation: Complex computer codes are often too time expensive for uncertainty propagation and global sensitivity analysis.Reduced models can make these studies computationally feasible.
- Metamodels: Metamodels replace costly computer codes with mathematical approximations that require acceptable calculation time.Response surfaces, surrogate models, and metamodels construct functions that simulate behavior across influential-parameter ranges.
- Motivation: An analytic predictor formula is useful because it enables direct calculation of sensitivity indices and output uncertainties while supporting interpretation of parameter interactions.The introduction contrasts interpretable but potentially inefficient polynomials with more efficient models that lack analytic predictor formulas.
- Gaussian-process modeling: Gaussian-process models, characterized by mean and covariance functions, provide interpolation and interpretability advantages for computer experiments.They extend kriging by modeling correlations between code responses according to distances between input variables.
- Methodological challenge: Estimating covariance hyperparameters becomes particularly difficult with many inputs and inadequate sampling designs.The paper addresses nonlinear relations, highly dispersed or discontinuous outputs, high-dimensional inputs, and non-optimal existing data.
2 GAUSSIAN PROCESS MODELING
The paper models computer-code outputs with Gaussian processes combining a linear mean function and a stationary stochastic component. Its formulation supports prediction uncertainty and parameter estimation, while the proposed sequential strategy targets high-dimensional, irregular optimization settings.
- Theoretical model: Gaussian-process modeling represents deterministic code outputs as realizations of a random function with regression and centered stochastic components.The deterministic function supplies the mean approximation, while the stochastic process captures correlated variation.
- Theoretical model: The regression component uses a one-degree polynomial basis to add an external drift and permit a nonstationary global model.The study restricts the mean function to a linear parametric form, while allowing extensions to other regression bases.
- Theoretical model: The generalized exponential correlation function is selected because its parameters generate a wide spectrum of covariance shapes, including exponential and Gaussian cases.The correlation is constructed from one-dimensional components, with p = 1 yielding exponential correlation and p = 2 Gaussian correlation.
- Joint and conditional distributions: Conditioning on training data produces a Gaussian predictive distribution whose conditional mean is the predictor and whose variance is a local MSE indicator.The predictive distribution also supports uncertainty and sensitivity analysis, quantile evaluation, and sampling strategies.
- Parameter estimation: Parameter estimation minimizes a likelihood-based objective over correlation and noise parameters, but the optimization becomes costly and difficult in high dimensions with irregular sampling.Many parameters, broad domains without prior bounds, and local minima motivate sequential and stochastic optimization strategies.
- Parameter estimation: The proposed methodology progressively selects input variables in both the regression and covariance components to address limitations of existing procedures for complex applications.The paper specifically contrasts high-dimensional settings with methods requiring starting points, bounds, or direct use of all input variables.
3 MODELING METHODOLOGY
The methodology builds Gaussian-process metamodels through standardized inputs, ranked variable selection, sequential parameter estimation, and model validation. It combines cross-validation, Q2, residual diagnostics, and AICC to select predictive regression and covariance structures.
- Step 0 - Standardization of input variables: Uniform input transformation appears well adapted to estimating correlation parameters and simplifies parameter bounds and starting points.The transformation uses theoretical or empirical distributions with piecewise linear approximation to obtain U[0, 1] variables.
- Step 1 - Initial input variables ranking: Inputs are initially ranked by their correlation with the output to reduce possible models and provide efficient starting information for parameter estimation.Correlation coefficients provide a simple influence hierarchy, especially as an initial guide when no other information is available; nonlinear cases may require qualitative or sensitivity-based alternatives.
- Step 3 - Successive inclusion of input variables: Correlation parameters are estimated by progressively adding covariance inputs, reusing estimates from the preceding step as starting values.At step i, inputs 1 through i enter the covariance function and parameter pairs (θ_l, p_l) are estimated.
- Step 3.3: Optimal regression model selection: AICC selects the regression model after parameter estimation, with the required model minimizing the criterion according to regression and covariance model sizes and sample log-likelihood.The criterion accounts for m1 regression inputs, m2 covariance inputs, and lY, the sample log-likelihood.
- Step 3.4: Q2 evaluation: Q2 is relatively fast to evaluate while producing a final model that optimizes predictive performance.Validation may use K-fold cross-validation or a test sample, alongside absolute error, relative-residual summaries, and graphical diagnostics.
4 APPLICATIONS
The methodology is evaluated on the Sobol g-function and applied to MARTHE simulations of 90Sr transport. It performs especially well for high-dimensional inputs and complex hydrogeological outputs, although its sequential estimation is computationally costly.
- 4.1 Analytical test case: For d > 6, the methodology with double input selection clearly outperforms the other implementations and provides more robust prediction accuracy.The improvement is attributed to rerunning estimation after sorting inputs by decreasing ∆Q2.
- 4.1 Analytical test case: The methodology is considered interesting for high-dimensional models, inadequate or small sampling designs, and cases where simpler methods have failed.The authors frame its use as a compromise between sampling-design cost and metamodel-construction time.
- 4.2.4 Analysis: For output 16, all but one observed value lies inside the Gaussian-process 95% prediction interval, confirming good model adequacy for that output.The interval is based on the MSE local estimator.
- 4.2.4 Analysis: The application results confirm the potential of the Gaussian-process model and support applying the methodology to complex computer-code data.The input-selection procedure also selects the best input set in regression for a fixed covariance-input set, but sequential ordering and rerunning are necessary for good Q2 results.
5 CONCLUSION
The paper presents Gaussian processes as a practical alternative for complex computer codes, while addressing parameter-estimation difficulties and high-dimensional settings. Its methodology is promising for nonlinear, thresholded, and small-sample problems, though sequential input-design selection remains an open area.
- Parameter estimation can suffer from instability and a high number of parameters, making robust implementation methodology necessary for complex industrial problems.
- The proposed estimation and selection procedures address covariance-parameter complexity, including 2d correlation parameters for the generalized exponential covariance function.
- The methodology is intended for complex models with nonlinearities or threshold effects, high-dimensional inputs (d > 10), and small samples.
- Results on the MARTHE computer code are described as very encouraging, placing Gaussian processes as an alternative to boosting trees and neural networks.
- Gaussian processes offer exact interpolation, explicit prediction formulas, and statistical tools including mean squared error for assessing local accuracy.
- Sequential selection of the input design remains an active research domain and is identified as a possible improvement to the construction algorithm.
ACKNOWLEGMENTS
The authors acknowledge the MRIMP project and CEA divisions for support, and thank two referees for comments that significantly contributed to the work.
- The work was supported by the MRIMP project of the “Risk Control Domain”.
- The project is managed by CEA/Nuclear Energy Division/Nuclear Development and Innovation Division.
- The authors thank two referees whose comments significantly contributed to the work.