Source-linked AI summary
Polynomial-Chaos-based Kriging
R. Schoebi, B. Sudret, J. Wiart
TL;DR
Expensive and uncertain computational models motivate surrogate methods for analyses requiring many evaluations, but PCE and Kriging had developed with limited interaction. The paper combines them in PC-Kriging, using sparse polynomial structure with Kriging, and finds it performs at least as well as the distinct approaches on benchmark functions, especially for small designs.
Problem
Expensive computational models make repeated evaluations for reliability and optimization difficult, while PCE and Kriging had been developed largely independently.
Method
PC-Kriging combines a sparse PCE representation of global behavior with Kriging for local variability, using LAR-based polynomial selection.
Results
PC-Kriging is better than, or at least as good as, ordinary Kriging and PCE for small experimental designs on benchmark analytical functions.
Takeaways & Limitations
OPC-Kriging reduces the regression polynomial count and metamodel complexity compared with SPC-Kriging, but requires higher calibration cost.
Takeaways & Limitations
The paper evaluates PC-Kriging only on benchmark analytical functions; applications to realistic engineering problems remain ongoing research.
Abstract
from arXiv · showhide
Computer simulation has become the standard tool in many engineering fields for designing and optimizing systems, as well as for assessing their reliability. To cope with demanding analysis such as optimization and reliability, surrogate models (a.k.a meta-models) have been increasingly investigated in the last decade. Polynomial Chaos Expansions (PCE) and Kriging are two popular non-intrusive meta-modelling techniques. PCE surrogates the computational model with a series of orthonormal polynomials in the input variables where polynomials are chosen in coherency with the probability distributions of those input variables. On the other hand, Kriging assumes that the computer model behaves as a realization of a Gaussian random process whose parameters are estimated from the available computer runs, i.e. input vectors and response values. These two techniques have been developed more or less in parallel so far with little interaction between the researchers in the two fields. In this paper, PC-Kriging is derived as a new non-intrusive meta-modeling approach combining PCE and Kriging. A sparse set of orthonormal polynomials (PCE) approximates the global behavior of the computational model whereas Kriging manages the local variability of the model output. An adaptive algorithm similar to the least angle regression algorithm determines the optimal sparse set of polynomials. PC-Kriging is validated on various benchmark analytical functions which are easy to sample for reference results. From the numerical investigations it is concluded that PC-Kriging performs better than or at least as good as the two distinct meta-modeling techniques. A larger gain in accuracy is obtained when the experimental design has a limited size, which is an asset when dealing with demanding computational models.
1 Introduction
Computer simulations can be expensive and uncertain, motivating non-intrusive surrogate models for analyses requiring many evaluations. This paper combines PCE and Kriging into PC-Kriging to improve flexible metamodeling accuracy.
- Motivation: High-fidelity computational models may require hours to days per run, making repeated evaluations costly across engineering disciplines.Finite element models are given as an example of such expensive simulations.
- Motivation: Input uncertainty can arise from natural variability or lack of knowledge, affecting model outputs and system performance.The paper distinguishes aleatory and epistemic uncertainty.
- Motivation: Surrogate models replace computational models with easy-to-evaluate functions, enabling reliability and optimization analyses with many evaluations.This addresses Monte Carlo procedures that may require thousands to millions of model runs.
- Existing approaches: PCE approximates outputs with orthogonal polynomials matched to input distributions, while Kriging models the computational response as a Gaussian process.Both are presented as non-intrusive metamodeling approaches built from model evaluations.
- Contribution: Because PCE and Kriging had been applied largely independently, the paper introduces PC-Kriging to combine their characteristics and advantages.The paper presents PC-Kriging as a new, more accurate and flexible metamodeling technique.
2 Polynomial Chaos Expansions
Polynomial Chaos Expansions approximate uncertain computational models with orthonormal polynomial series selected according to input distributions. Practical PCE construction requires truncating and sparsifying the expansion, calibrating coefficients non-intrusively, and controlling generalization error.
- PCE formulation: Multivariate basis functions are formed by tensor products of univariate polynomials, with each multi-index component specifying a polynomial degree.The univariate bases satisfy orthonormality under the corresponding marginal probability density.
- PCE formulation: PCE represents a computational model’s output as an infinite series of multivariate orthonormal polynomials matched to the input distribution.The polynomial coefficients multiply basis functions indexed by multi-indices; independent input variables yield a product joint density.
- Truncation schemes: Truncation replaces the infinite expansion with a finite multi-index set so the response can be approximated accurately under an error measure.The total-degree set includes indices satisfying |α| ≤ p, where p is the maximal polynomial degree.
- Truncation schemes: The total-degree set becomes impractical for highly nonlinear or high-dimensional models because its cardinality grows with both input dimension M and degree p.This computational difficulty is identified as the curse of dimensionality.
- Truncation schemes: Hyperbolic truncation reduces higher-order interaction terms using a q-norm, with decreasing q producing smaller sets and q → 0 yielding an additive model.For q = 1, the hyperbolic and total-degree index sets are equivalent.
- Coefficient estimation: Non-intrusive PCE estimates coefficients from repeated model evaluations on an experimental design, often using least-squares systems and sparse regression.The information matrix contains evaluated basis functions, while generalized regression methods can reduce the predictor set.
- Error assessment: Generalization error measures prediction deviation, whereas empirical error can underestimate it when the polynomial count approaches the sample count, causing overfitting.When N = |A|, predictors may interpolate the experimental-design points without achieving low true error.
3 Kriging
Kriging models computational responses as Gaussian-process realizations, combining a trend with an autocorrelation structure to predict outputs and quantify uncertainty. Its interpolating behavior motivates leave-one-out error estimation, while PCE appears as an uncorrelated special case.
- Model definition: Kriging treats the computational-model response as a realization of a Gaussian random process.The model is also known as Gaussian process modeling.
- Model definition: The Gaussian process combines a trend with variance and a stationary autocorrelation function parameterized by hyper-parameters.The trend coefficients and correlation parameters are calibrated from experimental-design responses.
- Calibration: Maximum likelihood is preferable to cross-validation for well-specified autocorrelation families, whereas cross-validation is more robust when the family is uncertain.The latter setting corresponds to practical black-box modeling.
- Prediction: Kriging predicts a Gaussian random variable at new inputs; its mean serves as the surrogate and its variance indicates local precision.Prediction uses correlations between the new input and experimental-design samples.
- Error estimation: Because interpolation makes empirical error vanish without a nugget effect, Kriging uses leave-one-out error for global accuracy assessment.An analytical universal-Kriging solution avoids explicitly computing all N leave-one-out models.
- PCE connection: PCE is a special case of universal Kriging when experimental-design samples are uncorrelated and the correlation matrix equals the identity matrix.Under this setting, Kriging equations and leave-one-out error reduce to their PCE counterparts.
4 PC-Kriging
PC-Kriging combines PCE’s global polynomial trend with Kriging’s local Gaussian-process variability. Sparse polynomial selection and Kriging calibration yield sequential and iterative formulations evaluated through leave-one-out error.
- 4.1 Principle: PC-Kriging uses orthonormal polynomials to capture global behavior and a Gaussian process to capture local output variability.It is formulated as universal Kriging with a non-standard trend.
- 4.1 Principle: Building PC-Kriging requires selecting a sparse polynomial set and calibrating correlation hyper-parameters and Kriging parameters.Least-angle regression with hyperbolic index sets determines the sparse polynomial trend.
- 4.2 Algorithm: SPC-Kriging determines the polynomial trend with PCE first, then embeds that trend into a universal Kriging model.Its underlying assumption is that the PCE-selected polynomial set is also suitable for the Kriging trend.
- 4.2 Algorithm: The two PC-Kriging approaches use the experimental design, model responses, stochastic-input description, and autocorrelation-function specification as input information.They are compared with ordinary Kriging and PCE across analytical benchmarks and increasing experimental-design sizes.
- 4.2 Algorithm: OPC-Kriging adds ranked polynomials one at a time, recalibrating the trend and autocorrelation parameters iteratively.The final model is selected by minimizing the leave-one-out error.
5 Analytical benchmark functions
Across six analytical benchmarks, PC-Kriging generally matched or exceeded traditional PCE and Kriging, with performance depending on function complexity and experimental-design size. Its increased accuracy came with higher computational cost, especially for OPC-Kriging.
- Benchmark functions: The study evaluates PC-Kriging against PCE and Kriging on six analytical functions with different dimensions, input distributions, and smoothness properties.The benchmarks include Ishigami, Sobol’, Rosenbrock, Morris, Rastrigin, and O’Hagan functions.
- Small experimental design: For Ishigami, ordinary Kriging performs best at N = 20 because PC-Kriging is more prone to overfitting with small designs.With larger designs, PC-Kriging outperforms traditional approaches; for N ≥50 it performs similarly to, though slightly better than, PCE.
- Small experimental design: For Rosenbrock, highly accurate surrogate models are obtained with only 20 samples, and OPC-Kriging performs best for small sample sizes.Its polynomial structure allows accurate modeling with a small number of polynomial terms.
- Small experimental design: For Sobol’, OPC-Kriging performs significantly better at N = 16 and 32, while all approaches behave similarly at N = 64 and 128.The function requires more samples because it is eight-dimensional and non-smooth at x_i = 0.5.
- Small experimental design: For Rastrigin, OPC-Kriging has statistically the lowest relative generalization errors across experimental-design sizes, while ordinary Kriging performs worst.The function’s high-frequency oscillations require many samples despite its low dimensionality.
- Overall comparison: Across all six functions, PC-Kriging performs better than or at least as well as traditional PCE and Kriging, but its accuracy requires higher computational cost.OPC-Kriging is most expensive because it iteratively selects polynomials and repeatedly calibrates Kriging models.
6 Conclusion
The paper introduces PC-Kriging to address the resource demands of repeated computational-model runs, combining PCE and Kriging for surrogate modeling. On benchmark analytical functions, PC-Kriging performs at least as well as distinct approaches, while its broader engineering applicability remains under study.
- Meta-models approximate computational models with easy-to-evaluate functions to address the resource limitations of repeated runs in reliability assessment and design optimization.
- PC-Kriging combines sparse polynomial regression from PCE with universal Kriging, using LAR to select polynomials; OPC-Kriging iteratively fits correlation hyperparameters.
- On benchmark analytical functions, PC-Kriging is better than or at least as good as ordinary Kriging and PCE for small experimental designs.
- OPC-Kriging reduces regression complexity relative to SPC-Kriging by using fewer polynomials, but requires higher computational calibration cost.
- PC-Kriging performance is evaluated only on benchmark analytical functions, while applications to realistic reliability and design-optimization problems remain ongoing.