Source-linked AI summary
Recursive co-kriging model for Design of Computer experiments with multiple levels of fidelity with an application to hydrodynamic
Loic Le Gratiet
TL;DR
The paper addresses how to build fast surrogates for computationally expensive codes available at multiple fidelity levels. It proposes recursive multi-fidelity co-kriging through independent kriging models, extending classical kriging results while reducing complexity, and successfully applies the model to a hydrodynamic simulator.
Problem
Sensitivity analysis and optimization of expensive computer codes require fast surrogates, including models that use cheaper, less accurate simulations across multiple fidelity levels.
Method
A recursive multi-fidelity co-kriging formulation builds an s-level model through s independent kriging models and provides universal co-kriging and fast cross-validation equations.
Results
The multi-fidelity model was successfully applied to emulate a hydrodynamic code, with the recursive formulation reducing complexity while preserving predictive efficiency.
Takeaways & Limitations
The recursive formulation makes multi-fidelity co-kriging computationally more manageable and allows classical kriging results to be adapted to the co-kriging model.
Takeaways & Limitations
The closed-form cross-validation formulas do not permit re-estimation of correlation-function hyper-parameters, although they can still reduce cross-validation complexity.
Abstract
from arXiv · showhide
In many practical cases, a sensitivity analysis or an optimization of a complex time consuming computer code requires to build a fast running approximation of it - also called surrogate model. We consider in this paper the problem of building a surrogate model of a complex computer code which can be run at different levels of accuracy. The co-kriging based surrogate model is a promising tool to build such an approximation. The idea is to improve the surrogate model by using fast and less accurate versions of the code. We present here a new approach to perform a multi-fidelity co-kriging model which is based on a recursive formulation. The strength of this new method is that the co-kriging model is built through a series of independent kriging models. From them, some properties of classical kriging models can naturally be extended to the presented co-kriging model such as a fast cross-validation procedure. Moreover, based on a Bayes linear formulation, an extension of the universal kriging equations are provided for the co-kriging model. Finally, the proposed model has the advantage to reduce the computational complexity compared to the previous models. The multi-fidelity model is successfully applied to emulate a hydrodynamic simulator. This real example illustrates the efficiency of the recursive model.
1 Introduction
Complex computer codes motivate fast surrogate models, and multiple fidelity levels offer cheaper information for modeling expensive simulations. The paper presents a recursive co-kriging approach that reduces computational complexity while extending classical kriging results.
- Complex simulators make fast surrogate models necessary for sensitivity analysis and optimization when direct computation is prohibitively costly.
- Low-fidelity codes provide cheaper, less accurate simulations that can supply information for multi-fidelity surrogate modeling.
- Co-kriging can provide accurate predictions but becomes computationally expensive and vulnerable to ill-conditioned covariance matrices as observations accumulate across fidelity levels.
- The recursive model builds an s-level co-kriging model through s independent kriging models, enabling separate parameter estimation and smaller matrix inversions.
- The approach extends universal kriging equations and fast cross-validation to multi-fidelity co-kriging, then demonstrates them with a hydrodynamic simulator.
2 Multi-fidelity Gaussian process regression
The paper formulates multi-fidelity co-kriging recursively, expressing the model through conditioned Gaussian processes and independent kriging models. This formulation preserves the original predictive distribution while reducing computational complexity and enabling additional kriging-based results.
- 2.1 The classical autoregressive model: The model assumes code levels are ordered by increasing fidelity, with cheaper, less accurate codes preceding the most accurate and costly code.The responses are modeled by Gaussian processes, and nested experimental designs facilitate parameter estimation.
- 2.2 Recursive multi-fidelity model: The recursive model builds an s-level co-kriging model through s independent kriging models, dividing simulations according to fidelity level.The formulation expresses each higher-fidelity process using the conditioned lower-fidelity process and a discrepancy process.
- 2.2 Recursive multi-fidelity model: The recursive model yields surrogate models for all response levels and makes each level’s contribution to model error more interpretable.The predictive mean at level t uses responses from the first t levels, while its variance is described recursively.
- 2.2 Recursive multi-fidelity model: The recursive and Kennedy–O’Hagan formulations have the same predictive Gaussian distribution, including identical predictive mean and variance.The equivalence is established for the conditioned models under the stated Gaussian-process assumptions.
- 2.2 Recursive multi-fidelity model: The recursive formulation provides closed-form universal co-kriging mean and variance expressions and extends fast cross-validation to the multi-fidelity model.These are presented as extensions of classical kriging results.
- 2.3 Complexity analysis: Inverting s covariance matrices of sizes n_t × n_t is less expensive and requires less memory than inverting and storing the combined covariance matrix.The original approach uses a matrix whose size is the sum of observations across fidelity levels.
3 Universal co-kriging model
The paper integrates parameter uncertainty into recursive multi-fidelity co-kriging predictions, yielding closed-form posterior mean and variance despite a non-Gaussian predictive distribution. These universal co-kriging equations are presented as a novelty enabled by the recursive formulation.
- Integrating posterior parameter distributions produces a predictive distribution that is not Gaussian, but its posterior mean and variance have closed-form expressions.The distribution is conditioned only on observations after integrating parameter uncertainty.
- The recursive formulation enables a novel proposition for closed-form posterior mean and variance conditioned only on observations.The authors state that this result does not appear obtainable directly from the Kennedy–O’Hagan formulation.
- The model considers nested designs with D_s ⊆ D_{s−1} ⊆ ··· ⊆ D_1 across fidelity levels.The Gaussian-process formulation is defined for s processes observed at nested design points.
- The predictive mean replaces model parameters with their posterior means, while the predictive variance uses the posterior mean of the variance parameter and an added estimation-uncertainty term.The added term represents uncertainty from estimating regression parameters, including the adjustment coefficient.
4 Fast cross-validation for co-kriging surrogate models
The recursive formulation extends cross-validation shortcuts from kriging to multi-fidelity co-kriging, including re-estimation of regression and variance parameters. Closed-form expressions reduce computational complexity, although correlation-function hyper-parameters are not re-estimated.
- The recursive formulation extends fast cross-validation ideas to co-kriging, unlike the original Kennedy–O’Hagan formulation.The procedure uses training and test-set splits and supports leave-one-out cross-validation as a special case.
- Re-estimating the variance coefficient is important because fixing it can cause huge cross-validation predictive-variance errors with small observations or larger test sets.
- Nested experimental designs allow test points removed from the highest fidelity level to be removed from corresponding lower fidelity levels.The result is stated for multi-fidelity co-kriging and can be extended from the non-informative to the informative parameter-estimation case.
- The closed-form expressions provide cross-validation errors and predictive variances for points removed from the highest code levels.The proposition defines errors and variances for the corresponding test-set procedure.
- The formulas remain valid for s = 1 and therefore also cover classical kriging with re-estimated regression and variance parameters.They provide closed-form k-fold cross-validation expressions derived from universal co-kriging equations.
- The closed-form cross-validation formulas do not re-estimate correlation-function hyper-parameters, though they can still reduce computational complexity in that setting.
5 Illustration: hydrodynamic simulator
The MELTEM application evaluates recursive multi-fidelity co-kriging for accurate prediction and uncertainty assessment with few expensive-code runs. It compares model accuracy, parameter estimation, experimental-design strategies, and simple versus universal co-kriging.
- Application: MELTEM simulates second-order turbulence in gaseous mixtures using two input coefficients over [0.5, 1.5] × [1.5, 2.3].Its outputs are the dissipation factor eps and mixture characteristic length Lc.
- Parameter estimation: Hyper-parameters are estimated by concentrated restricted likelihood, while recursive leave-one-out cross-validation reduces complexity through separate matrix inversions and sequential parameter estimation.The recursive formulation estimates φ1 first, then φt for t = 2, . . . , s.
- Kriging comparison: The comparison uses 500 nested designs, with n1 = 25 cheap-code runs, n2 ∈ {5, 10, 15, 20, 25} expensive-code runs, and a 175-observation test set.Both models use a Matern 5/2 covariance kernel with constant ρ, β1, and β2.
- Kriging comparison: For small n2, co-kriging has significantly lower prediction error than kriging; as n2 approaches n1, their errors converge because only expensive-code observations remain when D2 = D1.This comparison concerns the output eps.
- Dissipation factor eps: The estimated correlation between the cheap and expensive codes is 82.64%, and βρ1 is close to 1, indicating substantial cheap-code information and high code correlation.The results also identify important correlation between βρ1 and β2 for parameter estimation.
- Mixture characteristic length Lc: An input-dependent adjustment coefficient improves the co-kriging model: Case 2 is clearly better than Case 1, with ρ1 averaging 1.18 and decreasing from 1.42 to 0.94 across x1.The Case 2 variance estimate is much smaller, which the authors associate with better learning.
6 Conclusion
The paper presents a recursive multi-fidelity co-kriging formulation that builds surrogate models from simulations at different fidelity levels. It reduces model complexity while preserving predictive efficiency, extends kriging-based procedures, and is successfully applied to a hydrodynamic code.
- The recursive formulation builds a multi-fidelity co-kriging surrogate from simulations at different fidelity levels.
- The model reduces co-kriging complexity by making construction equivalent to building s independent kriging models.
- Bayesian parameter estimation yields closed-form posterior expressions, including universal co-kriging mean and covariance formulas.
- Closed-form cross-validation equations considerably reduce the complexity of cross-validation for the co-kriging surrogate model.
- The suggested model is successfully applied to a hydrodynamic code, including a practical multi-fidelity experimental-design procedure.
A.1 Proof of Proposition 1
The proof establishes that the recursive model's predictive mean and covariance satisfy the same recursive relations as the corresponding co-kriging quantities. This identifies the recursive construction with the established co-kriging formulation.
- The co-kriging mean satisfies the recursive relation of the proposed model, with mZ1(x) = µZ1(x).
- The recursive model defines a sequence of conditioned Gaussian processes whose means are related across fidelity levels.
- The co-kriging covariance follows the same recursive relation as the covariance of the recursive model.
- Evaluating the covariance equality at x = x′ proves the second equality of Proposition 1.
A.2 Proof of Proposition 2
The proof derives posterior predictive mean and covariance relations for the recursive model using total expectation, total variance, and inverse-Gamma moments.
- The predictive mean does not depend on σ2_t, allowing the law of total expectation to derive the required mean relation.
- The covariance derivation uses the law of total variance together with the preceding mean and variance relations.
- Independence between E_t terms supports the recursive variance calculation.
- The final equality follows because an inverse-Gamma distribution IG(a, b) has mean b/(a −1).
A.3 Proof of Proposition 3
The proof derives cross-validation equations for the co-kriging model by removing selected observations and applying blockwise covariance-matrix identities. It treats fixed and re-estimated trend and variance parameters.
- Cross-validation removes the k last points of D_s, collected in D_test, from the design.
- In the simple co-kriging case, block-wise inversion gives the covariance relation needed for cross-validation.
- The prediction residual for removed observations is εZs,ξs = zs(Dtest) −µZs(Dtest).
- When trend and variance parameters are unknown, they are re-estimated after observations are removed.
- The resulting parameter estimates and matrix identities complete the universal co-kriging cross-validation equations.