Source-linked AI summary
Adaptive surrogate modeling for high-dimensional spatio-temporal output
Berkcan Kapusuzoglu, Shunsaku Matsumoto, Yoshitomo Miyagi, Daigo Watanabe, Sankaran Mahadevan
TL;DR
High-dimensional spatio-temporal engineering models are expensive to evaluate repeatedly, creating a need for accurate surrogates with limited training data. The paper combines dimension reduction, error quantification, and exploration–exploitation adaptive sampling, producing very good results across test problems and acceptable accuracy for a turbine-blade model.
Problem
Expensive physics-based models and limited training resources make it difficult to construct accurate surrogates for high-dimensional spatio-temporal engineering outputs.
Method
The approach reduces outputs to a low-dimensional space, quantifies original-space errors, and adaptively selects training points using exploration and exploitation.
Results
The method yields very good results across investigated test problems and reaches an acceptable accuracy level for the turbine-blade example.
Takeaways & Limitations
The proposed strategy supports adaptive surrogate improvement for engineering systems with high-dimensional spatio-temporal outputs.
Takeaways & Limitations
The in-house code cannot be shared because of its confidential proprietary nature.
Abstract
from arXiv · showhide
This paper develops an adaptive surrogate modeling method for problems with very high-dimensional spatio-temporal outputs. The analysis of spatio-temporal multi-physics systems is computationally expensive and consists of a large number of inputs and outputs. Surrogate models are often constructed to replace the physics-based model to achieve computational efficiency in analyses such as uncertainty quantification and optimization that require many function calls. In order to address the challenge introduced by the high dimensionality of spatio-temporal output, a dimension reduction method is first employed to map the high-dimensional output to a low-dimensional latent space. This is followed by the construction of the surrogate model in the low-dimensional space. The prediction error in the original space, which includes both the reconstruction error and surrogate model error, is evaluated using different error metrics. Based on the prediction accuracy of the surrogate model, new training points are identified for adaptive improvement of the surrogate model. We present a novel adaptive sampling technique that combines exploration and exploitation to improve the surrogate model accuracy with the fewest possible runs of the expensive physics-based model. Thermo-mechanical analysis of a gas turbine engine blade is used to analyze the effectiveness of the proposed method.
1 Introduction
The paper addresses the computational cost of analyzing nonlinear, coupled multi-physics systems with high-dimensional spatio-temporal outputs. It proposes adaptive surrogate modeling that combines output dimension reduction, error quantification, and exploration–exploitation sampling.
- Motivation: Repeated physics-model runs support uncertainty quantification, model calibration, optimization, reliability assessment, and system health diagnosis, creating substantial computational costs.These analyses require evaluating engineering responses across many input realizations.
- Challenge: Gas-turbine components pose an especially difficult setting because coupled fluid, thermal, and structural equations produce multivariate outputs varying over space and time.The resulting quantity of interest is a high-dimensional field quantity.
- Contributions: The paper contributes dimension reduction, error quantification in both reduced and original spaces, and sequential adaptive sampling for multivariate spatio-temporal outputs.Original-space prediction error includes reconstruction error and reduced-space surrogate error.
- Research gap: Existing adaptive learning methods often target a single limit state or scalar output and are not readily applicable to dynamic multi-physics systems with high-dimensional spatio-temporal outputs.This motivates a strategy that models the entire input space and multivariate outputs.
- Proposed approach: The proposed method uses randomized singular value decomposition to obtain a lower-dimensional output representation, then constructs a surrogate and adaptively selects training runs.The sampling strategy combines exploration and exploitation for time-dependent multi-physics problems.
2 Background: Output Dimension Reduction
Because the model outputs vary over space and time and are very high-dimensional, the paper reduces response fields to an uncorrelated low-dimensional latent space before constructing surrogate models. It uses randomized SVD to obtain this representation efficiently for very large data matrices.
- Motivation: High-dimensional spatio-temporal outputs make it impractical to construct surrogate models directly for every output, location, and time instant.The proposed alternative first maps the response field to an uncorrelated latent space.
- Singular Value Decomposition: SVD maps large high-dimensional datasets to a low-dimensional latent space and handles spatial correlation in non-square matrices.Basic SVD can be computationally expensive and memory-intensive for very large matrices.
- Randomized SVD: rSVD projects the original data matrix into a small random subspace, then applies matrix multiplications, QR decomposition, and SVD on small matrices.Its five-component procedure begins with constructing a random matrix and projecting the original data matrix onto it.
- Low-Rank Representation: The resulting rank-r representation replaces the original dimension N, with r << N, while approximating the column space of X with high probability.The reduced basis is formed through random projection, QR decomposition, and SVD of the projected matrix.
- Computational Efficiency: rSVD is powerful for very large data matrices because its low-rank basis makes the QR decomposition and subsequent SVD not time consuming.rSVD remains a linear mapping like basic SVD while operating on substantially smaller matrices.
3 Proposed Methodology
The proposed methodology constructs an efficient surrogate model for high-dimensional spatio-temporal response prediction through dimension reduction, surrogate modeling, error quantification, and adaptive training-point selection.
- Dimension reduction: Dimension reduction obtains a lower-dimensional representation of high-dimensional spatio-temporal outputs using an adapted rSVD method with a two-step mapping strategy.The reduced representation is used as features for subsequent surrogate modeling.
- Surrogate model construction: Multiple surrogate-modeling options are investigated, and the best model is selected for multivariate output.
- Proposed Methodology: The methodology comprises four steps: two-step dimension reduction, surrogate model construction, cross-validation-based error quantification, and adaptive training-point selection.These steps are described in the following subsections.
3.1 Dimension Reduction
The method extends randomized singular value decomposition (rSVD) into a two-step dimension-reduction approach for high-dimensional spatio-temporal outputs. It first reduces output features and then removes temporal correlation to obtain a lower-dimensional representation suitable for surrogate construction, with reconstruction accuracy evaluated by RMSE.
- Two-step dimension reduction: The first reduction step uses rSVD to reduce the large output matrix’s original features to a smaller subset of important features.The output matrix S has more features (columns) than observations (rows).
- Two-step dimension reduction: The second rSVD removes temporal dependency, producing low-rank features that are not time series and can be used to build the surrogate model.The rows retain design-domain realizations, while the columns represent important non-temporal features.
- Accuracy evaluation: Reconstruction accuracy is quantified with RMSE after mapping the second feature space back to the original space.The RMSE decreases as the number of important features used to approximate S increases.
3.2 Surrogate Model Construction
The surrogate model can use established surrogate and machine-learning techniques, including boosting-based methods. Extra-Trees regressor performs best in the numerical example, while the model is intended to support both design and changing future health-management needs.
- Surrogate model construction: Available surrogate techniques include PCE, GP, SVR, and DNN, alongside boosting-based machine-learning algorithms such as lightGBM and XGBoost.The paper explores boosting methods to improve the performance of simple machine-learning models.
- Surrogate model construction: Extra-Trees regressor gives the best performance in the numerical example based on comparisons of k-fold cross-validation errors.Tree-based ensemble methods randomly construct multiple decision trees to improve generalization performance.
- Surrogate model construction: The surrogate model supports component design and future health management, where different quantities may become critical as usage changes.The model is intended to accommodate changing requirements over time.
3.3 Surrogate Model Error Quantification
Surrogate-model error is quantified with LOOCV predictions mapped from the latent space back to spatio-temporal physical outputs. Node-level MAE and rMAE, along with PNMAE and PNrMAE, characterize prediction errors for adaptive improvement.
- Leave-One-Out Cross-Validation: LOOCV trains a separate surrogate on N − 1 observations for each omitted observation and evaluates accuracy at the omitted point.This procedure uses k-fold cross-validation with k = N.
- Error Metrics: MAE provides physically interpretable, quantity-specific thresholds but is scale-dependent and requires threshold identification.MAEj denotes the mean absolute error at FEM node j, averaged across time for each spatial location.
- Error Metrics: rMAE addresses MAE’s scale dependence and compares the proposed model with a benchmark model.rMAE < 1 indicates the proposed model is better than the benchmark, whereas rMAE > 1 indicates it is worse.
- Error Evaluation: Error metrics are evaluated at each FEM node after latent-space predictions are mapped back to the original spatio-temporal output space.This evaluation uses LOOCV predictions for the physical quantities.
- Error Evaluation: PNMAE is the percentage of nodes with MAE above a threshold, while PNrMAE is the percentage of nodes with rMAE greater than one.Different metrics support different engineering priorities, such as emphasizing relative error where strong creep behavior is expected.
3.4 Proposed Adaptive Sample Selection Method
The method adaptively selects new training points by combining exploitation of high predicted error with exploration of under-covered input regions. It evaluates a learning function using surrogate-based error improvement and maximin distance, then adds the candidate with the largest value until accuracy or resource criteria are met.
- Initial design: An initial maximin Latin hypercube design uniformly covers the input space, after which LOOCV errors are calculated for the training points.The initial design uses a specified number of points and supports subsequent surrogate-error evaluation.
- Exploration and exploitation: The adaptive strategy combines exploitation of high-error regions with exploration of previously under-covered regions to avoid clustered samples.Exploration uses input-space coverage, whereas exploitation uses output information from previous training points.
- Learning function: The proposed learning function combines expected improvements based on PNMAE and PNrMAE with a maximin-distance term to select new samples.The parameters β1 and β2 weight the two expected-improvement terms, while α and γ control exploitation and exploration.
- Sample selection: The candidate with the largest learning-function value is selected as the new training point, with candidate evaluations performed using the current surrogate model.This makes evaluating many candidate points computationally inexpensive relative to running the original physics model.
- Stopping criteria: Adaptive sampling stops when the LOOCV error metrics ePNMAE and ePNrMAE reach acceptable values or the computational resource limit is reached.The resource limit is expressed through the number of training points, N.
3.5 Summary of Methodology
The methodology constructs a high-dimensional output matrix, reduces it to a lower-dimensional feature representation, and builds a surrogate model in feature space. Predictions are reconstructed in the original space, evaluated by cross-validation, and adaptively refined until accuracy or computational limits are reached.
- The methodology repeats matrix construction, reduction, prediction, evaluation, and adaptive sampling until accuracy is acceptable or computational resources are exhausted.
- The output data matrix S represents time instants for FEM runs as rows and spatial locations of multiple output quantities as columns.
- A two-step dimension reduction approach produces the lower-dimensional representation I(2), after which one surrogate model is built for all features versus process inputs.
- Surrogate predictions in feature space are projected back to the original space to obtain predictions ˆS for test-set points.
- Cross-validation evaluates surrogate accuracy in the original space, and unacceptable accuracy triggers adaptive sequential sampling to propose additional training points.
4 Evaluation of Proposed Approach using Benchmark Problems
The proposed adaptive sampling approach is evaluated against six existing techniques on seven benchmark functions using average LOOCV NRMSE. EICV and CVVor achieve the most complete accuracy, while methods differ in exploration, exploitation, dependence on initial samples, and computational effort.
- Benchmark comparison: The proposed technique is compared with EI, CVV, LOLA, MSD, MASA, and CVVor on seven benchmark functions of varying complexity.The evaluation analyzes each technique’s ability to accurately capture the benchmark test functions.
- Evaluation metric: Average LOOCV NRMSE is used to distinguish techniques because R2 values are close across cases.NRMSE is computed for each LOOCV fold and then averaged across folds.
- Experimental setup: 20 initial points are used for each two-dimensional test function under the rule m = 10n.The initial design size is selected because exploitation-heavy methods depend strongly on the initial sample.
- Sampling behavior: MASA emphasizes exploration and is less dependent on initial sample size, whereas MSD performs poorly because it exploits less than the other methods.The contrasting behavior is illustrated through the adaptive sample locations and NRMSE results.
- Accuracy results: CVVor and EICV yield the most complete performance and best accuracy, with updated-surrogate NRMSE values close to zero across all test functions.The updated surrogate adds 25 samples, increasing the total from 20 to 45 samples.
- Computational effort: EICV and SFEICV require more computational effort because each constructs two GP models, but their accuracy is better or comparable to other methods.Unlike CVVor and LOLA, they do not require tessellating the input space into Voronoi cells.
5 Engineering Application
The engineering application evaluates the adaptive surrogate method on a gas turbine engine blade with six high-dimensional spatio-temporal outputs. Dimension reduction and an Extra-Trees surrogate are combined with adaptive FEM sampling, reaching acceptable analyst-level accuracy after 66 training points despite not meeting the target PNrMAE threshold.
- Engineering application: The gas turbine blade study considers six quantities—CEEQ, Dc, Mises, and x-, y-, and z-direction displacements—at many spatial and temporal points.These outputs are generated by a simplified physics-based engine blade model.
- Dimension reduction: The output matrix for 66 simulations contains approximately 629 million data points, with rows spanning simulations and time steps and columns spanning nodes and QoIs.Its dimensions are (54×66)×(29374×6).
- Dimension reduction: 99% variance is explained by the top 50 and 20 features in the first and second feature spaces, respectively, while reconstruction RMSE is approximately 0.0006 versus average predicted magnitude 0.01.RMSE is used to evaluate reconstruction accuracy.
- Surrogate modeling: 0.10513 average RMSE is achieved by the Extra-Trees regressor in cross-validation, 15% better than the next best model.The comparison uses 7-fold cross-validation.
- Adaptive sampling: The adaptive procedure expands the initial 46 FEM runs to 66 training points in four batches, and surrogate accuracy generally improves with each adaptively selected batch.Batching is used because selecting one point at a time is inefficient in industrial settings.
- Adaptive sampling: 15.8% average PNrMAE is obtained after 66 training points, above the desired average PNrMAE ≤15%, although PNMAE and RMSE are acceptable to the analyst.The procedure stops at N = 66, and the authors suggest a few more FEM runs could reach the desired level.
6 Conclusion
The paper presents an adaptive surrogate-modeling approach for high-dimensional spatio-temporal engineering outputs, combining rSVD-based dimension reduction with cross-validation-guided surrogate selection. Tests and a turbine-blade example demonstrate strong results, while future work targets broader applications, parallelized sampling, and error separation.
- Method: The approach uses two-step rSVD dimension reduction to map high-dimensional spatio-temporal outputs into an uncorrelated space, then uses cross-validation to identify the most accurate surrogate type.These steps support adaptive surrogate-model construction for engineering problems with high-dimensional spatio-temporal output.
- Validation: The proposed adaptive sampling technique yields very good results across investigated test problems and is effective for a turbine-blade multi-physics system with high-dimensional spatio-temporal outputs.The turbine-blade example uses a time-dependent multi-physics dynamic system model.
- Future work: Future work includes evaluating additional large-scale engineering applications, studying behavior across problems, improving the methodology, parallelizing sequential sampling for global optimization, and isolating reconstruction error from overall prediction error.These extensions are proposed to further evaluate and refine the adaptive sampling technique.
Replication of Results
The in-house code used in this work cannot be shared because it is confidential proprietary material identified by the sponsor.
- The authors cannot share the in-house code because of its confidential proprietary nature, as identified by the sponsor.
A Benchmark Test Functions
The paper uses two-dimensional benchmark functions to evaluate the proposed method and compare it with existing adaptive surrogate model improvement methods. These test problems are defined in Table 5, including modified functions from prior studies.
- Benchmark test functions: Two-dimensional benchmark functions are used to evaluate the proposed method and compare it with existing adaptive surrogate model improvement methods.The test problems are defined in Table 5.
- Benchmark test functions: One benchmark is a modified form of a function from Meckesheimer et al..
- Benchmark test functions: Another benchmark is a modified form of a function from Jin et al..