Source-linked AI summary
Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques
Haibin Xiong, Shaoheng Dai, Peng Lan, Xuzhen He, Chenxi Tong, Sheng Zhang, Daichao Sheng
TL;DR
Predicting undrained shear strength is difficult because geotechnical data are sparse, variable, and uncertain. The study combines imputation with probabilistic machine learning and finds that the MHA-PNN approach improves prediction and uncertainty quantification over the MN-based model.
Problem
Probabilistic prediction of undrained shear strength is challenged by substantial uncertainty, sparsity, and variability in geotechnical data.
Method
The study combines imputation, PXGB-based evaluation, and a multi-head-attention probabilistic neural network for undrained shear-strength prediction.
Results
R2 increased from -1.31 to 0.55 and wCI decreased by about 96% for the proposed model relative to the comparison.
Takeaways & Limitations
The findings indicate potential for probabilistic indirect modelling of undrained shear strength with sparse and incomplete geotechnical data.
Takeaways & Limitations
Imputed values should not be interpreted as accurate reconstructions.
Abstract
from arXiv · showhide
Accurate prediction of undrained shear strength (su) is crucial for geotechnical design, but is often hampered by substantial uncertainty in traditional empirical methods. This study uses the CLAY/10/7490 global database to develop probabilistic indirect models to predict su based on Atterberg limits and piezocone cone penetration (CPTU) measurements. Firstly, the dataset has a high missing data rate and variability. We test three imputation methods - multivariate normal (MN), multiple imputation by chained equations (MICE), and miss forest (MF) - to fill the missing values. To validate their effectiveness, a Probabilistic Extreme Gradient Boosting (PXGB) model is developed, and the imputation methods are evaluated by comparing the PXGB's performance when trained on the imputed datasets against that on the original incomplete data. Secondly, the indirect model is built by integrating a multi-head attention (MHA) mechanism into an artificial neural network (ANN) to enhance information extraction from limited data, which leads to the MHA-based probabilistic neural networks (MHA-PNN) model. The models' performance, alongside a conventional MN-based prediction model, was evaluated using root mean square error (RMSE), coefficient of determination (R2), mean absolute percentage error (MAPE), conditional interval width (wCI), and coverage rate (CR). Results demonstrate that the proposed MN-enhanced MHA-PNN model substantially outperforms other models in both prediction accuracy and uncertainty quantification. These findings highlight the potential of this integrated strategy for building robust probabilistic indirect models in geotechnical applications, particularly when confronted with sparse and incomplete datasets.
1. Introduction
Accurate undrained shear strength prediction is difficult because geotechnical variability and incomplete datasets undermine conventional deterministic and data-driven approaches. This study therefore combines imputation methods with probabilistic machine learning, including PXGB and MHA-PNN, to quantify uncertainty and improve prediction.
- Motivation: Accurate assessment of undrained shear strength remains challenging in geotechnical design, especially under transient loading.Applications include seismic events, tunnelling construction, and slope stability assessments.
- Data challenge: The CLAY/10/7490 database contains 10 geotechnical features and 7490 rows from 251 studies across 30 countries or regions, but its completion rate is only 34.1%.Discarding incomplete records would cause substantial data loss and forfeit valuable information.
- Methods: MN, MICE, and MF imputation methods are evaluated using PXGB trained on imputed datasets versus the original incomplete data.MN is the simple baseline, MICE the standard statistical approach, and MF a representative advanced machine-learning method.
- Proposed framework: The proposed probabilistic framework integrates imputation strategies with machine learning by incorporating multi-head attention into an ANN, forming MHA-PNN.It is applied to CLAY/10/7490 and compared with a conventional MN-based prediction model for prediction accuracy and uncertainty quantification.
2. Prediction and Imputation Based on Multivariate Normal (MN) Model
The MN-based approach addresses severe incompleteness and variability in CLAY/10/7490 by transforming non-Gaussian parameters into normal space and modelling their statistical relationships. Its predictions benefit from informative CPTU features, while the global database remains mainly suitable for first-order rather than site-specific estimates.
- MN transformation: Because the parameters are non-Gaussian, the MN model first applies Johnson-system transformations to approximate normality.The fitted Johnson-SU curves align well with the observed distributions, supporting transformation into normal space.
- MN prediction framework: The model transforms the original parameters into normal space, estimates their correlations, and uses these relationships to predict target-variable probability distributions.The transformed parameters pass marginal-normality checks based on Q-Q plots and Anderson-Darling tests.
- Feature relationships: Atterberg limits show little to no correlation with su(mob)/σ′v, whereas CPTU parameters exhibit a strong linear relationship with the target.This supports selecting CPTU measurements as informative features for probabilistic prediction.
- Prediction performance: The MN-based prediction model yields narrower confidence intervals and improved agreement with observations when informative variables are included.The model is presented as a statistically rigorous approach for prediction under missing-data and data-sparsity conditions.
3. Imputation Based on Multiple Imputation and Miss Forest
The section evaluates MICE and miss forest imputation for the high-missingness Clay/10/7490 database after finding that MN imputation preserved statistical relationships but did not improve predictive performance. MICE generally preserves the original distributions most closely, followed by MF and MN.
- MICE imputation: MICE iteratively imputes each variable from the others’ observed and newly imputed values, repeating until convergence and generating multiple databases.Unlike MN, MICE models each variable according to its own distribution without requiring approximate normality.
- MF comparison: MICE is a variant of MF that uses regression models rather than random forests, and the imputation process is generally successful based on similarity with the original data.Overall, the MICE database most closely aligns with the original, followed by MF and then MN.
4. Two Probabilistic Indirect Models
The section develops PXGB for probabilistic prediction with missing data and introduces MHA-PNN, which combines multi-head attention with probabilistic neural-network outputs. Results show that imputation and model choice strongly affect accuracy and uncertainty quantification, with MN-enhanced MHA-PNN performing best overall.
- PXGB model: PXGB extends XGBoost into a probabilistic framework by assuming a normally distributed target and optimizing negative log-likelihood for uncertainty quantification.Its native handling of missing data enables direct comparison between original and imputed datasets using MN, MICE, and MF.
- MHA-PNN model: MHA-PNN incorporates a multi-head attention layer into a neural network whose output is the target probability distribution.The architecture contains input, hidden, MHA, and output modules, with dropout regularization in the hidden and attention layers.
- MHA-PNN model: MHA independently projects hidden representations into multiple attention heads, captures distinct input aspects, concatenates their outputs, and applies a residual connection.Scaled dot-product attention computes feature relevance before the head outputs are combined and passed to probabilistic modelling.
- Comparative results: Across five folds, MHA-PNN consistently outperformed PXGB and MN-pred. models in predictive accuracy and uncertainty quantification, while MN offered the best trade-off.The results also show poor predictive performance when three or fewer input features are available, regardless of dataset version.
5. Conclusions
The study develops a probabilistic framework combining imputation and machine learning to predict undrained shear strength from sparse, variable CLAY/10/7490 data with uncertainty quantification. The MHA-PNN improves predictive accuracy and uncertainty estimation but requires site-specific testing for reliable analysis.
- Framework and data: The framework combines MN, MICE, and MF imputation with PXGB and probabilistic neural-network models using the CLAY/10/7490 database.The database has a low completion rate of around 34.1% and high variability.
- Feature sensitivity and imputation: Using fewer than four input features produced largely unreliable predictions, while imputed databases closely matched the original distributions despite feature sensitivity.The inputs included Atterberg limit and CPTU parameters.
- Limitations: Imputed values should not be treated as accurate reconstructions of unobserved data, so site-specific investigations remain essential for reliable decisions.The limitation applies despite the completed databases’ similarity to the original distributions.
- Model performance: The MHA-PNN consistently achieved lower errors, higher coverage rates, and narrower uncertainty intervals than the evaluated comparison models.It predicts the target variable’s probability distribution after integrating multi-head attention into an ANN.
- Limitations: Engineers should treat model outputs as preliminary guidance and supplement them with local testing, especially for geologically distinct, structured, or soft clays.Site-specific data are mandatory for reliable analysis in sensitive conditions.