Source-linked AI summary
A Deep Learning-Based Stacking Ensemble Framework for Turbofan Engine Remaining Useful Life Prediction
Limon Bin Hossain, Md. Salehin Seyam, Md Rashedul Islam, Abdur Rahman, Md Sharifuzzaman
TL;DR
Accurate turbofan RUL prediction matters for safety-critical aerospace health management, while existing methods often underuse heterogeneous learners and report incomplete metrics. This study combines four deep architectures through an XGBoost two-level stacking ensemble and evaluates it on C-MAPSS FD001 and FD003. The ensemble outperforms the compared baselines, achieving RMSE 9.989 and 8.613 for FD001 and FD003, respectively.
Problem
Existing methods often rely on single architectures or unstructured combinations, and few report MAE and R2 alongside RMSE, limiting comprehensive evaluation.
Method
A two-level stacking ensemble combines LSTM, CNN, CNN-LSTM, and CNN-GRU predictions through an XGBoost meta-learner.
Results
RMSE 9.989 and 8.613 were achieved for FD001 and FD003, respectively, outperforming the compared baseline and deep learning models.
Takeaways & Limitations
The results support stacked generalization as effective for prognostics and health management across the evaluated C-MAPSS subsets.
Abstract
from arXiv · showhide
This study proposes a two-level stacking ensemble framework for Remaining Useful Life (RUL) prediction of turbofan engines, evaluated on the NASA C-MAPSS benchmark using the FD001 and FD003 subsets. The framework integrates four heterogeneous deep learning base learners: Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), CNN-LSTM, and CNN-GRU, whose out-of-fold predictions are combined by an XGBoost meta-learner to capture complex degradation patterns while mitigating individual model biases. Comprehensive experiments demonstrate that the stacking ensemble achieves superior predictive performance, with Root Mean Square Error (RMSE) of 9.989 and 8.613, Mean Absolute Error (MAE) of 7.081 and 5.195, and R-squared values of 0.899 and 0.906 for FD001 and FD003, respectively. Compared to the best-reported baseline (TCAT: RMSE 11.12 and 11.02), the proposed method achieves RMSE reductions of 10.2 percent and 21.8 percent for FD001 and FD003, respectively. Feature correlation analysis, residual diagnostics, and training convergence curves validate the model's robustness. These findings underscore the efficacy of stacking ensemble methods for prognostics and health management in safety-critical aerospace applications.
I. INTRODUCTION
RUL prediction is important for safe, reliable, and efficient turbofan operation, yet degradation variability and incomplete exploitation of heterogeneous learners remain challenges. The study addresses these challenges with a structured stacking ensemble evaluated comprehensively on C-MAPSS.
- RUL estimates the time until failure or unacceptable performance, linking prediction accuracy to flight safety, reliability, and maintenance efficiency.
- The study develops preprocessing, four heterogeneous deep learning base learners, and an XGBoost-based two-level stacking ensemble for C-MAPSS degradation data.
- The framework is evaluated on FD001 and FD003 using RMSE, MAE, R2, correlation analysis, residual diagnostics, and convergence analysis.
- Existing approaches often use single architectures or unstructured combinations, limiting systematic use of complementary heterogeneous learners through stacked generalization.
- The proposed study responds to incomplete evaluation practice by reporting MAE and R2 alongside RMSE while comparing against contemporary C-MAPSS methods.
III. PROPOSED METHODOLOGY
The methodology maps recent multivariate sensor windows to RUL estimates and prepares C-MAPSS inputs through sensor selection and normalization. The formulation retains informative signals while applying training-only scaling to the test data.
- A. Problem Formulation: RUL prediction learns a mapping from a recent multivariate measurement window to the true RUL at that window’s last cycle.
- Data preprocessing: FD001 and FD003 recordings contain 21 sensors and 3 operating settings, with seven flat channels discarded to retain 14 informative sensors.
- Data preprocessing: Each retained sensor is linearly rescaled to the [0, 1] interval to accommodate heterogeneous physical units and magnitudes.
- Data preprocessing: Normalization extrema are estimated using training data only and then applied unchanged to test data, preventing information leakage.
3) Piecewise-Linear RUL Target:
The target formulation models healthy operation with a capped constant RUL and degradation with linear decay. Fixed-length windows preserve temporal context for the sequence models and define inference inputs.
- 3) Piecewise-Linear RUL Target:: Rmax = 125 cycles caps the target under an assumed constant healthy phase followed by linear RUL decay.
- 3) Piecewise-Linear RUL Target:: The piecewise-linear target discourages implausibly large predictions far from failure and focuses learning on the informative degradation region.
- Window construction: A window length of Lw = 40 with unit stride produces labeled samples using the window’s last cycle.
- Window construction: During inference, one final 40-cycle window is constructed for each test unit and used for prediction.
- Base-learner rationale: The four complementary architectures are trained independently so partially decorrelated prediction errors can support ensemble fusion.
1) Long Short-Term Memory (LSTM) Architecture:
The LSTM captures long-range temporal dependencies with gated memory, while the CNN extracts local degradation patterns using temporal convolutions. Their architectures end in regression heads that produce scalar RUL estimates.
- 1) Long Short-Term Memory (LSTM) Architecture:: The LSTM models long-range temporal dependencies through gated memory cells using the current input and previous hidden state.
- 1) Long Short-Term Memory (LSTM) Architecture:: Two stacked LSTM layers, dropout, and a fully connected regression head map the final hidden state to a scalar RUL.
- 2) Convolutional Neural Network (CNN) Architecture:: The one-dimensional CNN extracts local degradation patterns by sliding learnable kernels along the time axis.
- 2) Convolutional Neural Network (CNN) Architecture:: Stacked convolution-pooling blocks are flattened and passed to a dense layer for RUL regression.
3) CNN–LSTM Architecture:
The CNN–LSTM hybrid extracts salient local features with convolutional blocks before modeling their temporal evolution with an LSTM. This combines spatial and temporal degradation modeling.
- Convolutional blocks distil salient local features from the input window before recurrent processing.
- The LSTM models the temporal evolution of convolutionally extracted features.
- CNN processing reduces noise and dimensionality, while the LSTM captures long-term degradation trends.
- The CNN–GRU hybrid replaces the recurrent block with a lighter GRU while retaining comparable temporal modeling capacity.Its update and reset gates support sequence modeling with fewer parameters.
- The GRU’s fewer parameters reduce overfitting risk on limited training windows.
D. Stacking Ensemble
The stacking ensemble trains a meta-learner on cross-validated predictions from heterogeneous base models rather than raw inputs. XGBoost then learns a regularized nonlinear combination intended to exploit complementary strengths and suppress individual-model errors.
- The meta-learner is trained on base-model predictions rather than raw inputs.
- K = 5 cross-validation generates unbiased out-of-fold meta-features while avoiding target leakage.
- For each fold, every base model is trained without the held-out fold and predicts that fold.
- The out-of-fold predictions from all base learners are stacked into a meta-feature vector.
- Each base model is retrained on the full training set to create test-time meta-features for unseen windows.
- XGBoost maps the stacked meta-features to final RUL estimates using an additive ensemble of regression trees.
- The regularized tree objective uses squared-error loss, leaf-count and L2 penalties, with second-order gradients and Hessians.
- A nonlinear regularized combination of base predictions exploits complementary strengths and suppresses idiosyncratic errors.
E. Evaluation Metrics
Performance is assessed with RMSE, MAE, and the coefficient of determination. Lower RMSE and MAE, together with higher R2, indicate better predictive performance.
- RMSE penalizes large prediction deviations.
- MAE measures the average magnitude of prediction deviations.
- The coefficient of determination quantifies the variance explained by predictions.
- Lower RMSE and MAE combined with higher R2 indicate better predictive performance.
F. Training Configuration
The experiments use the NASA C-MAPSS benchmark and focus on FD001 and FD003, both operating under a single condition. Deep learners and the meta-learner are trained with specified optimization, regularization, and early-stopping settings.
- Training Configuration: The deep base learners use Adam with initial learning rate 10^-3 and minimize mean squared error.
- Training Configuration: Training uses batch size 256 for up to 50 epochs, with early stopping after 10 validation epochs without improvement.
- Training Configuration: A dropout rate of 0.2 regularizes recurrent and dense layers.
- Training Configuration: The XGBoost meta-learner uses squared-error loss, learning rate η = 0.05, maximum depth 4, 0.8 subsampling ratios, and λ = 1.
- Dataset: C-MAPSS combines thermodynamic engine models with actuator and sensor noise to produce multivariate turbofan degradation time series.
- Dataset: FD001 and FD003 both use a single operating condition, isolating the effect of fault-mode complexity.
V. RESULTS AND DISCUSSION
The stacking ensemble delivers strong RUL prediction across FD001 and FD003, outperforming individual deep-learning baselines and the previously best-reported TCAT baseline. Correlation, residual, and convergence analyses further support the reported predictive reliability and training stability.
- Comparative performance: FD001 performance reached RMSE 9.989, MAE 7.081, and R2 0.899, exceeding all listed individual deep-learning baselines.The compared baselines were LSTM, CNN, CNN-LSTM, and CNN-GRU.
- Comparative performance: FD003 performance reached RMSE 8.613, MAE 5.195, and R2 0.906, with the ensemble explaining approximately 90% of RUL variance.The reported FD003 comparison includes the LSTM baseline and the ensemble metrics.
- Diagnostic analyses: Feature correlation analysis identified distinct FD001 and FD003 correlation structures, with sensors 12, 7, and 15 strongly correlated with operational settings.These correlations informed feature selection for model development.
- Diagnostic analyses: Actual-versus-predicted plots showed excellent agreement with the ideal fit line, matching R2 values of 0.899 for FD001 and 0.906 for FD003.The plot-based assessment corroborated the ensemble’s predictive reliability.
- Diagnostic analyses: Residuals were homoscedastic and approximately normally distributed with minimal bias, while training curves converged smoothly without overfitting.Together, these diagnostics supported the model assumptions and indicated effective regularization.
- Comparative performance: 9.989 and 8.613 RMSE were achieved on FD001 and FD003, respectively, outperforming TCAT values of 11.12 and 11.02.These correspond to RMSE improvements of 10.2% and 21.8% over TCAT.
B. Future Work
The study identifies several directions for extending the stacking framework and strengthening its generalizability and operational applicability.
- Attention mechanisms and transformer blocks could be added as base learners to potentially improve predictive accuracy.
- Evaluation could be extended to the multicondition FD002 and FD004 subsets and real-world industrial datasets to strengthen generalizability.
- Domain adaptation and uncertainty quantification could improve deployment in operational settings with limited labeled data.