PaperlayerView exact revision on arXiv

Source-linked AI summary

Remaining Useful Lifetime Prediction via Deep Domain Adaptation

Paulo R. de O. da Costa, Alp Akcay, Yingqian Zhang, Uzay Kaymak

arXiv:1907.07480v1cs.LGstat.ML

TL;DR

The paper addresses RUL prediction when labelled run-to-failure data and target sensor data come from different domains. It combines LSTM time-window feature extraction with adversarial domain adaptation, reporting lower target errors than non-adapted models and identifying source-target similarity as an important boundary.

  • Problem

    RUL models often lack target run-to-failure labels and must handle distribution shifts caused by operating conditions, fault modes, noise, and equipment changes.

  • Method

    LSTM-DANN uses time windows and adversarial learning to learn domain-invariant features from labelled source RUL data for prediction on unlabelled target sensor data.

  • Results

    The method achieves lower target RUL errors than models without adaptation and transfers more effectively when source data include more fault modes or operating conditions than the target.

  • Takeaways & Limitations

    The methodology could be useful when source and target domains contain similar degradation data with some distribution shift, including practical PHM settings with unlabelled target data.

  • Takeaways & Limitations

    Adaptation accuracy depends on the degree of distribution variation, and the paper does not establish a reliable rule for when transfer should be performed.

Abstract

from arXiv · show

In Prognostics and Health Management (PHM) sufficient prior observed degradation data is usually critical for Remaining Useful Lifetime (RUL) prediction. Most previous data-driven prediction methods assume that training (source) and testing (target) condition monitoring data have similar distributions. However, due to different operating conditions, fault modes, noise and equipment updates distribution shift exists across different data domains. This shift reduces the performance of predictive models previously built to specific conditions when no observed run-to-failure data is available for retraining. To address this issue, this paper proposes a new data-driven approach for domain adaptation in prognostics using Long Short-Term Neural Networks (LSTM). We use a time window approach to extract temporal information from time-series data in a source domain with observed RUL values and a target domain containing only sensor information. We propose a Domain Adversarial Neural Network (DANN) approach to learn domain-invariant features that can be used to predict the RUL in the target domain. The experimental results show that the proposed method can provide more reliable RUL predictions under datasets with different operating conditions and fault modes. These results suggest that the proposed method offers a promising approach to performing domain adaptation in practical PHM applications.

1. Introduction

PHM RUL models often face distribution shifts and limited run-to-failure data across equipment and operating conditions. The paper introduces LSTM-based adversarial domain adaptation to transfer predictions from labelled source data to unlabelled target data.

  • Motivation: Real-world PHM systems collect heterogeneous sensor data across equipment versions, malfunctions, installation times, usage patterns, and degradation levels.These differences create changing multivariate temporal inputs for health-state estimation.
  • Problem: Classical predictive models require sufficient annotated historical data, while run-to-failure observations become scarcer as time-based maintenance is adopted.Retraining can delay prognostics actions until enough data supports accurate prediction.
  • Problem: Domain adaptation addresses models trained on specific run-to-failure data that must handle different features, distributions, and limited fault information.Existing approaches often inadequately capture temporal dependencies in multivariate condition-monitoring data.
  • Approach: The proposed method uses LSTMs and adversarial learning to learn domain-invariant temporal features from a labelled source domain and transfer RUL prediction to an unlabelled target domain.The approach targets related RUL tasks with differing feature distributions.
  • Evaluation: Experiments use four C-MAPSS NASA Turbofan run-to-failure datasets spanning different operating conditions and failure modes.The paper compares adapted and non-adapted methods under these domain differences.

2. Related Work

Prior PHM research includes neural and recurrent models for sequential RUL prediction, while domain adaptation has largely addressed non-sequential data or classification. This work applies unsupervised adversarial adaptation to RUL regression using LSTM temporal features.

  • RUL Prediction: PHM studies have used linear regression, SVR, fuzzy-logic systems, and neural networks to predict remaining useful lifetime from sequential data.Neural architectures are attractive because they model complex functions directly from raw data.
  • RUL Prediction: LSTM-based models have outperformed several recurrent, boosting, and convolutional alternatives in reported RUL prediction tasks.Prior work supports LSTMs as effective temporal feature extractors for prognostics.
  • Domain Shift: Most previous RUL studies assume training and testing data share distributions and feature spaces, although real PHM data can differ across devices and operating conditions.Target RUL labels may also be absent.
  • Domain Adaptation: Earlier domain-adaptation methods reduce source-target discrepancy using approaches such as MMD, reweighting, alignment, or adversarial representations.Much of this literature focuses on classification or non-sequential data rather than multivariate temporal RUL regression.
  • Proposed Position: The paper performs unsupervised adaptation from labelled source failure data to unlabelled target data using a gradient-reversal layer and a domain-invariant representation.A single neural network learns the source regression task while performing adversarial training with backpropagation through time.

3. LSTM Deep Adversarial Neural Network

The proposed LSTM-DANN predicts RUL across domains by combining temporal feature extraction with adversarial domain adaptation. It uses source RUL labels, unlabeled target sensor data, and a shared feature representation intended to support both RUL prediction and domain invariance.

  • 3.1. Problem Definition: The problem formulation uses labeled source sensor sequences and RUL values, alongside unlabeled target sensor sequences drawn from a different marginal distribution.Adaptation additionally assumes that the true input–output mapping is somewhat similar across domains.
  • 3.2. Time Window Approach: A time-window transformation collects previous multivariate sensor measurements to form inputs for predicting RUL at each time point.Left zero-padding handles short sequences, while a fixed window size maintains consistency between source and target data.
  • 3.3. Long Short-Term Memory Neural Network: LSTM layers extract temporal features from preceding windows using recurrent memory cells with forget, input, and output gates.These gates regulate information flow and update the cell state across time.
  • 3.4. LSTM Deep Adversarial Neural Network: LSTM-DANN maps inputs through an LSTM feature extractor, a fully connected RUL regressor, and a domain classifier operating on the shared feature vector.The model is trained to predict a real-valued RUL and a binary domain label.
  • 3.4. LSTM Deep Adversarial Neural Network: Training combines source-domain regression with adversarial domain classification to produce features that remain discriminative for RUL prediction while reducing domain differences.A gradient reversal layer changes the gradient sign during updates, enabling the adversarial objective.

4. Design of Experiments

The experiments evaluate domain adaptation across four C-MAPPS turbofan degradation datasets that differ in operating conditions and fault modes. Preprocessing, RUL construction, and two target-domain performance metrics define the experimental setup.

  • 4.1. C-MAPPS Datasets: The C-MAPPS benchmark contains four datasets with turbofan degradation data from 21 sensors and 3 operational settings.Training records include run-to-failure information, whereas testing records terminate before failure.
  • 4.1. C-MAPPS Datasets: The experiments treat each C-MAPPS dataset as a source or target domain and perform adaptation across different source-target pairs.The setup uses run-to-failure data from the source and evaluates transfer to datasets with different conditions or fault modes.
  • 4.1. C-MAPPS Datasets: FD001 and FD003 use one operating condition, while FD002 and FD004 use six operating-condition combinations.The combinations vary altitude, throttle resolver angle, and Mach.
  • 4.2. Data Preprocessing: The inputs are individually min-max normalised to (0-1), with normalisation performed separately for each dataset.The procedure is intended to increase domain discrepancy and support gradient-based adaptation.
  • 4.2. Data Preprocessing: Training RUL targets use a piece-wise linear degradation model with an initial constant period of Re = 125 cycles.The paper states that the choice of Re affects prediction performance and could be further optimised.
  • 4.2. Data Preprocessing: Near failure, FD001-FD003 and FD002-FD004 have smaller distribution shifts because each pair shares operating conditions, although differing fault modes still create shifts.The paper notes that these shifts can make models data-specific across datasets.
  • 4.3. Performance Metrics: Performance is measured with RMSE and a scoring function that penalises positive RUL errors more heavily than negative errors.RMSE estimates target prediction accuracy, while the scoring metric reflects the prognostic impact of error direction.

5. Training and Hyperparameter Selection

Training uses source labels and unlabeled target sensor data with shared LSTM-DANN feature extraction. Hyperparameters are selected without target RUL labels using cross-validation and domain-classification criteria.

  • 5.1. Training Procedure: Each training run uses one C-MAPPS dataset as source and the other three as targets, with source RUL labels but only target sensor inputs.Both domains undergo individual normalisation and a time-window transformation with Tw = 30.
  • 5.1. Training Procedure: The architecture uses LSTM and fully connected hidden layers, ReLU activation, and SGD or RMSProp weight updates.The implementation varies architectural and optimisation settings during model development.
  • 5.1. Training Procedure: The LSTM-DANN shares feature-extraction layers between a source-domain RUL regression model and an adversarial domain model.The adversarial model uses a gradient reversal layer to swap gradient signals and learn domain-invariant features.
  • 5.1. Training Procedure: Training lasts up to 200 epochs and stops early after 20 epochs without improvement.The learning rate is reduced by a factor of 0.1 after 100 epochs, and SGD gradients are clipped at norm 1.
  • 5.2. Hyperparameter Selection: Grid search tunes the optimiser and learning rates, while remaining parameters are manually fine-tuned.Validation avoids target-domain RUL labels and uses cross-validation error together with domain-classification performance.

6. Experimental Results

Across C-MAPPS cross-domain experiments, LSTM-DANN generally improved RUL prediction over SOURCE-ONLY models, especially when source data covered diverse operating conditions and fault modes. Benefits were smaller for similar domains, while transfer remained harder when source data were less diverse than the target.

  • Experimental setup: 12 experiments averaged over 10 trials evaluated each C-MAPPS dataset as source against the other datasets as targets.Performance was reported using mean and standard deviation for each model.
  • Comparison to non-adapted models: For FD004 as source, LSTM-DANN reduced target error relative to SOURCE-ONLY on FD001 and FD003, while FD002 showed similar predictions because distribution shift was low.FD004’s six operating conditions and two fault modes supported correspondences across source-target pairs.
  • Comparison to non-adapted models: LSTM-DANN yielded lower RMSE than SOURCE-ONLY for FD003-to-FD002 and FD003-to-FD004 despite the greater domain differences.For FD003-to-FD001, the domains were more similar and results were better than for the other transfer directions.
  • Comparison to non-adapted models: LSTM-DANN improved performance over almost all but one SOURCE-ONLY method in the experiments.Adapted-model performance depended on the information contained in the source dataset.
  • Comparison to non-adapted models: FD004 produced the best results as a source because it contained six operating conditions and two fault modes.This broader source coverage helped the method find correspondences between source-target operating conditions and fault modes.
  • Comparison to non-adapted models: When source data contained the target data, adaptation achieved lower RMSE and a better fit than SOURCE-ONLY models.Transfer from fewer operating conditions and fault modes to more varied targets was harder, although estimates still improved over SOURCE-ONLY.
  • Comparison to domain adaptation approaches: Compared with TCA and CORAL, the proposed method achieved lower average RMSE in all but one experiment pair.TCA and CORAL were adapted using time-indexed features because they are not tailored to temporal data.
  • Comparison to domain adaptation approaches: The LSTM structure and adversarial classification loss extracted temporal features from multivariate time series for source-target adaptation.The results support use when observed RUL data are limited in one domain and target conditions or fault modes differ.

7. Discussion

Standardising inputs improves SOURCE-ONLY performance, but LSTM-DANN-STD still outperforms the baseline models on target data. Standardised training also causes faster saturation and overfitting, which can reduce adaptation performance.

  • Zero-mean, unit-variance standardisation aligns each input feature by subtracting its local mean and dividing by its standard deviation.
  • The experiment compares SOURCE-ONLY, TARGET-ONLY, and LSTM-DANN-STD on standardised target cross-validation datasets using RMSE.
  • SOURCE-ONLY models achieve considerably improved average RMSE performance after zero-mean, unit-variance normalisation.
  • LSTM-DANN-STD still outperforms the baseline models and provides a better fit to target data after standardisation.
  • Standardised training makes LSTM-DANN-STD models saturate and overfit faster, negatively affecting adaptation performance on FD003.

8. Conclusion

The paper proposes LSTM-DANN to transfer RUL prediction from labelled source data to unlabelled target data under distribution shift. Experiments support lower target errors than non-adapted models, while identifying incomplete life-span data, computational burden, and limited hyperparameter search as boundaries.

  • LSTM-DANN combines LSTM temporal feature extraction with domain adversarial learning for RUL transfer from labelled source data to unlabelled target data.
  • The method uses time windows to incorporate long-term sequences into the feature extraction layers.
  • The proposed method generally achieves lower prediction errors than models without adaptation features, without using target RUL values.
  • Transfer is more effective from datasets with more fault modes or operating conditions than their target counterparts, while the reverse direction is harder.
  • Real-world online applications often lack complete equipment life-span data, motivating adaptation methods that accommodate incomplete target observations.
  • Further architecture optimisation is needed because hyperparameter search was restrained, while larger datasets may become computationally intractable.
Loading 1907.07480v1…