Source-linked AI summary
Predicting Remaining Useful Life using Time Series Embeddings based on Recurrent Neural Networks
Narendhar Gugulothu, Vishnu TV, Pankaj Malhotra, Lovekesh Vig, Puneet Agarwal, Gautam Shroff
TL;DR
Sensor-based RUL estimation must handle uncertain degradation patterns, noisy readings, and missing values. Embed-RUL uses sequence-to-sequence RNNs to learn embeddings from multivariate time-series subsequences, and the authors report favorable performance across turbofan and pump datasets. The embeddings capture machine behavior while filtering noise and distinguish normal from degraded operation.
Problem
Sensor-based RUL methods may assume degradation trends, while practical sensor data can be noisy and contain missing values.
Method
Embed-RUL trains a sequence-to-sequence RNN Encoder-Decoder to generate fixed-dimensional embeddings for multivariate time-series windows.
Results
Embed-RUL embeddings are useful for RUL estimation, robust under noisy conditions, and favorable against prior benchmarks on turbofan and pump datasets.
Takeaways & Limitations
The embeddings capture important machine-behavior patterns while filtering noise, supporting comparisons between normal and degraded behavior for RUL estimation.
Takeaways & Limitations
The paper identifies degradation-trend assumptions and omitted temporal information as shortcomings of conventional health-index mappings.
Abstract
from arXiv · showhide
We consider the problem of estimating the remaining useful life (RUL) of a system or a machine from sensor data. Many approaches for RUL estimation based on sensor data make assumptions about how machines degrade. Additionally, sensor data from machines is noisy and often suffers from missing values in many practical settings. We propose Embed-RUL: a novel approach for RUL estimation from sensor data that does not rely on any degradation-trend assumptions, is robust to noise, and handles missing values. Embed-RUL utilizes a sequence-to-sequence model based on Recurrent Neural Networks (RNNs) to generate embeddings for multivariate time series subsequences. The embeddings for normal and degraded machines tend to be different, and are therefore found to be useful for RUL estimation. We show that the embeddings capture the overall pattern in the time series while filtering out the noise, so that the embeddings of two machines with similar operational behavior are close to each other, even when their sensor readings have significant and varying levels of noise content. We perform experiments on publicly available turbofan engine dataset and a proprietary real-world dataset, and demonstrate that Embed-RUL outperforms the previously reported state-of-the-art on several metrics.
1 INTRODUCTION
Sensor-based RUL estimation is valuable but faces uncertain degradation trends and noisy readings. Embed-RUL uses RNN-derived embeddings to address these challenges and performs favorably against prior benchmarks.
- Sensor data supports machine health monitoring tasks including anomaly detection, fault detection, and RUL estimation.
- Many data-driven methods assume a degradation trend, but complex machines may exhibit intermittent symptoms followed by non-exponential growth.
- Embed-RUL encodes recent multivariate sensor history into fixed-dimensional embeddings and compares them with embeddings from normal behavior.
- The approach avoids degradation-trend assumptions, handles noise and missing values, and captures complex temporal dependencies among sensors.
- Embed-RUL embeddings are reported useful for RUL estimation, robust under noisy readings, and favorable against prior benchmarks on turbofan and pump datasets.
2 RELATED WORK
Prior work applies diverse models to sensor-based RUL estimation, robust health monitoring, and time-series representation learning. Embed-RUL combines these themes by using RNN-based embeddings specifically for machine health monitoring and RUL estimation.
- Earlier approaches estimate RUL directly from sensor relationships using similarity methods, support vector regression, RNNs, or convolutional neural networks.
- Existing robust-estimation methods use wavelet filters or ensembles, whereas Embed-RUL learns robust representations through RNN Encoder-Decoder models.
- RNN-based unsupervised representation learning has been applied across text, video, speech, and time-series domains.
- Related models include LSTM and GRU encoders, denoising autoencoders, and time-series embeddings for classification or anomaly detection.
- The paper positions Embed-RUL as the first use of RNN-based multivariate sensor embeddings for machine health monitoring and RUL estimation.
3 BACKGROUND
Traditional health-index methods often rely on assumed degradation curves and ignore temporal sensor structure. RNNs and autoencoders provide alternatives by modeling temporal dependencies and learning representations that retain patterns while reducing noise.
- Health-index curve matching estimates RUL by comparing a test machine’s trajectory with trajectories from failed training instances.
- A simple health-index mapping uses current sensor readings, but such methods assume a degradation trend and omit temporal information.
- RNN-based health monitoring captures complex temporal and instantaneous dependencies between sensor readings.
- Autoencoder hidden representations are intended to retain important input patterns while ignoring noise.
- RNN autoencoders can estimate health and RUL from reconstruction error by learning normal multivariate time-series behavior without supervision.
- The proposed sequence-to-sequence RNN autoencoder uses GRU-based layers to learn fixed-dimensional representations of time-series windows.
4 RUL ESTIMATION USING EMBEDDINGS
Embed-RUL converts multivariate sensor windows into fixed-dimensional RNN embeddings, using sequence reconstruction to capture machine behavior while filtering noise. It estimates health by comparing embeddings with normal behavior and estimates RUL by matching the resulting health curve to failed training instances, while incorporating missingness information.
- Obtaining embeddings: Sensor histories are divided into fixed-length windows, and an unsupervised RNN Encoder produces one fixed-dimensional embedding for each window.The embeddings represent windows from both normal and faulty operation.
- Obtaining embeddings: The encoder-decoder maps each multivariate time-series window to an embedding and reconstructs the window through a decoder initialized from the encoder state.Removing decoder inputs beyond the encoder state forces that state to retain the information needed for reconstruction.
- Handling missing values: Masking and delta vectors identify missing sensors and elapsed time since recent observed values, while training reconstructs only available sensor readings.The loss is modified so missing sensors do not contribute incorrect reconstruction penalties.
- Obtaining HI curves: Health is estimated as the distance from a window embedding to its nearest normal embedding, producing a health-index curve over time.The paper notes that this health index is inverse-scaled: lower values indicate normal health and higher values indicate poor health.
- RUL estimation: RUL is estimated by comparing a test instance’s health-index curve with failed training curves while allowing a time lag between their initial health states.The candidate RUL is determined from the matched training instance’s remaining operational duration after accounting for the lag.
5 EXPERIMENTAL EVALUATION
The evaluation uses a public turbofan engine dataset and a proprietary real-world pump dataset to compare embedding-distance and reconstruction-error approaches for RUL estimation and assess robustness.
- Embed-RUL is evaluated on the C-MAPSS Turbofan Engine dataset and a proprietary real-world pump dataset.The experiments compare embedding-distance approaches with previously reported reconstruction-error approaches.
5.1 Datasets Description
The experiments use simulated turbofan engines and real-world pumps with different sensor configurations, operational histories, and missing-data conditions. Validation and implementation choices are used to configure the models and evaluation.
- Datasets: The engine dataset contains 24-sensor trajectories for 100 training and 100 test turbofan engines, with test trajectories truncated before failure.Actual RUL values are available for the test instances.
- Datasets: The engine training set is split into 80 training and 20 validation instances, with truncated validation trajectories used for RUL estimation.PCA selects the number of components based on the validation set.
- Datasets: The pump dataset contains hourly readings from 38 failed and 24 operational pumps over 2.5 years, with seven sensors per pump.Failed instances are split into training, validation, and test sets, while operational instances are used only for training and validation.
- Preprocessing: Daily pump data produces 28 derived sensors from four daily statistics per sensor and reduces missing readings from 45% to 33%.Masking and delta vectors are added as inputs to handle missing values.
- Configuration: Model hyperparameters include representation size, RNN depth and width, dropout, window length, time lag, similarity threshold, maximum RUL, and λ.Window length can be selected using domain knowledge in practice.
5.2 Embeddings for RUL Estimation
Embedding-distance representations are compared with reconstruction-error and regression baselines on engine and pump datasets. The reported results favor embedding-based approaches, while t-SNE analysis separates normal and degraded windows.
- Experimental comparison: The evaluation compares Embed-RUL with Recon-RUL, embedding- and reconstruction-based linear regression models, and an RNN regression model.Embed-LR1 and Embed-LR2 use normalized or squared normalized embedding-distance health-index curves as regression targets.
- Engine dataset: Each embedding-distance variant achieves a better timeliness score S than its corresponding reconstruction-error variant on the engine dataset.Error ranges are also more spread out for reconstruction-error models than for embedding-distance models.
- Reported evaluations: The reported engine and pump comparisons use performance tables and plots of actual versus estimated RUL values and prediction-error distributions.The pump table reports MSE and MAE because standard τ1 and τ2 values are unavailable for that dataset.
- Pump dataset: The pump Embed-RUL model performs significantly better than the other evaluated approaches and is approximately 35% better than Recon-RUL.Linear-regression approaches perform worse than raw embedding-distance or reconstruction-error approaches, indicating the importance of temporal sensor information for HI estimation.
- Embedding analysis: t-SNE maps the original engine and pump embeddings to two dimensions to compare windows from normal and degraded portions of machine life.The analysis examines the first 25% and last 25% of test-instance trajectories.
5.3 Robustness of Embeddings to Noise
The robustness evaluation adds Gaussian noise to sensor readings and compares Embed-RUL with Recon-RUL. Embedding-distance models show lower variability across noise levels and more stable health-index estimates.
- Noise evaluation: Gaussian noise is added to test-instance sensor readings to evaluate the robustness of Embed-RUL and Recon-RUL.The noisy reading is formed by corrupting the original sensor reading with additive Gaussian noise.
- Performance under noise: The standard deviation of MSE across noise levels is much lower for Embed-RUL than for Recon-RUL on both datasets.The results suggest greater noise robustness for embedding-distance models.
- Performance under noise: 819±41 versus 1189±110: engine timeliness score S is lower for Embed-RUL than for Recon-RUL under the noise evaluation.These values are reported for Embed-RUL and Recon-RUL, respectively.
- Health-index behavior: Embedding-distance health-index estimates remain fairly stable as noise varies, whereas reconstruction-error health indices vary significantly.Both reconstruction error and embedding distance increase over time, indicating gradual degradation in the illustrated scenario.
6 DISCUSSION
Embed-RUL uses unsupervised RNN Encoder-Decoder embeddings for health-index and RUL estimation, addressing noisy readings, missing data, and unknown degradation trends. Embedding-distance estimates outperform reconstruction-error comparisons and remain robust to noise across engine and pump datasets.
- Embed-RUL addresses noisy sensor readings, missing data, and the absence of prior degradation-trend knowledge.
- An unsupervised RNN Encoder-Decoder learns fixed-dimensional embeddings that capture machine behavior.
- Health is estimated by comparing recent embeddings with embeddings representing normal behavior.
- Embedding distances outperform RNN Encoder-Decoder reconstruction error on the engine dataset and produce better results on the real-world pump dataset.
- Embedding-distance RUL estimates are robust to noise.
A PERFORMANCE METRICS
The paper evaluates prognostics models using timeliness, accuracy, error, and false-classification metrics. Timeliness penalizes prediction errors asymmetrically, while false positives and negatives are defined by separate error thresholds.
- Performance is measured with Timeliness Score, Accuracy, MAE, MAPE, FPR, and FNR.
- Timeliness Score uses the difference between estimated and actual RUL to evaluate prediction performance.
- Late predictions are penalized more than early predictions when τ1 > τ2, and lower Timeliness Score indicates better performance.
- The indicator I(Δ(u*)) equals one when the error lies within [−τ1,τ2] and zero otherwise.
- A prediction is false positive when Δ(i*) < −τ1 and false negative when Δ(i*) > τ2.
B BENCHMARKS ON TURBOFAN ENGINE DATASET
The paper compares RUL-estimation approaches on the turbofan engine dataset, with results presented for test FD001.txt. The comparison includes reported values and methodological notes about dataset settings and parameter tuning.
- The benchmark compares approaches for RUL estimation on the turbofan engine dataset using test FD001.txt.
- Table 4 reports the performance of various approaches on turbofan engine data and marks unavailable values as NR.
- A footnote states that one comparison method tunes parameters on the test set, whereas this paper learns parameters on a validation set.
- The same footnote reports similar Timeliness Score performance despite the different parameter-tuning procedures.