PaperlayerView exact revision on arXiv

Source-linked AI summary

Time-Series Regeneration with Convolutional Recurrent Generative Adversarial Network for Remaining Useful Life Estimation

Xuewen Zhang, Yan Qin, Chau Yuen, Lahiru Jayasinghe, Xiang Liu

arXiv:2101.03678v1cs.LGcs.AI

TL;DR

Run-to-failure data can be insufficient for machine-learning RUL estimation, and few studies have combined time-series regeneration with hybrid RUL models. This paper pairs CR-GAN with existing RUL models to generate multivariate time series for non-cyclic and cyclic degradation. For variable S21, CR-GAN’s AUROC is 99%, compared with 95% for RCGAN.

  • Problem

    Run-to-failure data can be scarce, and few studies have combined time-series regeneration with hybrid RUL models.

    Source: §I · pp. 3, 5

  • Method

    The framework uses CR-GAN to generate multivariate time series and combines the generated data with current RUL estimation models.

    Source: §I · p. 5

  • Results

    For variable S21, CR-GAN’s AUROC is 99%, compared with 95% for RCGAN.

    Source: 1) Multivariate time-series generation · p. 16

  • Takeaways & Limitations

    The paper presents time-series self-generation as a low-cost, rapid, and safe way to enrich data for existing RUL models.

    Source: §I · p. 5

    The proposed method has slightly higher time complexity than RCGAN and T-CGAN because it uses more CNN and LSTM layers.

    Source: 1) Multivariate time-series generation · p. 17

Abstract

from arXiv · show

For health prognostic task, ever-increasing efforts have been focused on machine learning-based methods, which are capable of yielding accurate remaining useful life (RUL) estimation for industrial equipment or components without exploring the degradation mechanism. A prerequisite ensuring the success of these methods depends on a wealth of run-to-failure data, however, run-to-failure data may be insufficient in practice. That is, conducting a substantial amount of destructive experiments not only is high costs, but also may cause catastrophic consequences. Out of this consideration, an enhanced RUL framework focusing on data self-generation is put forward for both non-cyclic and cyclic degradation patterns for the first time. It is designed to enrich data from a data-driven way, generating realistic-like time-series to enhance current RUL methods. First, high-quality data generation is ensured through the proposed convolutional recurrent generative adversarial network (CR-GAN), which adopts a two-channel fusion convolutional recurrent neural network. Next, a hierarchical framework is proposed to combine generated data into current RUL estimation methods. Finally, the efficacy of the proposed method is verified through both non-cyclic and cyclic degradation systems. With the enhanced RUL framework, an aero-engine system following non-cyclic degradation has been tested using three typical RUL models. State-of-art RUL estimation results are achieved by enhancing capsule network with generated time-series. Specifically, estimation errors evaluated by the index score function have been reduced by 21.77%, and 32.67% for the two employed operating conditions, respectively. Besides, the estimation error is reduced to zero for the Lithium-ion battery system, which presents cyclic degradation.

I. INTRODUCTION

Machine-learning RUL estimation depends on run-to-failure time series, which can be scarce because failures are costly and unsafe to collect. The paper proposes CR-GAN-based time-series regeneration to augment current RUL models across non-cyclic and cyclic degradation patterns.

I. INTRODUCTION

  • Run-to-failure data remain scarce because collecting them can take months or years, while safety concerns lead to component replacement before failure.This shortage challenges training and generalization for machine-learning RUL models.Source: §I · pp. 3–4
  • Few studies had combined time-series regeneration with hybrid RUL models, and existing generators may not capture industrial series’ temporal, multivariate, and cycle-related characteristics.The authors identify non-stationarity and long-term temporal dependence as challenges for run-to-failure data.Source: §I · pp. 4–5
  • The proposed framework combines CR-GAN with existing RUL models, using a CNN-LSTM generator and an LSTM discriminator to generate realistic multivariate time series.The generator’s two-channel fusion model learns spatial and temporal evolution; the discriminator evaluates generated data.Source: §I · p. 5
  • The paper frames time-series self-generation as a low-cost, rapid, and safe way to enrich data and improve existing RUL models.It presents the approach for both non-cyclic and cyclic degradation patterns.Source: §I · p. 5
  • The framework is evaluated on NASA aero-engine and lithium-ion battery systems representing non-cyclic and cyclic degradation.The introduction identifies these as practical evaluation cases for assessing generated-series quality and RUL accuracy.Source: §I · p. 5

A. Problem Formulation of RUL Estimation

The section distinguishes one-time continuous degradation from repeated, cycle-wise degradation, then formulates RUL estimation around their corresponding data. It motivates data-driven augmentation because run-to-failure series are long relative to the available training examples.

1) RUL estimation of non-cyclic degradation pattern:

  • Non-cyclic degradation is a one-time continuous decline represented by process-variable measurements over time and a linearly decreasing RUL vector.For each run-to-failure object, the formulation pairs a data matrix with remaining-life labels.Source: 1) RUL estimation of non-cyclic… · p. 6

2) RUL estimation of cyclic degradation pattern:

  • Cyclic degradation repeats inconsistently but similarly, with performance indicators deteriorating from cycle to cycle.The battery example describes capacity loss accumulating over charging and discharging cycles.Source: 2) RUL estimation of cyclic… · p. 7
  • For cyclic degradation, each cycle has process-variable measurements and a performance indicator reflecting the cumulative influence of the whole cycle.The formulation collects run-to-failure cycles as paired inputs and indicators.Source: 2) RUL estimation of cyclic… · p. 7
  • The section focuses on sensor-based regression and excludes autoregressive RUL models, although cyclic degradation makes forecasting from past performance a promising alternative.The authors note that autoregression may require real-time performance indicators.Source: 2) RUL estimation of cyclic… · p. 7

B. Motivation

  • Both degradation patterns can be organized as a three-dimensional array indexed by component or cycle, process variable, and sample length.The component dimension represents run-to-failure objects for non-cyclic data or cycles for cyclic data.Source: B. Motivation · p. 7
  • Because complete run-to-failure series are long relative to the number of available examples, the paper motivates data-driven enrichment instead of physically collecting more cycling data.The section describes the full run-to-failure time series as the basic data unit and notes that its length can greatly exceed the number of examples.Source: B. Motivation · p. 8

III. RUL ESTIMATION WITH TIME-SERIES GENERATION

The framework prepares and segments run-to-failure data, learns degradation trajectories with CR-GAN, and combines generated sequences with real data for RUL training and online prediction.

A. Data Preparation

  • Run-to-failure data are arranged to preserve within-object structure and, for cyclic degradation, cycle order and spatial correlations across cycles.Non-cyclic data are represented as complete run-to-failure objects; cyclic data are arranged in ascending cycle order.Source: A. Data Preparation · p. 9

B. Data Segmentation

  • Sliding fixed-size windows convert each complete run-to-failure series into overlapping local trajectories, enlarging N series to N(K − L + 1).Choosing L at 80%–90% of K is suggested to enrich the series count 10–20 times without changing degradation behavior.Source: B. Data Segmentation · p. 9, Figure 3 · p. 10

C. The CR-GAN Model

  • CR-GAN combines LSTM and CNN to capture temporal correlations and similarities and dissimilarities across cycles.The framework uses the learned degradation patterns to generate similar trajectories, then mixes generated and real sequences for RUL training and online prediction.Source: §III · p. 8, C. The CR-GAN Model · p. 10

1) The structure of CR-GAN Model:

CR-GAN fuses spatial and temporal learning in its generator and uses a discriminator to assess generated sequences. Conditional inputs guide generation, while iterative training and a classifier filter unqualified series.

1) The structure of CR-GAN Model:

  • The generator integrates CNN layers for spatial features with LSTM layers for temporal correlations, while an LSTM and fully connected discriminator classifies generated versus real series.The generator’s noise vector is assumed to follow a Gaussian distribution.Source: 1) The structure of CR-GAN Model · p. 10, Figure 4 · p. 11
  • Time or cycle index serves as condition Y, which is supplied to both generator and discriminator as additional input.The condition replaces D(x) and G(z) with D(x|Y) and G(z|Y).Source: 1) The structure of CR-GAN Model · p. 11

2) Iterative solution of CR-GAN Model:

  • The discriminator objective favors generated samples classified as real and disfavors those classified as synthetic; generator and discriminator are optimized iteratively.The process continues until convergence, producing generated run-to-failure cycles.Source: 2) Iterative solution of CR-GAN Model · p. 12
  • A CNN trained on real run-to-failure series identifies irrational generated series, which are discarded before subsequent modeling.This filtering procedure uses the Train on Real, Test on Synthetic concept [21].Source: 2) Iterative solution of CR-GAN Model · p. 12

D. Enhanced RUL Model with Data Generation

The enhanced framework adds generated sequences to real training data for RUL estimation. For online use, new inputs are normalized and passed to the trained model; SF and RMSE assess estimation error.

D. Enhanced RUL Model with Data Generation

  • Generated cycles are merged with real data along the cycle direction to form Xn, which trains a selected RUL model.The augmented matrix has dimensions Nn × J × K, with Nn = Ng + N.Source: D. Enhanced RUL Model with Data… · p. 13

E. Online Estimation using the Enhanced RUL Model

  • For online estimation, new non-cyclic vectors or cyclic matrices are normalized using the training normalization information and passed to the trained RUL model.The inputs are xnew for non-cyclic components and Xnew for cyclic components.Source: E. Online Estimation using the… · p. 13
  • SF and RMSE compare estimation performance, with lower values better; SF especially penalizes delayed estimates.RMSE weights early and delayed estimation equally, and the error distribution can distinguish methods with similar RMSE.Source: E. Online Estimation using the… · pp. 13–14

IV. RESULTS AND DISCUSSIONS

The numerical case tests CR-GAN on sinusoidal and smooth signals, alongside practical cases described for aero-engine and lithium-ion battery data. In the numerical case, generated sine waves are reported as visually close to the real waves.

IV. RESULTS AND DISCUSSIONS

  • The evaluation comprises a numerical generation case and two practical cases for non-cyclic aero-engine and cyclic lithium-ion battery degradation.The practical cases assess generated-series quality and RUL estimation accuracy.Source: §IV · p. 14

A. Time-series Generation with A Numerical Case

B. RUL Estimation for Non-cyclic Degradation Pattern

The non-cyclic evaluation uses C-MAPSS datasets FD001 and FD003, which have single data distributions; CR-GAN achieves strong variable-wise generation results, with somewhat higher but not significantly increased training time.

B. RUL Estimation for Non-cyclic Degradation Pattern

  • FD001 and FD003 provide 100 training and 100 testing engines each, with 21 variables; both are single-distribution C-MAPSS datasets.The proposed CR-GAN does not account for multiple distributions, so FD002 and FD004 are excluded.Source: B. RUL Estimation for Non-cyclic… · p. 16

1) Multivariate time-series generation:

  • Generation quality is compared with RCGAN and T-CGAN using seven shared variables and a classifier-based AUROC evaluation.The shared variable set provides a fair comparison with the competing generation methods.Source: 1) Multivariate time-series generation · p. 16
  • CR-GAN’s AUROC is very close to 1 on FD001; for S21, it reaches 99% versus RCGAN’s 95%, while RCGAN’s S7 result is below 60%.For FD003, almost all CR-GAN AUROC values exceed 90%.Source: 1) Multivariate time-series generation · p. 16
  • The proposed method has slightly higher time complexity than RCGAN and T-CGAN, but its increased training time is not significant.Once trained offline, the model can be used directly in online estimation.Source: 1) Multivariate time-series generation · p. 17

2) Enhanced RUL estimation with generated time-series:

Adding CR-GAN-generated time-series improves the reported RUL estimates for TCMN, LSTM, and CapsNet on FD001 and FD003. The evaluation also compares per-engine errors and their distributions, using data prepared from the two single-distribution C-MAPSS datasets.

2) Enhanced RUL estimation with generated time-series:

  • RUL modeling uses all variables after constant-valued variables are removed, and retains samplings from the final 125 hours with a window size of 100.This preparation expands the original time-series 26-fold for basic CR-GAN training.Source: 2) Enhanced RUL estimation with… · pp. 18–19
  • CapsNet’s SF falls from 276.34 to 216.17 on FD001 and from 283.81 to 191.10 on FD003 with the proposed framework.CapsNet’s reported state-of-the-art RMSE values are 12.59 for FD001 and 11.71 for FD003.Source: 2) Enhanced RUL estimation with… · p. 20
  • TCMN’s RMSE falls from 23.57 to 18.39 on FD001 and from 21.70 to 17.92 on FD003; LSTM’s falls from 16.14 to 15.30 and 16.18 to 15.58, respectively.The reported SF also decreases for enhanced TCMN.Source: 2) Enhanced RUL estimation with… · p. 20
  • For both FD001 and FD003, enhanced-model error distributions place a higher ratio of errors in low-value zones than their counterparts.The plots show per-engine errors and histograms, with engines ordered by ascending real RUL; the histogram intervals are four units wide.Source: Figure 8 · p. 18, 2) Enhanced RUL estimation with… · p. 20

C. RUL Estimation for Cyclic Degradation Pattern

The cyclic-degradation evaluation uses NASA Ames Lithium-ion battery data, with capacity-based end-of-life and RUL definitions. Battery current and voltage are used for capacity inference.

C. RUL Estimation for Cyclic Degradation Pattern

  • The NASA Ames Lithium-ion battery data are used to test the enhanced framework on cyclic degradation.The passage describes the battery data as cycling run-to-failure time-series and an ideal benchmark for this pattern.Source: C. RUL Estimation for Cyclic… · p. 21
  • A battery is considered due for replacement when real capacity falls to an end-of-life threshold, usually 70% of rated capacity.The passage states that RUL is the difference between predicted end of life and its real value.Source: C. RUL Estimation for Cyclic… · p. 21
  • Discharging current and voltage can be measured to infer capacity for RUL estimation.Source: C. RUL Estimation for Cyclic… · p. 21

1) Multivariate time-series generation:

The cyclic evaluation uses battery current, voltage, and capacity signals, with time index as the condition information. Adding generated cycles improves the reported LSTM RUL estimates, including a zero RUL error for Battery B0018 in one setting.

1) Multivariate time-series generation:

  • The selected datasets are B0005, B0006, and B0018; 150 cycling data located in the first 50 cycles are generated.Current, voltage, and ground-truth capacity are generated variables, and time index is the condition information.Source: 1) Multivariate time-series generation · p. 21

2) Enhanced RUL estimation with generated time-series:

  • The evaluation uses the LSTM structure from, taking current and voltage signals as inputs and predicting battery capacity.It considers three generated-data amounts: 50, 100, and 150.Source: 2) Enhanced RUL estimation with… · p. 21
  • RUL estimation errors decrease with generated time-series, and Battery B0018’s RUL error reaches zero with 150 generated cycles.Both RMSE and RUL errors are reported as significantly reduced; the results consider 50, 100, and 150 generated time-series.Source: 2) Enhanced RUL estimation with… · p. 21
  • The authors report reduced RUL estimation errors for both cyclic and non-cyclic degradation systems with regenerated run-to-failure time-series.Source: 2) Enhanced RUL estimation with… · pp. 21–22

V. CONCLUSION AND FUTURE WORK

The paper presents time-series generation as a self-learning enhancement for RUL models and reports experiments on aero-engine and Lithium-ion aging data. It identifies unresolved choices in statistical constraints, noise distributions, and filtering irrational sequences.

V. CONCLUSION AND FUTURE WORK

  • Experiments on aero-engine and Lithium-ion aging data demonstrate the framework's strengths across diverse evaluation indices.The conclusion describes the work as verifying CR-GAN feasibility for generating time-series to improve RUL estimation.Source: §V · p. 22
  • The framework synthesizes realistic multivariate time-series with CR-GAN to enrich training for existing RUL models.The paper studies non-cyclic and cyclic degradation patterns and compares current models with enhanced versions.Source: §V · p. 22
  • The original GAN objective may not preserve all run-to-failure time-series statistics, motivating statistical constraints for more reliable generation.Source: §V · p. 22
  • The authors identify testing noise statistics beyond the commonly used Gaussian distribution as a future research direction.Source: §V · p. 22
  • The authors recommend filtering irrational time-series during adversarial learning, potentially with a discriminator that uses engineers' domain knowledge.The paper notes that irrational sequences can currently be filtered through post-processing.Source: §V · p. 22

Algorithm 1: Iterative solution of CR-GAN

The listed CR-GAN procedure uses normalized real run-to-failure series and Gaussian noise, sampling noise and time indices to generate time-series. The associated tables identify configuration and comparison scopes.

Algorithm 1: Iterative solution of CR-GAN

Loading 2101.03678v1…