PaperlayerView exact revision on arXiv

Source-linked AI summary

Factorized axis convolutional gated recurrent unit with dynamic adaptive pooling for remaining useful life prediction of rolling bearings

Hanbyeol Park, Jungho Choo, Hyerim Bae

arXiv:2609.30972v1cs.AI

TL;DR

Conventional CNN kernels and global average pooling can miss axis-specific TFR patterns and dilute localized activations relevant to bearing RUL prediction. FAAC-GRU combines anisotropic convolution, attention-based adaptive pooling, recurrent modeling, and MC dropout; across two datasets, it reports lower mean prediction errors than baseline models.

  • Problem

    Conventional CNN square kernels lack axis-specific scale priors, and GAP can dilute localized activations regardless of degradation relevance.

    Source: §1 · p. 2

  • Method

    FAAC-GRU combines anisotropic convolution, dual-axis attention, dynamic adaptive pooling, GRU temporal modeling, and MC dropout for uncertainty estimation.

    Source: §1 · p. 3

  • Results

    FAAC-GRU achieved the lowest mean RMSE and Score on both evaluated datasets, and lower reported mean prediction errors than baselines across operating conditions.

    Source: §4.4 · p. 19, §6 · p. 27

  • Takeaways & Limitations

    The evaluations support FAAC-GRU's predictive performance across the two bearing datasets and operating conditions studied.

    Source: §6 · p. 27

    FAAC-GRU had the highest operation count and longest measured inference time among evaluated models.

    Source: §4.4 · p. 21

Abstract

from arXiv · show

Convolutional neural networks (CNN) are widely used to predict the remaining useful life (RUL) of rolling bearings from time-frequency representations (TFRs) of vibration signals. However, during degradation, characteristic structures in TFRs align predominantly along the frequency or time axis, making it challenging for conventional CNN isotropic kernels to capture directional structure. Furthermore, global average pooling (GAP) averages across axes, potentially obscuring the locations and concentrations of salient activations. This study introduces a factorized-axis convolutional gated recurrent unit (GRU) that employs multiscale anisotropic convolution and dual-axis convolution block attention module to enhance directional features and highlight salient time-frequency regions. Dynamic adaptive pooling (DAP) adaptively aggregates the time-frequency-axis information from the extracted feature maps, whereas a GRU captures temporal dynamics in the latent representations and Monte Carlo dropout enables predictive uncertainty estimation. Experiments on two public bearing datasets demonstrate that the proposed model outperforms existing RUL prediction methods across operating conditions. Ablation experiments demonstrate that the factorized axis-wise design achieves lower mean errors than convolutional isotropic kernels. DAP yields clear improvements on one dataset while matching GAP on the other, highlighting the importance of anisotropic feature extraction and adaptive feature aggregation for TFR-based RUL prediction.

1 Introduction

Accurate bearing RUL prediction supports reliable machinery operation and safety, but conventional CNN kernels and GAP can miss directional patterns and dilute localized activations. FAAC-GRU combines anisotropic feature extraction, attention-based aggregation, recurrent temporal modeling, and dropout-based uncertainty estimation.

1 Introduction

  • Accurate bearing RUL prediction is presented as important for reliable, efficient machinery operation and safety.Bearing degradation can culminate in failure and contribute to machinery breakdowns.Source: §1 · pp. 1–2
  • TFRs jointly localize spectral energy over time and frequency, making them suitable for nonstationary bearing vibration, while fixed-window STFT imposes a global resolution trade-off.Prior TFR-based approaches combine CNNs with temporal models and attention, or use CWT with degradation modeling and other deep architectures.Source: §1 · p. 2
  • Conventional square kernels do not encode axis-specific scale priors, while GAP can dilute localized activations regardless of their relevance to degradation.The introduction motivates axis-wise convolution and input-dependent weighted aggregation for TFR-based bearing RUL prediction.Source: §1 · pp. 2–3
  • FAAC-GRU combines multiscale anisotropic convolution, dual-axis attention, and dynamic adaptive pooling with a GRU for temporal modeling and MC dropout for uncertainty estimation.Its shared FAAC block extracts segment-wise TFR embeddings, which are concatenated into record-level representations before the GRU and regression head.Source: §1 · p. 3

2 Preliminary

The preliminary section introduces the CWT representation, sequential channel and spatial attention, generalized axis-wise pooling, and the MC dropout basis for uncertainty estimation.

2.1 Continuous wavelet transform

  • CWT represents signals using translated and scaled mother wavelets to examine localized variations across scales; this study uses Morlet-wavelet coefficient magnitudes as scalograms.The scalograms are the input representation used by the proposed model.Source: §2.1 · p. 4

2.2 Convolutional block attention module

  • CBAM sequentially applies channel attention to weight feature channels and spatial attention to emphasize locations, using pooled descriptors and learned attention maps.The channel module uses both average and max pooling; axis-wise average and max pooling generalize aggregation across channel, height, or width axes.Source: §2.2 · pp. 4–5

2.3 Monte Carlo dropout

  • MC dropout keeps dropout active across stochastic inference passes, approximating weight uncertainty and propagating it to predictions.The framework treats the exact weight posterior as intractable and approximates it with a variational distribution induced by dropout masks.Source: §2.3 · pp. 5–6
  • With fixed observation variance, the Gaussian negative log-likelihood is proportional to squared-error loss up to an additive constant.The practical dropout objective approximates the KL term using weight decay.Source: §2.3 · p. 6

3 Methodology

The methodology converts bearing vibration records into ordered segment-level scalograms and processes them through a shared FAAC feature extractor and GRU for RUL prediction. The feature extractor combines multiscale anisotropic convolution, dual-axis attention, and adaptive pooling; EWMA smoothing and MC-dropout inference are applied to predictions.

3.1 Data preprocessing: segment-wise input construction

  • Each vibration record is divided into five contiguous segments, transformed independently into magnitude scalograms, and stacked in original order; the latest L observations form a causal input window.When fewer than L observations are available, the first observation is repeated at the beginning of the window.Source: §3.1 · pp. 7–8

3.2 Shared FAAC feature extractor

  • The shared FAAC extractor maps each segment scalogram to an embedding using anisotropic convolution, sequential channel/spatial refinement, and frequency-to-time and time-to-frequency pooling paths.Its parameters are shared across observation periods and segments.Source: §3.2 · p. 9

3.2.1 Multi-scale anisotropic convolution

  • MSAC uses three parallel receptive-field branches, each applying a frequency-elongated anisotropic kernel followed by a local-time-elongated kernel.Branch outputs are concatenated and fused with a 1 × 1 convolution before 2 × 2 max pooling.Source: §3.2.1 · pp. 9–10

Dual-axis convolutional block attention module

  • DCBAM retains sequential channel–spatial attention but replaces the single spatial convolution with parallel anisotropic convolutions along frequency and local time.The directional responses are concatenated and fused to form a spatial attention map applied to the channel-refined feature map.Source: Dual-axis convolutional block… · pp. 10–11

Dynamic adaptive pooling

3.3 Ordered segment fusion and sequence construction

  • At each observation, the shared extractor embeds each segment and concatenates the embeddings in their original segment order to form a record-level representation.Representations from the latest L observations are arranged in acquisition order as the GRU input sequence.Source: §3.3 · p. 13

3.4 GRU-based temporal modeling and RUL regression

  • A unidirectional GRU processes the observation sequence, and its normalized final hidden state is passed to a regression head with fully connected layers, ReLU, dropout, and sigmoid outputs.Training uses mean squared error on predicted versus target RUL.Source: §3.4 · pp. 13–14

3.5 RUL smoothing and MC-dropout inference

  • EWMA smooths the RUL predictions, while MC-dropout inference keeps trained parameters fixed and enables dropout in the regression head across stochastic forward passes.The resulting smoothed samples are used to calculate pointwise dropout uncertainty intervals.Source: §3.5 · p. 14

4 Experiments

Experiments evaluated FAAC-GRU on FEMTO-ST and XJTU-SY bearing datasets using RUL error metrics and comparisons with five deep-learning models. FAAC-GRU achieved the lowest mean RMSE and Score across both datasets, while its measured computational cost was highest among the evaluated models.

FEMTO-ST bearing dataset

  • The FEMTO-ST dataset comprised 17 run-to-failure experiments under three operating conditions, with vibration sampled every 10 seconds.Signals were sampled at 25.6 kHz for 0.1 seconds per acquisition.Source: FEMTO-ST bearing dataset · p. 15, Figure 7 · p. 16

XJTU-SY bearing dataset

  • The XJTU-SY dataset contained 15 complete run-to-failure bearing records from accelerated degradation experiments under three operating conditions.Signals were sampled at 25.6 kHz for 1.28 seconds per record, at one-minute intervals.Source: XJTU-SY bearing dataset · p. 16

4.2 Evaluation metric

  • Evaluation used MAE, RMSE, and a Score metric that penalizes later-than-actual predicted failure more heavily than earlier predictions for the same absolute error.Lower Score values indicate better performance.Source: §4.2 · p. 16

4.3 Experimental settings

  • FB splits followed the official PHM 2012 challenge guidelines, while XB evaluation was limited to Conditions 1 and 2 and outer-race-fault bearings using leave-one-out cross-validation.The comparison included five deep-learning models mapping their respective input representations directly to RUL.Source: §4.3 · p. 17, Table 2 · p. 17, Table 3 · p. 17

4.4 Experimental results

  • FAAC-GRU consistently achieved the lowest mean prediction errors across all three FB conditions and both evaluated XB conditions.Some other models achieved lower errors on individual test bearings.Source: Table 5 · p. 18, Table 6 · p. 18, §4.4 · p. 19
  • FAAC-GRU achieved the lowest mean RMSE and Score on both datasets, while comparison-model rankings varied by dataset.Table 9 compares mean RMSE and Score across the FB and XB test bearings.Source: Table 9 · p. 19, §4.4 · p. 19
  • For the selected bearings, FAAC-GRU tracked decreasing RUL trajectories through signal fluctuations, while comparison models sometimes overestimated, underestimated, or fluctuated.For FB2-3, several comparison models fluctuated near an abrupt raw-signal change around time step 250.Source: §4.4 · p. 20, Figure 8 · p. 20, Figure 9 · p. 21
  • FAAC-GRU used 0.939 M parameters and 9.814 G FLOPs, with a 1.306 ms mean inference time per sample—the highest operation count and longest measured inference time.The authors identify this computational cost as a limitation for resource-constrained deployment.Source: §4.4 · p. 21, Table 10 · p. 21

5 Ablation study

The ablations compare attention modules, feature-map aggregation, and isotropic versus anisotropic convolution. DCBAM maintained or improved mean performance, DAP improved results on FB but was not uniformly better on XB, and anisotropic kernels generally reduced errors with some bearing-level exceptions.

5.1 Channel block attention module design

  • Replacing CBAM with DCBAM maintained or improved mean performance across condition groups, with the most significant improvements under FB Conditions 2 and 3.The authors note that mean-error differences were generally modest and benefits were not uniform across individual bearings.Source: §5.1 · p. 22, Table 11 · p. 22, Table 12 · p. 22

5.2 Feature map aggregation design

  • DAP added only marginal computational cost relative to GAP, whereas FLAT required the most parameters and operations.DAP used 0.939 M parameters and 9.814 G FLOPs; GAP used 0.905 M parameters and 9.811 G FLOPs.Source: §5.2 · pp. 22–23, Table 13 · p. 23
  • DAP improved aggregation results on FB, but its advantage was not consistent across bearings and FLAT achieved the lowest errors under XB Condition 2.Under FB Condition 1, DAP achieved MAE 0.076 and RMSE 0.102, compared with 0.105 and 0.139 for GAP.Source: Table 14 · p. 23, Table 15 · p. 23, §5.2 · p. 24
  • DAP produced input-dependent aggregation, with attention sometimes near-uniform and sometimes selective in relation to feature-map activations.For the illustrated FB examples, frequency-axis weights varied more than time-axis weights.Source: §5.2 · pp. 24–25

5.3 Convolution kernel design

  • ANISO generally achieved lower errors than ISO across the evaluated FB and XB conditions, although ISO performed better on FB2-6, FB3-3, and XB2-4.For FB1-4, ANISO achieved MAE 0.037 and RMSE 0.054, compared with 0.147 and 0.168 for ISO.Source: §5.3 · p. 25, Table 16 · p. 26, Table 17 · p. 26
  • ANISO Grad-RAM maps were more concentrated in specific time-frequency regions than the more spatially diffuse ISO maps.The maps partially coincided with high-intensity regions in the input scalograms.Source: §5.3 · pp. 26–27

6 Conclusion

FAAC-GRU combines directional feature extraction, adaptive aggregation, and temporal modeling to predict bearing RUL from TFRs. It outperformed baseline models across tested operating conditions, while the conclusion notes computational and input-representation limits.

6 Conclusion

  • FAAC-GRU achieved lower reported mean prediction errors than baseline models across operating conditions on two public bearing datasets.The conclusion reports that gains were not uniform across individual bearings.Source: §6 · p. 27
  • Anisotropic convolutions, DCBAM, and DAP extract localized time–frequency patterns, followed by GRU modeling of temporal dependencies and MC-dropout prediction intervals.Source: §6 · p. 27
  • Ablations highlighted anisotropic kernels and DAP, especially on the FB dataset, while DCBAM produced only marginal mean-performance improvements.Source: §6 · p. 27
  • FAAC-GRU used higher FLOPs and longer inference times than comparison models, despite millisecond-scale inference on the tested GPU.End-to-end real-time performance under target deployment conditions remains for future investigation.Source: §6 · p. 27
  • TFR-only inputs may discard phase information and fine-scale signal details through magnitude representation and resizing.Future work will explore multi-domain fusion to integrate complementary representations while mitigating reliance on one domain.Source: §6 · p. 27

A.1 Non-injectivity of GAP and input-adaptive weighting of DAP

GAP can map spatially distinct activation patterns to the same output because it retains only channel-wise spatial means. DAP instead uses input-dependent axis-wise weights, with uniform weights reducing its time-to-frequency path to GAP.

A.1 Non-injectivity of GAP and input-adaptive weighting of DAP

  • GAP assigns equal weight to every spatial location, so its output preserves each channel's spatial mean but not activation locations or concentrations.Distinct concentrated and diffuse maps can have identical GAP outputs when their per-channel sums match.Source: §A.1 · p. 30
  • DAP's time-to-frequency path first aggregates over time, then frequency, using axis-specific softmax weights computed from the input.Source: §A.1 · pp. 30–31
  • The path's effective spatial weights are nonnegative, sum to one, and form a separable outer product of its two axis-specific weight vectors.For a local region, DAP can assign more total weight than GAP's spatial-occupancy fraction.Source: §A.1 · p. 31
  • When both axis-specific weight vectors are uniform, the path output equals GAP; otherwise, DAP can adapt its effective spatial weights to the input.This single-path result does not establish that learned weights emphasize degradation-relevant regions or fully preserve spatial information after pooling and projection.Source: §A.1 · p. 31

A.2 Quantitative analysis of directional structure in TFRs

The analysis quantifies local directional structure in a feature map with a structure tensor built from gradient vectors in each window. Its eigenvalues characterize anisotropy, while its eigenvectors define dominant gradient and structural directions.

A.2.1 Local structure tensor coherency and dominant orientation

  • A local structure tensor is formed by summing outer products of gradient vectors within each window, using central differences on the interior grid.The outermost rows and columns are omitted because central differences cannot be computed there.Source: §A.2.1 · p. 32
  • Coherency compares the tensor's ordered eigenvalues: values near one indicate strongly anisotropic local gradient energy, while values near zero indicate similar energy along both principal directions.The measure is defined for windows with nonzero total gradient energy.Source: §A.2.1 · p. 32
  • The eigenvector for the larger eigenvalue gives the dominant gradient direction, and the smaller-eigenvalue eigenvector gives structural orientation, measured modulo a half-turn.Orientations near zero or π align with the time axis; those near π/2 align with the frequency axis.Source: §A.2.1 · p. 32

A.2.2 Quantitative results for (64, 64) TFRs

Across the analyzed 64 × 64 TFRs, local anisotropic gradient energy consistently favored frequency-axis alignment. Original TFRs also had greater local coherency than pixel-permuted references under the specified aggregation procedure.

A.2.2 Quantitative results for (64, 64) TFRs

  • 0.633, 0.593, 0.791, and 0.890 were the mean frequency-minus-time axis preference indices for FEMTO-H, FEMTO-V, XJTU-H, and XJTU-V at w = 5.The index was positive in all 180 bearing, channel, and window-size combinations, indicating frequency-axis preference in the analyzed TFRs.Source: §A.2.2 · p. 33, Figure A1 · p. 33
  • Predominant orientation near 90° across dataset–channel combinations indicated that anisotropic gradient energy favored structures aligned with the frequency axis.Figure A2 shows the orientation distribution across window sizes.Source: §A.2.2 · p. 34, Figure A2 · p. 34
  • 0.331, 0.224, 0.404, and 0.538 were the mean coherency differences for FEMTO-H, FEMTO-V, XJTU-H, and XJTU-V at w = 5.ΔC was positive across 18 bearings, two channels, and five window sizes; this comparison does not by itself establish statistical significance or that structures encode degradation.Source: §A.2.2 · p. 35, Figure A3 · p. 35

B Visualization of raw bearing vibration signals.

Figures B1 and B2 show raw vibration signals over the remaining time to failure (RtF) for the FB and XB bearings under three and two operating conditions, respectively. Across many bearings, amplitudes changed as failure approached, but trajectories were not uniformly monotonic.

B Visualization of raw bearing vibration signals.

C FAAC-GRU architecture.

Table C1 presents FAAC-GRU's layer configuration and output structures for input TFR sequences, while Table 10 presents the architecture's computational complexity.

C FAAC-GRU architecture.

Related papers

Loading 2609.30972v1…