PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does Domain Adaptation Help on Physical Vibration Sensors? A Held-Out-Bearing Study of Neural-Operator and Convolutional Models

Kumbha Nagaswetha, Rabi Pathak

arXiv:2609.31639v1cs.LGcs.AI

TL;DR

High reported bearing-diagnosis accuracy often comes from splits that place the same physical bearing in training and test, leaving transfer to unseen bearings difficult to assess. The study evaluates held-out bearings and compares order tracking with frequency-domain inputs and matched-feature architectures. It finds that representation, rather than the alignment method, determines whether adaptation helps in this task, while the protocol also exposes failures on unseen fault geometries.

  • Problem

    Window-level splits can place the same physical bearing in training and test, making it difficult to assess transfer to unseen bearings despite frequently reported accuracy above 99 percent.

    Source: §1 · p. 1

  • Method

    The study uses a held-out-bearing protocol and compares order tracking with frequency-domain inputs across matched-feature architectures and domain-adaptation choices.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Order tracking improves neural-operator transfer and enables unsupervised alignment, while a matched-feature convolutional network is almost unaffected; held-out fault geometries can defeat even target-supervised models.

    Source: §1 · p. 2, §6 · pp. 7–8

  • Takeaways & Limitations

    For this task, practitioners should consider whether a shift moves fault frequencies, choose the representation accordingly, control what the representation change also alters, and evaluate on unit-disjoint splits.

    Source: §6 · p. 8

    Order tracking assumes the target shaft rate is known and constant within a recording, as in the fixed operating conditions studied.

    Source: §3 · p. 3

Abstract

from arXiv · show

Diagnosing rolling-element bearing faults from vibration is a canonical physical-sensing task and a widely used benchmark for domain adaptation under operating-condition shift. Accuracies above 99 percent are commonly reported, but under evaluation splits that place the same physical bearing in both training and test. We revisit the task under a held-out-bearing protocol, assigning every bearing unit entirely to either the training or the test set, and find that source-only transfer is far weaker than such numbers suggest: on a change of shaft speed it reaches only $0.36$, against a target-supervised ceiling of 0.97. We then study what governs transfer. Treating computed order tracking, a shaft-angle resampling that places fault frequencies at fixed shaft orders independent of running speed, as a controlled change of representation, we find that a Fourier Neural Operator raises source-only transfer from $0.36$ to $0.61$ on the speed shift, where the fault peaks move, while a convolutional network of matched feature dimension stays near chance in both representations. The representation also decides whether unsupervised alignment can work: with the same normalized RBF-MMD loss and no target labels, the operator reaches 0.71 in the frequency domain but 0.95 in the order domain, within 0.02 of the target-supervised ceiling and above $0.86$ on every held-out bearing fold. Once the representation is right, a small label budget adds little. These results indicate that, for this task, the input representation rather than the alignment method decides whether adaptation helps. A second dataset, whose held-out units are fault diameters rather than bearings, shows that the same protocol exposes failures that even a target-supervised model cannot avoid.

1 Introduction

High reported accuracy can obscure whether bearing-fault models transfer to unseen physical units. The paper evaluates held-out bearings and uses order tracking to test how input representation shapes adaptation.

1 Introduction

  • Window-level splits can put the same bearing in training and test, allowing models to recognize unit-specific signatures rather than fault types.The authors therefore evaluate with every physical bearing assigned entirely to training or test.Source: §1 · pp. 1–2
  • Order tracking fixes fault-frequency positions across shaft speeds, providing a controlled representation change for comparing matched-feature architectures.The paper compares a Fourier Neural Operator with a convolutional network and studies whether small target-label budgets recover the remaining gap.Source: §1 · p. 2
  • Order tracking improves neural-operator transfer and enables unsupervised alignment, while a matched-feature convolutional network is essentially unaffected.The introduction identifies input representation, rather than the alignment loss, as the factor determining whether adaptation helps.Source: §1 · p. 2
  • On a second dataset, held-out fault geometries expose failures that a target-supervised model also shares.The paper also reports a systematic comparison across five alignment choices and six target-label budgets.Source: §1 · p. 2

2 Related Work

Prior bearing-diagnosis work uses deep models and alignment losses for transfer across operating conditions, while order tracking is an established speed-invariant representation. This paper positions order tracking as a controlled intervention and highlights the importance of unit-disjoint evaluation.

Data-driven bearing fault diagnosis and domain adaptation

  • Deep networks, especially one-dimensional convolutional networks, and losses such as correlation alignment, adversarial training, and MMD are established tools in bearing adaptation.Benchmark studies collect these methods on public datasets and report transfer accuracy.Source: §2 · p. 2

Order tracking and the order domain

  • Order tracking is established in diagnostics and prior deep-learning work; here it serves as a controlled intervention, not a novel method.Prior work paired order representations with convolutional classifiers and explicit or source-free adaptation.Source: §2 · p. 2

Neural operators and spectral models

  • The Fourier Neural Operator retains selected Fourier modes and has a bias toward global spectral structure, unlike convolutions’ local patterns in time.That contrast motivates testing whether the operator responds to shifts in bearing-fault peak locations.Source: §2 · p. 2

Evaluation practice

  • Window-level splits can leak physical-bearing signatures across train and test, motivating bearing-wise partitioning to assess transfer to unseen units.The section also notes a recent study recommending bearing-wise partitioning and identifying residual leakage when splitting a single healthy recording.Source: §2 · p. 2

Related-work context

  • The study compares frequency and order representations, two matched-feature architectures, and adaptation across operating conditions under a held-out-bearing protocol.The overview also shows raw vibration reduced to a common envelope before classification and adaptation.Source: Figure 1 · p. 3

3 Setup and Evaluation Protocol

The study compares bearing-fault transfer across two datasets, using common envelope preprocessing, frequency- and order-domain inputs, held-out units, and multiple alignment choices. Its protocol specifies the data splits, modeling comparisons, and assumptions behind order tracking.

3 Setup and Evaluation Protocol

  • PU uses 1500 rpm as source and targets speed, torque, and radial-force shifts; CWRU targets differ in load and have only a 3.7% shaft-rate change.PU contains three physical bearings per class; CWRU uses type-6205 recordings and selected outer-race fault positions.Source: §3 · p. 3
  • Both datasets are reduced to normalized one-second, 2-kHz Hilbert-envelope signals after resonance band-pass filtering and decimation.For CWRU, raw windows are additionally RMS-normalized, and the 48-kHz healthy recordings are decimated to 12 kHz before filtering.Source: §3 · p. 3
  • Order tracking resamples at the known nominal shaft rate into 15 revolutions × 128 samples, fixing fault-order positions across speeds.The method assumes the target shaft rate is known and constant; bandwidth- and duration-matched controls address differences introduced by the representation.Source: §3 · p. 3, Figure 2 · p. 4
  • PU folds hold out one bearing per class; CWRU folds hold out fault diameters and healthy segments, a weaker guarantee than distinct-unit holdout.CWRU also drops overlapping windows at segment boundaries; model selection uses source-domain validation, not labeled target windows.Source: §3 · pp. 3–4
  • The study compares source-only and target-label references with five alignment choices, reporting means and standard deviations over nine runs per configuration.The target-label fractions are 0, 0.03, 0.05, 0.10, 0.20, and 0.30; paired tests assess headline differences.Source: §3 · p. 4
  • The operator and convolutional network share a 16-dimensional feature layer, but their total parameter counts are not matched.Both use shared hyperparameters; the operator usually reaches 80 epochs, while the convolutional network typically stops around epoch 50.Source: §3 · pp. 4–5

4 Experiments and Results

Under held-out-bearing evaluation, transfer depends strongly on representation: order tracking improves the neural operator on the speed shift, and normalized RBF-MMD works far better in the order domain. A small target-label budget narrows the remaining gap, while tests on a second dataset expose failures on unseen fault geometries.

  • 0.95 ± 0.06 versus 0.71 ± 0.33: normalized RBF-MMD reaches higher accuracy for the operator in the order domain than the frequency domain, within 0.02 of the oracle.Order-domain accuracy is at least 0.86 on every held-out-bearing run; the convolutional network’s alignment results degrade in the order domain.Source: §4 · p. 5
  • At f = 0.03, operator accuracy rises from 0.61 to 0.92 without alignment and to 0.96 with correlation alignment; by f = 0.20, all four stable alignment choices lie within 0.03 of the oracle.Linear MMD remains unstable at every label fraction, while the convolutional network’s best variant reaches 0.83 at f = 0.30.Source: §4 · p. 5
  • On the PU speed shift, the operator exceeds the convolutional network in the order domain both without target labels (0.61 against 0.43) and at the oracle ceiling (0.97 against 0.75).Both models reach source-validation accuracy of 1.00; the reported difference is a transfer gap, though the authors do not isolate its mechanism.Source: §4 · p. 5
  • 0.61 versus 0.36: order tracking raises the Fourier Neural Operator’s source-only accuracy on the PU speed shift, while the convolutional network changes little across representations.Order tracking is benign on the fixed-speed torque and radial-force shifts, where the frequency-domain baseline is already near ceiling.Source: §4 · p. 5, Table 1 · p. 6
  • Controls show the order advantage persists across bandwidth matching and crop duration, while a fifteen-revolution frequency-domain crop reaches 0.63, matching the order-domain value.Zeroing fault-order modes reduces outer-race recall from 0.74 to 0.33, while control bands are inert.Source: §4 · p. 6
  • On CWRU, holding out fault diameters reveals failures that target supervision cannot consistently avoid: the held-out outer-race class in fold 2 has 0.00 recall even for the oracle.The authors distinguish unseen fault geometry from adaptation across load; in fold 1, order-domain operator accuracy is 0.91 versus 0.52 in the frequency domain.Source: §4 · p. 6

5 Discussion and Limitations

Held-out-unit evaluation exposes failures that pooled accuracy conceals, while controlled comparisons attribute the operator’s gains to representation rather than alignment loss. The discussion also identifies assumptions and scope limits on these findings.

5 Discussion and Limitations

  • No model, including the target-supervised oracle, recognizes the held-out 0.014 inch outer-race fault on CWRU.The CWRU folds hold out one fault diameter for testing.Source: §5 · p. 6, Table 3 · p. 7
  • On PU, the operator’s order-domain source-only accuracy is 0.68 ± 0.03, 0.49 ± 0.18, and 0.65 ± 0.00 by held-out-bearing fold.In fold two, KA03 is recognized 0.25 of the time and KI03 0.53.Source: §5 · p. 6
  • Order tracking is established practice, but here it serves as a controlled representation change; matched-band and matched-duration controls rule out those alternatives on both datasets.The controls address two differences that could otherwise account for the gains.Source: §5 · pp. 6–7
  • For the operator, every stable alignment loss transfers better in the order domain, and normalized RBF-MMD reaches the oracle only there.The convolutional network gains no benefit from alignment in either domain.Source: Table 2 · p. 7
  • Order tracking assumes a known, constant shaft rate within each recording; run-up or coast-down would require a tachometer or tacholess estimate.The measured sensitivity analysis indicates the assumed rate must be accurate to about one percent.Source: §5 · p. 7
  • CWRU oracle results are not a tight ceiling on folds 2 and 3, while linear MMD at weight 100 can collapse the operator to one class.The alignment weights and optimizer settings were selected on the operator and shared with the convolutional network, so their comparison is not a tuned benchmark.Source: §5 · p. 7
  • CWRU accuracies are coarsely quantized with wide error bars, and the study leaves two-dimensional time-frequency inputs untested.There is one healthy recording per load, and each fold is tested on one recording per fault class; the study uses a one-dimensional envelope representation.Source: §5 · p. 7

6 Conclusion

The conclusion argues that representation governs whether adaptation helps under held-out-unit evaluation, with order tracking improving operator transfer where fault frequencies shift. It also reports that a small target-label fraction closes much of the remaining gap, while unseen fault geometry can defeat even target-supervised models.

6 Conclusion

  • The input representation, not the alignment method, determines whether adaptation helps: order tracking improves operator transfer on shifts that move fault frequencies, while the matched-feature convolutional network is almost unaffected.The representation also lets unsupervised alignment close the gap for the operator.Source: §6 · pp. 7–8
  • At f = 0.03, the operator closes most of the gap; by f ≈0.2, stable alignment methods are indistinguishable.Figure 3 plots target accuracy against target-label fraction on the PU speed shift in the order domain.Source: Figure 3 · p. 8, §6 · p. 8
  • The held-out-unit protocol exposes unseen fault-geometry failures that even a target-supervised model cannot avoid.The conclusion recommends evaluating on unit-disjoint splits.Source: §6 · p. 8

7 Appendix

Appendix controls show that shaft-revolution alignment, rather than elapsed window duration or spectral bandwidth alone, accounts for the order-domain transfer advantage. Additional analyses locate class-specific fault-order effects and show sensitivity to assumed shaft rate; reported source-validation accuracy does not explain transfer gaps.

7.1 Full PU results with standard deviations

  • Every source-only model has source-validation accuracy of 1.00, so the reported transfer gaps are not fitting gaps.Table 4 reports mean ± standard deviation for source-only and oracle accuracy under both representations.Source: §7.1 · p. 9, Table 4 · p. 9

7.2 Paired statistics for the headline differences

  • Order-axis alignment changes run variability more than the paired win count: standard deviation falls from 0.33 to 0.06 and the run minimum rises from 0.33 to 0.86.The frequency-domain aligned model already reaches 1.00 for some seeds in folds 1 and 3.Source: §7.2 · p. 9, Table 5 · p. 9

7.4 Hyper-parameters and configuration

  • The operator uses two spectral blocks and 200 Fourier modes, while the convolutional network uses four blocks; both use dropout 0.3 and AdamW with learning rate and weight decay of 10^-3.The operator has 206,807 real-valued parameters; the convolutional network has 373,939 parameters.Source: §7.4 · pp. 9–10

7.3 Convolutional-network hyperparameter search

  • Table 6 compares the convolutional-network grid configurations by oracle and source-only accuracy over nine runs, alongside each configuration’s mean source-validation accuracy.The operator is included for reference.Source: Table 6 · p. 10

7.5 Additional controls: spectrum input, bandwidth, duration, mode ablation, and speed error

  • With both axes restricted to 333 Hz, source-only accuracy is 0.43 in frequency against 0.61 in order; a spectral MLP reaches 0.85 ± 0.16 against the operator’s 0.95 ± 0.06 under normalized RBF-MMD.The MLP reproduces part of the representation effect, while the operator’s spectral parametrization contributes beyond it; the order advantage also persists at matched bands.Source: §7.5 · p. 10
  • Matching physical duration leaves frequency-domain transfer at 0.37 ± 0.05, while matching fifteen revolutions raises it to 0.63 ± 0.15, matching the order-domain 0.61.The convolutional network remains at 0.43; the control supports a coordinate effect rather than a duration effect.Source: §7.5 · p. 10
  • Removing the BPFO band costs the source-only model 0.13 accuracy, all from outer-race recall, while oracle BPFO and BPFI removals affect only their corresponding fault classes.Control-band ablations change accuracy by less than 0.01; inner-race faults survive removal of the BPFI fundamental because their energy spreads into sidebands.Source: §7.5 · p. 10, Table 7 · p. 11
  • A one-percent shaft-rate error lowers source-only order-domain accuracy from 0.61 to 0.51, while a two-percent error brings it to the frequency-domain level of 0.35.Changing sampling density without displacing orders has no effect, with accuracy at 0.62 for perturbations up to five percent.Source: §7.5 · p. 11, Table 8 · p. 11
Loading 2609.31639v1…