PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Adaptive Transfer Learning for Fault Diagnosis

Qin Wang, Gabriel Michau, Olga Fink

arXiv:1905.06004v1stat.MLcs.LGstat.AP

TL;DR

Deep diagnosis models may generalize poorly to new machines and operating conditions, and prior fault-diagnosis studies had not sufficiently answered whether adaptation methods apply realistically or how they compare. The paper introduces DANN and evaluates it alongside two other methods under a unified protocol; all three improve target performance, with DANN competitive against MMD and superior to AdaBN.

  • Problem

    Deep models can overfit training data and perform poorly on new machines or operating conditions, while prior fault-diagnosis studies had not sufficiently established realistic applicability or comparative performance of domain adaptation.

    Source: §I · p. 1

  • Method

    The paper introduces DANN alongside two other domain-adaptation methods and compares them under a unified experimental protocol.

    Source: §I · p. 2, B. Domain Adaptation Methods · p. 3

  • Results

    All three methods improved target performance; DANN exceeded 99.0% average target accuracy, while MMD averaged 99.4% over all tasks.

    Source: 1) Model Performance · p. 6, §VI · p. 7

  • Takeaways & Limitations

    In this comparison, DANN offers competitive results with less training time than MMD and superior results to AdaBN.

    Source: §VI · p. 7, 3) Discussion · p. 7

Abstract

from arXiv · show

Thanks to digitization of industrial assets in fleets, the ambitious goal of transferring fault diagnosis models fromone machine to the other has raised great interest. Solving these domain adaptive transfer learning tasks has the potential to save large efforts on manually labeling data and modifying models for new machines in the same fleet. Although data-driven methods have shown great potential in fault diagnosis applications, their ability to generalize on new machines and new working conditions are limited because of their tendency to overfit to the training set in reality. One promising solution to this problem is to use domain adaptation techniques. It aims to improve model performance on the target new machine. Inspired by its successful implementation in computer vision, we introduced Domain-Adversarial Neural Networks (DANN) to our context, along with two other popular methods existing in previous fault diagnosis research. We then carefully justify the applicability of these methods in realistic fault diagnosis settings, and offer a unified experimental protocol for a fair comparison between domain adaptation methods for fault diagnosis problems.

I. INTRODUCTION

Deep fault-diagnosis models may generalize poorly when new machines or operating conditions differ from their training data. The paper introduces domain adaptation for this setting and argues that prior fault-diagnosis studies had not sufficiently answered whether these methods apply realistically or how they compare.

I. INTRODUCTION

  • Deep models can overfit unrepresentative training data and degrade on new machines or operating conditions.The text also notes that changed operating conditions may be misclassified as faults.Source: §I · p. 1
  • Domain adaptation uses limited unlabeled data from a new machine or operating condition to improve generalization from a labeled source domain.The paper introduces DANN to fault diagnosis, drawing on its use in computer vision.Source: §I · p. 1
  • Earlier fault-diagnosis studies had not sufficiently established whether domain adaptation applies in realistic settings or how methods compare.The paper proposes a unified protocol on the CWRU rolling-element-bearing dataset to address these questions.Source: §I · pp. 1–2

II. RELATED WORK

  • Related work aligns source and target distributions using feature-transfer methods, MMD [3], AdaBN [4], DANN [5], or classifier discrepancy.Fault-diagnosis studies have also applied AdaBN, adversarial alignment [8], and MMD.Source: §II · p. 2

III. PROBLEM DESCRIPTION

The paper frames diagnosis transfer between similar machines operating under different conditions, using labeled source data and unlabeled target data. The task is to improve performance on the target machine under a shared fault-label space.

III. PROBLEM DESCRIPTION

  • The setup transfers diagnosis knowledge from a long-running source machine to a similar target machine with less data and different operating conditions.The machines are assumed to share the same fault types.Source: §III · p. 2

A. Domain Adaptation Task

  • The task uses labeled source data and unlabeled target data to train a model whose performance is evaluated on the target machine.The target labels are unavailable during training.Source: A. Domain Adaptation Task · pp. 2–3
  • The formulation assumes target data cover most fault types, so source and target share the same label space.The paper identifies this as the setup used by most prior fault-diagnosis domain-adaptation papers.Source: A. Domain Adaptation Task · p. 3

IV. METHODS

The methods are compared under a unified protocol and a shared architecture to make the comparison fair. The common baseline consists of a feature extractor followed by a classifier.

IV. METHODS

  • A shared basic architecture is used across domain-adaptation methods to reduce unfairness from differences in network capacity.The methods section first presents this common backbone before introducing the adaptation methods.Source: A. Baseline Architecture · p. 3, §IV · p. 3

A. Baseline Architecture

  • The baseline CNN uses a feature extractor and classifier, with three 1-D convolutional layers and a 256-feature representation.The feature extractor uses filter length 3, hidden size 10, sigmoid activations, and dropout rate 0.5; the classifier has two fully connected layers.Source: A. Baseline Architecture · p. 3
  • All experiments use softmax cross-entropy classification loss.The basic backbone's softmax output is denoted by psi.Source: A. Baseline Architecture · p. 3

B. Domain Adaptation Methods

The paper compares three deep-model-compatible domain-adaptation methods and introduces DANN to fault diagnosis. It excludes TCA because prior experiments reported inferior performance.

B. Domain Adaptation Methods

  • DANN is introduced to fault diagnosis in this paper, alongside two other domain-adaptation methods.The methods were selected based on their applicability to deep models.Source: B. Domain Adaptation Methods · p. 3
  • Transfer Component Analysis was excluded because experiments in had shown inferior performance.TCA is identified as a classic method in.Source: B. Domain Adaptation Methods · p. 3
  • The comparison focuses on methods selected for applicability to deep models.The passage names three domain-adaptation methods for comparison.Source: B. Domain Adaptation Methods · p. 3

1) Domain-Adversarial Neural Networks (DANN [5]):

DANN aligns source and target feature distributions adversarially while retaining supervised label prediction. Its min-max objective trains the feature extractor to make domains hard to distinguish and the discriminator to identify feature origin.

1) Domain-Adversarial Neural Networks (DANN [5]):

  • DANN aligns source and target feature distributions by making them difficult for a discriminator to distinguish.The feature extractor is trained to fool a discriminator that predicts whether features come from source or target data.Source: 1) Domain-Adversarial Neural Networks… · pp. 3–4
  • DANN reduces H-Divergence between source and target feature distributions through adversarial training and gradient reversal.The paper adapts the method in [5] for its semi-supervised learning scenario.Source: 1) Domain-Adversarial Neural Networks… · pp. 3–4
  • The network comprises a feature extractor, label predictor, and discriminator; experiments use a three-layer fully connected discriminator.Its first two layers have 1024 hidden units with ReLU, and its last layer predicts source versus target using cross-entropy loss.Source: 1) Domain-Adversarial Neural Networks… · pp. 3–4
  • The training objective combines supervised label-prediction loss with adversarial domain-prediction loss to encourage domain-invariant features.The label-prediction term trains the feature extractor and label predictor; the domain term promotes alignment.Source: 1) Domain-Adversarial Neural Networks… · p. 4

2) Maximum Mean Discrepancy (MMD) Minimization:

MMD minimizes a discrepancy between source and target distributions using their kernel embeddings in an RKHS. The experiments use a multi-kernel loss alongside label-prediction loss, with modified architectures for MMD comparison experiments.

2) Maximum Mean Discrepancy (MMD) Minimization:

  • MMD measures discrepancy as the squared distance between source and target marginal-distribution kernel embeddings in an RKHS.The distributions are the labeled source and unlabeled target distributions.Source: 2) Maximum Mean Discrepancy (MMD)… · p. 4
  • Multiple kernels are used because kernel choice is crucial for estimating discrepancy; the experiments use equally weighted Gaussian kernels with widths 1, 2, 4, 8, and 16.The paper follows prior fault-diagnosis MMD settings and states this kernel choice is sufficient for its specific task.Source: 2) Maximum Mean Discrepancy (MMD)… · p. 4
  • The multi-kernel MMD loss is added to label-prediction loss to align source and target feature distributions.The comparison experiments use modified architectures for MMD and AdaBN.Source: 2) Maximum Mean Discrepancy (MMD)… · p. 4, Figure 4 · p. 4, Figure 5 · p. 4

3) Adaptive Batch Normalization (AdaBN) [4]:

AdaBN addresses target-machine performance under the assumption that deterioration stems from distribution discrepancies in intermediate layers. It replaces source Batch Normalization statistics with target statistics and fine-tunes those statistics on target data.

3) Adaptive Batch Normalization (AdaBN) [4]:

  • Batch Normalization normalizes features using mini-batch statistics during training and population statistics at test time, followed by learned scale and offset parameters.For each feature, the layer outputs a normalized value transformed by learned γ and β.Source: 3) Adaptive Batch Normalization… · pp. 4–5
  • AdaBN assumes target-machine deterioration is caused by intermediate-layer distribution discrepancy and replaces source Batch Normalization statistics with target statistics.This is expected to reduce distribution differences at each layer; AdaBN requires no additional parameters beyond a small number of Batch Normalization parameters.Source: 3) Adaptive Batch Normalization… · p. 5
  • In the AdaBN experiments, Batch Normalization layers follow each convolutional layer, and target-data fine-tuning updates μ and σ while fixing β and other trainable variables.Source: 3) Adaptive Batch Normalization… · p. 5

V. CASE STUDY

The case study compares domain-adaptation methods under a protocol intended to support realistic and fair evaluation. It uses CWRU bearing data across four working loads as source-to-target domain-adaptation tasks.

V. CASE STUDY

  • The comparison protocol holds backbone architecture and preprocessing constant, gives models similar hyperparameter-tuning budgets, and uses a realistically chosen validation set.Source: §V · p. 5

A. Dataset

  • The study uses drive-end accelerometer data and selects the public, widely used CWRU dataset for its experiments.The dataset is from the Case Western Reserve University Bearing Data Center.Source: A. Dataset · p. 5
  • The CWRU dataset supplies 10 bearing conditions: nine fault states and one healthy state, spanning inner-race, ball, and outer-race faults.Fault diameters are 7, 14, and 21 mils; experiments use 12 kHz data, down-sampling when needed.Source: A. Dataset · p. 5
  • The case study uses the CWRU bearing dataset and treats its four loads as working conditions for domain adaptation across loads.Task 0 → 1 denotes load 0 as source and load 1 as target.Source: A. Dataset · p. 5

B. Preprocessing

The preprocessing converts normalized vibration sequences into Fourier-coefficient vectors. A source-only feature extractor and classifier provide the baseline, with shared hyperparameters tuned on a difficult validation task.

B. Preprocessing

  • Signals are truncated to 120,000 points, split into 200 overlapping 1,024-point sequences, transformed into 512 Fourier coefficients, and normalized with a factor selected on the source-only validation task.The normalization factor is selected from {1, 8, 64, 512}.Source: B. Preprocessing · p. 5

C. Baseline

  • The baseline trains a feature extractor and basic classifier on source data only, with hyperparameters tuned on task 0 → 3 and then fixed across experiments.The study uses Adam with a 0.0002 learning rate; task 0 → 3 was chosen as one of the most difficult transfer pairs.Source: C. Baseline · p. 5

D. Budgets for Method-specific Hyper-parameters

The study compares domain-adaptation methods under shared experimental budgets and reports results across available CWRU adaptation tasks. All methods improve target-domain performance, with differing accuracy and computational characteristics.

D. Budgets for Method-specific Hyper-parameters

  • All models are trained for 2,000 epochs; DANN and MMD each receive a three-value hyperparameter pool, while AdaBN requires no additional hyperparameter.The pools are {0.1, 1, 10} for gradient-reversal factor λd and MMD discrepancy weight λMMD.Source: D. Budgets for Method-specific… · p. 5

E. Experimental Environment

  • Experiments are run five times, with average and maximum accuracy reported to reflect performance and stability.The experiments use an NVIDIA GTX 1080 and a Python/TensorFlow framework.Source: E. Experimental Environment · p. 6

1) Model Performance:

  • The source-only baseline reaches 94.99% average accuracy on CWRU and is described as stronger than results reported in [8],,.The authors argue that this strong baseline makes evaluation of adaptation methods fairer.Source: 1) Model Performance · p. 6
  • 99.4% average accuracy over all tasks is reported for MMD, while DANN achieves over 99.0% average accuracy on the target domain; AdaBN also improves performance, though less significantly.These outcomes assume unlabeled target-domain data are available; Table II presents domain-adaptation results.Source: 1) Model Performance · p. 6, Table II · p. 6
  • Results from previous works are not directly comparable because papers prepare target test sets differently, and this study reports performance on all available CWRU adaptation tasks.Some Table II cells are missing because the original papers did not report results for those tasks.Source: 1) Model Performance · p. 6

2) Model Efficiency:

DANN is presented as a trade-off between accuracy and computational cost, while MMD achieves the best bearing-dataset results at greater computational cost. The conclusion reports that all three methods improve target-data performance.

2) Model Efficiency:

  • MMD requires no additional trainable parameters but takes significantly longer to train; DANN uses more parameters yet trains faster through gradient-reversal layers integrated with standard gradient descent.MMD estimation in each iteration is time-consuming, while the gradient-reversal approach allows efficient H-divergence estimation.Source: 2) Model Efficiency · pp. 6–7

3) Discussion:

  • MMD yields the best bearing-dataset results at the largest computational cost and may be less efficient with larger training sets; DANN is described as a good accuracy–compute trade-off, while AdaBN is moderate with minimal additional cost.AdaBN requires few additional parameters and can be combined with other domain-adaptation methods without increasing model complexity.Source: 3) Discussion · p. 7

VI. CONCLUSION

  • All three methods improve target-data performance; DANN provides competitive results with significantly less training time than MMD and superior results over AdaBN.The conclusion frames the comparison as a unified experimental procedure for supervised fault diagnosis tasks.Source: §VI · p. 7
Loading 1905.06004v1…