PaperlayerView exact revision on arXiv

Source-linked AI summary

Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis

Cheng Cheng, Beitong Zhou, Guijun Ma, Dongrui Wu, Ye Yuan

arXiv:1903.06753v1cs.LGeess.SPstat.ML

TL;DR

Industrial fault diagnosis must transfer across changing conditions despite different source–target feature distributions and insufficient target labels. The paper proposes WD-DTL, which combines CNN feature extraction with Wasserstein adversarial domain adaptation, and evaluates it across 16 unsupervised and insufficiently supervised transfer tasks. WD-DTL achieves the best reported average transfer accuracy and improves over CNN and mostly over DAN.

  • Problem

    Fault-diagnosis models must handle different source–target feature distributions and insufficient or absent target labels under diverse industrial operating conditions.

  • Method

    WD-DTL uses a CNN-based feature extractor, a domain critic, and adversarial training to minimize source–target distribution discrepancy using Wasserstein distance.

  • Results

    WD-DTL achieves 95.75% average transfer accuracy across the reported tasks, with approximately 13.6% and 25% average-accuracy increases over CNN for motor-speed and sensor-location transfers.

  • Takeaways & Limitations

    Unsupervised transfer is sufficient for similar signals such as different motor speeds, while small labeled samples substantially improve transfer between less-similar sensor locations.

Abstract

from arXiv · show

The demand of artificial intelligent adoption for condition-based maintenance strategy is astonishingly increased over the past few years. Intelligent fault diagnosis is one critical topic of maintenance solution for mechanical systems. Deep learning models, such as convolutional neural networks (CNNs), have been successfully applied to fault diagnosis tasks for mechanical systems and achieved promising results. However, for diverse working conditions in the industry, deep learning suffers two difficulties: one is that the well-defined (source domain) and new (target domain) datasets are with different feature distributions; another one is the fact that insufficient or no labelled data in target domain significantly reduce the accuracy of fault diagnosis. As a novel idea, deep transfer learning (DTL) is created to perform learning in the target domain by leveraging information from the relevant source domain. Inspired by Wasserstein distance of optimal transport, in this paper, we propose a novel DTL approach to intelligent fault diagnosis, namely Wasserstein Distance based Deep Transfer Learning (WD-DTL), to learn domain feature representations (generated by a CNN based feature extractor) and to minimize the distributions between the source and target domains through adversarial training. The effectiveness of the proposed WD-DTL is verified through 3 transfer scenarios and 16 transfer fault diagnosis experiments of both unsupervised and supervised (with insufficient labelled data) learning. We also provide a comprehensive analysis of the network visualization of those transfer tasks.

I. INTRODUCTION

Industrial fault diagnosis needs intelligent methods because expert analysis is demanding, while changing operating conditions and scarce target labels limit conventional deep-learning transfer. The paper proposes WD-DTL, using Wasserstein-based adversarial domain adaptation with CNN features for unsupervised and insufficiently supervised diagnosis.

  • Motivation: Expertise-intensive fault diagnosis motivates artificial-intelligence methods that can support timely maintenance decisions and operational safety.Real-time diagnosis can help maintenance teams replace or repair affected components in advance.
  • Research gap: CNN-based fault diagnosis models face distribution differences between source and target datasets under new operating conditions.Their adaptability is limited when operational conditions and physical characteristics change.
  • Research gap: Deep transfer learning leverages knowledge from sufficiently labeled source domains to learn target tasks with insufficient or unlabeled data.This can avoid reconstructing a new diagnosis model and recollecting large labeled datasets.
  • Research gap: MMD-based domain adaptation can incur quadratically increasing computational cost with large sample sets, motivating Wasserstein distance as an alternative distribution measure.Prior work describes Wasserstein distance as more sensible for distributions supported by low-dimensional manifolds.
  • Proposed approach: WD-DTL combines a CNN feature extractor, a domain critic, and adversarial training to learn transferable features across operating conditions.The paper evaluates unsupervised and supervised transfer across three scenarios and 16 tasks, and reports superiority over DAN in almost all tasks.

A. Convolutional Neural Networks

The paper uses a pretrained 1-D CNN to extract fault-diagnosis characteristics and frames transfer learning as prediction in a target task using source-domain knowledge. Wasserstein distance supplies an optimal-transport view for comparing source and target distributions.

  • A. Convolutional Neural Networks: CNNs use local connections, shared weights, and local pooling to detect local information with fewer parameters.These characteristics motivate CNN use as the paper’s base deep-learning model.
  • A. Convolutional Neural Networks: The base model is a 1-D CNN pretrained on the source domain through convolutional, pooling, and fully connected layers optimized by backpropagation and SGD.The architecture is intended to extract and learn characteristics for bearing fault diagnosis.
  • B. Transfer learning: Transfer learning addresses unlabeled or insufficient target-domain data by using knowledge from a source domain to improve target-domain learning.The formal objective is to learn a target prediction function from source-domain knowledge.
  • B. Transfer learning: A domain consists of a feature space and marginal distribution, while a task consists of a label space and predictive function.The paper assumes source and target tasks share the same label space while domains may differ in distributions.
  • C. Wasserstein distance: Wasserstein-1 distance measures the minimum transport cost between two probability distributions over a compact metric space.It can be interpreted as finding a joint transport plan whose marginals are the source and target distributions.

III. WASSERSTEIN DISTANCE BASED DEEP TRANSFER LEARNING (WD-DTL)

WD-DTL addresses domain adaptation when target-domain labels are unavailable or limited, using labeled source data and unlabeled target data. Its objective is to reduce target classification error through Wasserstein-based adaptation and a three-stage training procedure.

  • A. Problem formulation: Target-domain fault diagnosis is constrained by scarce high-quality labels caused by sensor-retrofitting difficulty and expensive industrial labeling.These constraints make fine-tuning a pretrained architecture for supervised target adaptation infeasible.
  • A. Problem formulation: The domain-adaptation formulation uses labeled source samples and an unlabeled target dataset, typically with substantially more source than target samples.Source and target share a feature space but have different marginal distributions.
  • A. Problem formulation: WD-DTL minimizes target classification error by using source-task knowledge and Wasserstein distance for domain adaptation.The stated target is a transferable framework for the target task.
  • A. Problem formulation: WD-DTL training iterates through CNN feature extraction, Wasserstein-based domain adaptation, and discriminator-based classification.These three stages define the paper’s training flow.

B. CNN based feature extractor

The CNN-based feature extractor builds hierarchical representations from source-domain data and produces probability-distributed outputs for classification.

  • A source-domain CNN is pretrained using labelled data before transfer learning.
  • Convolution applies filters, biases, and nonlinear activation to local input subvectors, producing a feature map.The feature map uses a stride-dependent number of features.
  • Max pooling extracts maximum values over disjoint feature-map regions to capture prominent local features.
  • Stacked convolutional and pooling layers are flattened and passed through fully connected layers for classification.
  • The output layer uses Softmax to produce probability-distributed predictions over labels.Cross-entropy compares predicted labels with source-domain ground truth.

C. Domain adaptation via Wasserstein distance

WD-DTL uses Wasserstein-1 distance and adversarial domain-critic training to reduce source–target feature-distribution differences in a common latent space.

  • Wasserstein-1 distance is used to learn invariant feature representations between source and target domains through adversarial training.
  • The pretrained CNN feature extractor maps source and target mini-batches into source features hs and target features ht.The resulting feature distributions are denoted Ps and Pt.
  • A domain critic maps source and target features to real numbers and is optimized to reduce the distance between Ps and Pt.
  • The empirical Wasserstein-1 distance is approximated using a supremum over 1-Lipschitz critic functions.The resulting domain critic loss measures discrepancy between source and target data.
  • Gradient penalty is introduced to enforce the Lipschitz constraint during critic training instead of relying solely on weight clipping.

D. Classification with discriminator

WD-DTL combines supervised classification with domain-discrepancy optimization to make transferred feature representations more discriminative for the target domain.

  • Supervised learning of source data, and target data when available, is incorporated because unsupervised adaptation may produce insufficiently discriminative features.
  • The domain critic parameters remain trained from the adaptation step while feature-extractor parameters are modified to optimize the minimum operator.
  • The final objective combines discriminator cross-entropy with empirical Wasserstein distance through a domain-confusion coefficient λ.The gradient penalty is omitted when optimizing the minimum operator because it should not affect representation learning.
  • The overall WD-DTL framework consists of a CNN feature extractor, domain critic, and discriminator, with a training procedure summarized in Algorithm 1.
  • Training iteratively updates critic, discriminator, and feature-extractor parameters using source and target minibatches.The feature extractor is updated with classification and Wasserstein-loss terms.

A. Data description

The evaluation uses CWRU bearing data across multiple operating conditions and compares WD-DTL with non-transfer, MK-MMD-based, and handcrafted-feature transfer methods.

  • The CWRU benchmark covers healthy bearings and inner-race, outer-race, and roller faults sampled at 12 KHz.
  • Experiments vary motor speed across 1797rpm, 1772rpm, 1750rpm, and 1730rpm, with vibration signals recorded for diagnosis.
  • Preprocessing divides signals into 2000-measurement samples, applies FFT, and clips the spectrum to 1000 input measurements.Three transfer scenarios include two unsupervised scenarios and one supervised scenario.
  • The US-Speed scenario forms four-way classification tasks across four motor-speed domains.
  • Comparisons include CNN without transfer, DAN using MK-MMD, and TCA, JDA, and CORAL using handcrafted statistical features.
  • The framework includes a CNN feature extractor, Wasserstein-based domain critic, and classification discriminator for transfer diagnosis.

B. Implementation details

The experiments use standardized CNN, DAN, and WD-DTL implementations, with repeated Adam-optimized training and fixed evaluation settings.

  • All deep-learning models are trained with TensorFlow and the Adam optimizer.
  • Each approach is tested five times over 5000 iterations, reporting mean accuracy with 95% confidence intervals.
  • CNN: The CNN baseline contains two convolutional, two max-pooling, and two fully connected layers, with Softmax output and ReLU convolutional activations.
  • DAN: DAN adapts CNN features through an FC1 hidden layer by minimizing MK-MMD together with classification loss.
  • WD-DTL: WD-DTL uses CNN convolutional layers, a domain critic, a discriminator, gradient penalty, and scenario-specific balance coefficients.
  • Traditional baselines: TCA, JDA, and CORAL use regularization values selected from {0.001, 0.01, 0.1, 1.0, 10, 100}, with SVM classification for TCA and CORAL.

C. Results and discussion

WD-DTL generally outperforms CNN and DAN in transfer accuracy and produces better-separated feature clusters, although sensor-location transfer remains difficult even with all target samples.

  • Transfer performance: WD-DTL achieves 13.6% and 25% average-accuracy increases over CNN for motor-speed and sensor-location transfer, respectively.
  • Transfer performance: 95.75% average accuracy is reported for WD-DTL across the transfer tasks.
  • Transfer performance: WD-DTL exceeds most DAN results by an average 5% accuracy increase, while US(D) →US(A) differs by less than 1%.
  • Feature visualization: The study compares CNN, DAN, and WD-DTL network visualizations using t-SNE for learned feature representations.
  • Feature visualization: For US(C) →US(A), WD-DTL forms four clearly separated condition clusters and nearly mixes source and target features within the same clusters.
  • Feature visualization: For US(E) →US(F), WD-DTL clusters better than CNN and DAN, but fault types 1, 2, and 3 remain difficult to separate.
  • Feature visualization: The US(E) →US(F) visualization uses 100% of the target samples, yet the resulting performance is still described as insufficient.

2) Effect of sample size on unsupervised and supervised accuracy:

Increasing target-domain sample size improves WD-DTL accuracy, but unsupervised transfer between different sensor locations remains limited; a small labeled sample substantially improves performance. WD-DTL also shows higher accuracy and narrower confidence intervals than CNN and DAN in the evaluated tasks.

  • Supervised transfer: 100 labeled samples, equivalent to 25 per fault category, achieved 80% transfer-learning accuracy for supervised S(E) → S(F) transfer.More than 20 labeled samples already surpassed the unsupervised result obtained with 100% target-domain sample size.
  • Practical guidance: For similar signals, unsupervised transfer with unlabeled data can obtain fault-diagnosis accuracy above 95%, whereas less-similar signals benefit greatly from a small labeled sample.The paper distinguishes different motor speeds from different sensor locations as examples of these two cases.
  • Algorithm robustness: Across 12 motor-speed transfer tasks, WD-DTL achieved higher accuracy and narrower 95% confidence intervals than CNN and DAN.The reported comparison indicates both stronger transfer performance and greater robustness for these tasks.
  • Algorithm robustness: Mechanical-system transfer models showed worse robustness than image-classification transfer models, possibly because acquired acceleration signals contain substantial noise.The authors identify signal-processing-based noise filtering as future work.
Loading 1903.06753v1…