PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry Application

Te Han, Chao Liu, Wenguang Yang, Dongxiang Jiang

arXiv:1804.07265v1cs.LGstat.ML

TL;DR

The paper addresses fault diagnosis when training and testing distributions differ, making conventional diagnosis less applicable to new operating conditions and tasks. It proposes a deep transfer network using joint distribution adaptation to match marginal and conditional distributions with labeled source and unlabeled target data. Across three fault datasets and diverse transfer scenarios, DTN with JDA outperforms state-of-the-art approaches and shows smoother convergence with less negative adaptation than MDA.

  • Problem

    Conventional diagnosis assumes identical training and testing distributions, while prior transfer studies often adapt only marginal distributions despite differing conditional distributions.

  • Method

    DTN pre-trains on labeled source data and adapts the network with labeled source and unlabeled target data using joint distribution adaptation.

  • Results

    Across three fault datasets, DTN with JDA outperforms state-of-the-art approaches, with 97.5% average accuracy versus 80.0% for baseline CNN across ten tasks.

  • Takeaways & Limitations

    Jointly adapting marginal and conditional distributions supports robust fault diagnosis across operating conditions, fault severities and fault types.

Abstract

from arXiv · show

In recent years, an increasing popularity of deep learning model for intelligent condition monitoring and diagnosis as well as prognostics used for mechanical systems and structures has been observed. In the previous studies, however, a major assumption accepted by default, is that the training and testing data are taking from same feature distribution. Unfortunately, this assumption is mostly invalid in real application, resulting in a certain lack of applicability for the traditional diagnosis approaches. Inspired by the idea of transfer learning that leverages the knowledge learnt from rich labeled data in source domain to facilitate diagnosing a new but similar target task, a new intelligent fault diagnosis framework, i.e., deep transfer network (DTN), which generalizes deep learning model to domain adaptation scenario, is proposed in this paper. By extending the marginal distribution adaptation (MDA) to joint distribution adaptation (JDA), the proposed framework can exploit the discrimination structures associated with the labeled data in source domain to adapt the conditional distribution of unlabeled target data, and thus guarantee a more accurate distribution matching. Extensive empirical evaluations on three fault datasets validate the applicability and practicability of DTN, while achieving many state-of-the-art transfer results in terms of diverse operating conditions, fault severities and fault types.

I. INTRODUCTION

Traditional fault-diagnosis frameworks rely on manual feature engineering and matching training/testing distributions, limiting adaptation to new industrial tasks. The paper motivates transfer learning and introduces DTN as a framework for diagnosis across changing conditions.

  • Existing diagnosis frameworks: Traditional diagnosis uses data collection, manual feature extraction and selection, followed by fault classification.Manual feature design requires substantial labor and time and is often tailored to a specific application.
  • Existing diagnosis frameworks: Assuming identical training and testing distributions limits generalization across diverse application domains and can require relearning models from scratch.The source passage identifies distribution mismatch as a major obstacle to practical deployment.
  • Transfer-learning motivation: Transfer learning uses information or knowledge from a related source domain to facilitate learning in a new target domain while reducing required target samples.This motivation supports borrowing useful information from historical tasks rather than reconstructing a model for each new task.
  • Proposed framework: DTN applies transfer learning to intelligent fault diagnosis and uses unlabeled target data for domain adaptation.The framework is intended for real industrial applications and adapts across operating conditions, fault severities and fault types.
  • CNN foundation: CNNs learn hierarchical signal features through convolutional, pooling and fully connected layers before classification.For time-series fault signals, one-dimensional convolutional structures are used because they fit the signal characteristics.

B. Transfer learning

Transfer learning formalizes knowledge transfer between different domains or tasks, while domain adaptation addresses distribution differences between source and target data. Prior fault-diagnosis studies mainly adapted marginal distributions, motivating joint adaptation of marginal and conditional distributions.

  • Transfer-learning definitions: A domain consists of a feature space and a marginal probability distribution over instances.The paper represents a domain as 𝒟={𝒳, P(X)}.
  • Transfer-learning definitions: A task consists of a label space and a predictive function learned from instances, with the function represented as a conditional probability distribution.The paper defines f(X)=Q(Y|X).
  • Transfer-learning definitions: Transfer learning facilitates learning a target predictive function using related information from a source domain or task when domains or tasks differ.When both domain and task are unchanged, the setting reduces to traditional machine learning.
  • Prior work and gap: Prior fault-diagnosis transfer studies explored transfer learning but needed more challenging scenarios involving varied fault severities and fault types.These variations can produce larger distribution discrepancies across domains.
  • Prior work and gap: Adapting only marginal distributions neglects source-label discrimination structures, motivating joint reduction of marginal and conditional distribution discrepancy.The paper presents joint adaptation as a potential route to superior transfer performance.

C. Maximum mean discrepancy

MMD measures distribution discrepancy in an RKHS, while JDA combines marginal and conditional adaptation to align both feature distributions and class-conditional structures.

  • Maximum mean discrepancy: MMD measures the discrepancy between two distributions by comparing their mapped means in a reproducing Kernel Hilbert space.A value near zero indicates matched distributions, and MMD can regularize feature learning across domains.
  • Joint distribution adaptation: Domain adaptation must address differing marginal and conditional distributions because marginal alignment alone may not match discriminative hyperplanes.The target labels needed for conditional adaptation are unavailable, so pseudo labels are used and iteratively updated during optimization.
  • Problem formulation: The domain-adaptation objective minimizes discrepancies between transformed marginal and conditional distributions across source and target data.The feature transform is learned in an RKHS, with separate objectives for marginal and conditional alignment.
  • Joint distribution adaptation: JDA integrates marginal MMD with conditional MMD to align the joint distributions of source and target domains.The conditional term measures class-wise mismatches using source true labels and target pseudo labels.

B. Deep transfer network

DTN embeds JDA within a CNN-based transfer framework by combining source classification loss with joint-distribution regularization during target-domain adaptation.

  • Framework: DTN uses a CNN as its base model and adapts a source-pretrained network to an unlabeled target domain.The framework establishes domain adaptation under a deep-learning architecture.
  • Objective: The training objective combines cross-entropy loss with a JDA regularization term, written as ℒ(Θ) = ℓ_ce + λD_H(J_s, J_t).Θ contains the CNN parameters, and λ is a non-negative regularization parameter.
  • Feature adaptation: JDA regularization is applied to the last hidden fully connected layer, where the learned feature map is ϕ(x) = h_l−1(x).This layer is selected because deeper features may exhibit larger domain discrepancy; deep learning learns the nonlinear transform adaptively.
  • Optimization: Mini-batch SGD and backpropagation optimize the combined objective to further transfer and adapt the pretrained CNN for target tasks.The JDA discrepancy gradient is propagated with respect to the network parameters.

C. Training Strategy

DTN training proceeds through source pretraining followed by iterative target-domain adaptation using labeled source data and unlabeled target data.

  • Training Strategy: The procedure first trains a base deep network on the labeled source dataset and predicts initial pseudo labels for target samples.These pseudo labels initialize the unsupervised adaptation stage.
  • Training Strategy: Each adaptation iteration computes the JDA regularization, optimizes the network, and updates target pseudo labels until convergence or label stability.The transferred network is then evaluated on other target samples.
  • Training Strategy: DTN consists of source pretraining and target adaptation using both labeled source and unlabeled target data.Equal source and target sample counts are used during adaptation; resampling can balance domains with different dataset sizes.
  • Training Strategy: Batch size trades off transfer performance against computational effectiveness because larger batches cover more dataset variance but increase calculation burden.The framework divides datasets into small batches for network training.

D. Comparison studies

The comparison study evaluates standard diagnosis and transfer-learning methods across three fault datasets and multiple implementation settings.

  • Comparison methods: Nine methods are compared, including SVM, random forest, EMD, CNN, TJM, TCA, JDA, DTN with MDA, and DTN with JDA.The methods are grouped into standard diagnosis approaches and transfer-learning techniques.
  • Comparison methods: SVM and random forest use 29 statistical time- and frequency-domain features, while EMD uses energy distributions from the first five intrinsic mode functions.These handcrafted representations provide inputs for the standard classifiers.
  • Transfer-learning methods: TJM, TCA, and JDA perform unsupervised adaptation on 29 statistical features before source-domain classifier training and target-domain deployment.Both SVM and random forest are used as final classifiers for these shallow transfer methods.
  • Deep transfer methods: DTN with MDA is compared with DTN with JDA by removing the CDA term from the objective function.The deep methods use a source-domain optimal CNN as the pretrained base network.
  • Implementation details: Implementation uses labeled source data to train standard models that are then applied to unlabeled target data.For transfer methods, source and target data are processed jointly for adaptation before target diagnosis.

IV. EXPERIMENTS

Experiments on three mechanical fault datasets evaluate the efficiency, superiority, and practical value of the proposed transfer framework.

  • Three mechanical fault datasets are used to evaluate the proposed transfer framework.
  • The experiments assess the framework’s efficiency, superiority, and practical value.
  • The evaluation focuses on demonstrating the proposed framework through fault-diagnosis experiments.

A. Data description

The study constructs transfer tasks from wind-turbine, bearing, and gearbox datasets to examine operating-condition, fault-severity, and fault-type shifts.

  • Wind Turbine Fault Dataset: The wind-turbine experiments combine vibration signals from two accelerometers sampled at 20 kHz.Signals from the front and back bearing pedestals are combined into each sample.
  • Wind Turbine Fault Dataset: The wind-turbine dataset supports six transfer tasks across diverse operating conditions.Tasks A→B, B→A, C→D, D→C, E→F, and F→E are designed; wind speeds range from 5.8m/s to 11.5m/s.
  • Task Design: The transfer-task designs are summarized in Table I and Table II, including unlabeled target samples.
  • Bearing Fault Dataset: The bearing dataset defines two transfer tasks across fault diameters of 0.18mm and 0.53mm.Tasks G→H and H→G use four bearing conditions and different severity levels.
  • Gearbox Fault Dataset: The gearbox dataset contains health, gear-fault, and bearing-fault conditions and supports tasks I→J and J→I.The source and target datasets use different bearing and gear fault cases.

B. Results

Across ten transfer tasks, DTN with JDA achieves robust accuracy and outperforms comparison methods, while transfer difficulty varies with domain similarity.

  • DTN with JDA achieves almost over 95% accuracy across all transfer tasks.The authors describe these stable accuracies as evidence of effective and robust domain adaptation.
  • 97.5% average accuracy for DTN with JDA represents a 17.5% transfer improvement over the baseline CNN at 80.0%.
  • Standard diagnosis performance varies substantially across tasks, with RF obtaining the best results in the first six wind-turbine tasks.
  • C→D and D→C obtain 88.8% and 92.8%, whereas E→F and F→E obtain 77.3% and 60.8%.The passage relates the difference to greater operating-condition similarity between C and D than between E and F.
  • Domain-adaptation difficulty increases in hard tasks such as E→F and J→I, where source and target data may be substantially dissimilar.Comparative methods including DTN with MDA show performance drops in these scenarios.

C. Network visualization: how JDA outperforms MDA

t-SNE visualizations compare standard CNN, DTN with MDA, and DTN with JDA across operating-condition, severity-level, and fault-type transfers.

  • Task E→F: In task E→F, t-SNE visualizes last-hidden-layer representations for source and target samples across ten wind-turbine categories.S denotes source samples and T denotes target samples.
  • Task E→F: Standard CNN leaves same-category source and target distributions poorly aligned and produces overlaps among some target categories.The cited examples include categories 2, 3, and 8.
  • Task E→F: DTN with JDA aligns same-category source and target distributions while preserving a clear discriminant structure.
  • Task G→H: In task G→H, the visualization compares four bearing categories across fault-severity levels.
  • Task I→J: In task I→J, JDA simultaneously adapts marginal and conditional distributions and outperforms MDA in transfer accuracy and visualization.

D. Why JDA is better than MDA

JDA improves transfer over MDA by supporting smoother convergence and higher accuracy while avoiding negative adaptation when source and target domains differ substantially.

  • JDA regularization converges after iterations while target test accuracy continues increasing, unlike MDA’s fluctuating regularization loss and accuracy near 87%.The comparison is shown through transfer-loss and test-accuracy curves for task G→H.
  • JDA preserves discriminant source structure while reducing domain discrepancy, although competing optimization gradients can produce a temporary transfer-loss spike.The cross-entropy and regularization terms jointly balance domain alignment with source discrimination.
  • In task H→G, MDA achieves 63.7%, below the 70% no-transfer result, whereas JDA with CDA reaches 100% and avoids negative adaptation.The negative adaptation is associated with substantial source–target distribution differences and initial IF-to-RF misclassification by the pretrained network.
  • JDA addresses the practical mismatch between source and target domains by transferring a pretrained diagnosis model using unlabeled target data instead of retraining from scratch.This design targets industrial settings where sufficient labeled target data may be unavailable.
  • The framework’s broader applicability remains promising, with further work needed on additional real-data case studies and imbalanced machine-condition distributions.
Loading 1804.07265v1…