PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Adaptation Extreme Learning Machines for Drift Compensation in E-nose Systems

Lei Zhang, David Zhang

arXiv:1505.06405v1cs.LG

TL;DR

Sensor drift changes data distributions across E-nose batches, making recalibration laborious and limiting traditional ELM’s transfer capability with few target labels. The paper proposes DAELM, a unified cross-domain ELM framework with DAELM-S and DAELM-T, and reports that it outperforms existing drift-compensation methods on sensor-drift data without cumbersome measures.

  • Problem

    Changing domains and limited labeled target data make E-nose recalibration laborious while reducing ELM’s transfer capability and generalization across drifted batches.

  • Method

    DAELM uses limited labeled target-domain data for cross-domain classifier learning through source-domain adaptation and target-domain adaptation algorithms.

  • Results

    DAELM significantly outperforms existing drift-compensation methods on sensor-drift data without cumbersome measures.

  • Takeaways & Limitations

    The framework extends ELM with knowledge adaptability while retaining analytical solutions and efficient feature mapping for E-nose drift compensation.

Abstract

from arXiv · show

This paper addresses an important issue, known as sensor drift that behaves a nonlinear dynamic property in electronic nose (E-nose), from the viewpoint of machine learning. Traditional methods for drift compensation are laborious and costly due to the frequent acquisition and labeling process for gases samples recalibration. Extreme learning machines (ELMs) have been confirmed to be efficient and effective learning techniques for pattern recognition and regression. However, ELMs primarily focus on the supervised, semi-supervised and unsupervised learning problems in single domain (i.e. source domain). To our best knowledge, ELM with cross-domain learning capability has never been studied. This paper proposes a unified framework, referred to as Domain Adaptation Extreme Learning Machine (DAELM), which learns a robust classifier by leveraging a limited number of labeled data from target domain for drift compensation as well as gases recognition in E-nose systems, without loss of the computational efficiency and learning ability of traditional ELM. In the unified framework, two algorithms called DAELM-S and DAELM-T are proposed for the purpose of this paper, respectively. In order to percept the differences among ELM, DAELM-S and DAELM-T, two remarks are provided. Experiments on the popular sensor drift data with multiple batches collected by E-nose system clearly demonstrate that the proposed DAELM significantly outperforms existing drift compensation methods without cumbersome measures, and also bring new perspectives for ELM.

I. INTRODUCTION

The paper extends ELM to cross-domain learning for sensor-drift compensation, using limited labeled target-domain data while retaining ELM’s analytical efficiency. It proposes the unified DAELM framework with source- and target-domain adaptation algorithms.

  • Motivation and background: ELM is an efficient method for pattern classification and regression with analytically determined output weights.Its closed-form solution uses the Moore-Penrose generalized inverse and regularized least squares.
  • Motivation and background: Changing data distributions across experimental stages limit ELM’s transfer capability when only a few target-domain labels are available.E-nose data collection and labeling are described as tedious and labor-intensive.
  • Contribution: The paper extends ELM for domain adaptation between multiple domains using very few labeled target-domain guide instances.The application focus is sensor-drift compensation in E-nose systems.
  • Contribution: DAELM-S uses labeled source data and limited labeled target samples as regularization, whereas DAELM-T uses labeled target data and unlabeled target data.DAELM-T also approximates a source-trained base classifier’s predictions and can incorporate existing classifiers as the base classifier.
  • Contribution: The unified DAELM framework retains random feature mapping, analytical output-weight solutions, and ELM’s generalization properties.The framework is presented for sensor-drift compensation and gas recognition.

A. Sensor Drift Compensation in Electronic Nose

An electronic nose uses cross-sensitive gas sensors for recognition and related assessment tasks, but long-term aging creates sensor drift that changes feature distributions and degrades classifiers. Existing compensation approaches remain limited in generalizing to new drifted data.

  • Electronic nose and drift: Electronic noses combine gas sensor arrays with pattern recognition for gas recognition, quality assessment, diagnosis, monitoring, and concentration estimation.The sensors have cross-sensitivity and broad-spectrum characteristics.
  • Electronic nose and drift: Sensor drift arises from unknown dynamic processes such as poisoning, aging, and environmental variations, reducing classifier performance over time.Periodic retraining with new samples is required for recalibration.
  • Problem formulation: The drift problem is a distribution mismatch: classifiers trained on batch 1 degrade on later batches, with mismatch generally increasing with batch index and aging.Batch 1 is the labeled source domain, while later drifted batches are target domains with limited labels.
  • Existing approaches: Drift compensation methods include component correction, adaptive methods, and machine learning methods.CCPCA is described as attempting to identify and remove drift directions using PCA, while classifier ensembles have improved recognition accuracy on long-term drift data.
  • Existing approaches: The paper targets weak generalization to completely new drifted data through cross-domain learning.It aims to enhance classifier adaptation to new drifting or drifted data.

B. Principle of ELM

ELM represents inputs through a hidden layer and learns output weights by minimizing prediction error with weight regularization. The resulting regularized least-squares problem has a closed-form solution under either overdetermined or underdetermined training conditions.

  • ELM formulation: ELM maps each input through a hidden layer with L nodes and learns output weights β between the hidden and output layers.The hidden-layer activation may use functions such as RBF or sigmoid.
  • ELM formulation: Regularized ELM minimizes squared prediction loss together with the norm of the output weights to control over-fitting.The penalty constant C weights the training-error term.
  • Optimization: The constrained ELM objective is converted into an unconstrained regularized least-squares problem involving the hidden-layer output matrix H and target matrix T.H stacks hidden-layer outputs for all training samples, while T stacks their ground-truth vectors.
  • Optimization: The output weights are obtained in closed form by setting the objective gradient with respect to β to zero.The solution uses different forms depending on whether the number of training patterns exceeds the number of hidden nodes.
  • Optimization: When training patterns outnumber hidden nodes, ELM solves an overdetermined system; when they are fewer, it handles an underdetermined least-squares problem.The corresponding formulations use identity matrices of dimensions L×L or N×N.

III. PROPOSED DOMAIN ADAPTATION ELM FRAMEWORK

The proposed domain-adaptation ELM framework contains two methods, DAELM-S and DAELM-T, each presented with its corresponding learning algorithm.

  • Proposed framework: The framework introduces Source Domain Adaptation ELM (DAELM-S) and Target Domain Adaptation ELM (DAELM-T).Both methods are formulated as domain-adaptation extensions of ELM.

A. Source Domain Adaptation ELM (DAELM-S)

DAELM-S learns a source-domain classifier using all labeled source instances while incorporating a limited number of labeled target-domain guide samples as regularization. The method analytically solves the ELM output weights and predicts unlabeled target samples.

  • A. Source Domain Adaptation ELM (DAELM-S): DAELM-S learns classifier β_S from all labeled source-domain instances while leveraging limited labeled target-domain guide samples.The target samples make source-domain learning transferable through a regularization term.
  • A. Source Domain Adaptation ELM (DAELM-S): The optimization penalizes output-weight magnitude and prediction errors on labeled source and target samples.C_S and C_T weight the source- and target-domain prediction-error terms.
  • A. Source Domain Adaptation ELM (DAELM-S): The target-domain regularization makes guide-sample feature mappings approximate labels recognized by β_S, enabling knowledge transfer between domains.This coupling is the mechanism that makes the learned classifier transferable.
  • A. Source Domain Adaptation ELM (DAELM-S): DAELM-S uses source training data and labeled target data for classifier learning, while unlabeled target data are not used.The structure supports M target domains or tasks.
  • A. Source Domain Adaptation ELM (DAELM-S): For unlabeled target recognition, DAELM-S computes network outputs and assigns each sample the class associated with the maximum output value.The method summarizes this procedure in Algorithm 1.

B. Target Domain Adaptation ELM (DAELM-T)

DAELM-T learns a target-domain classifier from very limited labeled target data while exploiting numerous unlabeled target samples. It incorporates a source-trained base classifier through output-based regularization.

  • B. Target Domain Adaptation ELM (DAELM-T): DAELM-T learns classifier β_T from very limited labeled target samples while leveraging numerous unlabeled target-domain data.This contrasts with DAELM-S, which does not use unlabeled target data.
  • B. Target Domain Adaptation ELM (DAELM-T): A regularized ELM first trains a base classifier β_B with source data for use in DAELM-T.β_B supplies source-domain knowledge to the target-domain learning process.
  • B. Target Domain Adaptation ELM (DAELM-T): DAELM-T regularizes β_T by minimizing the discrepancy between base-classifier and target-classifier outputs on unlabeled target data.The unlabeled target samples are therefore incorporated into terminal classifier learning.
  • B. Target Domain Adaptation ELM (DAELM-T): The DAELM-T structure separates source training of β_B, tentative testing on unlabeled target data, terminal β_T learning, and β_B-to-β_T regularization.The framework supports M target domains or tasks.
  • B. Target Domain Adaptation ELM (DAELM-T): DAELM-T computes final outputs for numerous unlabeled target samples after solving the target-domain classifier.Algorithm 2 summarizes the implementation procedure.

IV. EXPERIMENTS

The experiments apply DAELM-S and DAELM-T to sensor-drift compensation using E-nose olfactory data.

  • IV. EXPERIMENTS: The experiments evaluate the proposed DAELM-S and DAELM-T algorithms on E-nose olfactory data for sensor-drift compensation.The supplied passage identifies the experimental objective but gives no results or quantitative comparisons.

A. Description of Experimental Data

The experiments use a three-year, multi-batch E-nose drift dataset and visualize drift as changing sensor-response and data-space distributions. Representative target-domain samples are selected with SSA for domain adaptation.

  • Dataset: The UCI dataset contains 13,910 measurements collected over 36 months using 16 sensors and six pure gases at varying concentrations.Each observation is represented by 128 features formed from eight features per sensor.
  • Drift visualization: Batch 1 is treated as non-drift, while batches 2, 7, and 10 represent drifted data for sensor-response visualization.The visualization covers acetone at 10, 50, 100, 150, 200, and 250 ppm across 40 samples.
  • Drift visualization: PCA projections of ten batches show significant changes in data-space distributions caused by drift over time.This motivates treating drift compensation as space-distribution adaptation rather than direct response calibration.
  • Guide-sample selection: SSA selects representative labeled guide samples from each target domain using Euclidean-distance-based selection.The procedure begins with farthest patterns and iteratively adds samples to cover the global data space.

B. Experimental Setup

The evaluation compares DAELM variants with domain-adaptation, semi-supervised, regularized-ELM, and drift-compensation baselines under two batch-based settings. Models are averaged over repeated runs to account for ELM randomness.

  • Implementation: The default configuration uses 1,000 hidden neurons, an RBF activation with kernel width 1, feature scaling to (-1,1), and empirically fixed DAELM penalties.DAELM-S uses CS=0.01 and CT=10 throughout the experiments.
  • Experimental settings: Setting 1 fixes batch 1 as the source-domain training set, whereas Setting 2 trains on batch K-1 and tests on batch K for K=2,…,10.The experiments compare recognition accuracy under fixed-source and dynamically changing source domains.
  • Compared methods: The comparison includes SVM-rbf, SVM-gfk, SVM-comgfk, ML-rbf, ML-comgfk, regularized ELM-rbf, CC-PCA, classifier ensemble, and the proposed DAELM methods.These baselines cover domain adaptation, semi-supervised learning, regularized ELM, and traditional drift compensation.
  • Evaluation protocol: ELM, DAELM-S, and DAELM-T are run ten times, with average values reported because hidden-layer parameters are randomly selected.ELM uses the same labeled source and target data as DAELM for comparison.

C. Results and Comparisons

Across the sensor-drift experiments, DAELM improves recognition and knowledge transfer over traditional ELM and baseline drift-compensation methods. Its accuracy generally increases with more labeled target-domain data, although the benefit varies by batch.

  • Overall comparisons: 91.86% is the highest recognition accuracy obtained under sensor drift using the proposed DAELM algorithm.The reported DAELM-S and DAELM-T results improve average recognition accuracy by about 30% over traditional ELM.
  • Overall comparisons: 67.3% is the average accuracy reported for ML-comgfk, exceeding the other baseline methods in Setting 1.SVM-comgfk generally outperforms CC-PCA and other SVM-based methods, except on batches 4 and 8.
  • Overall comparisons: The reported results support the robustness of the proposed domain-adaptive methods for E-nose drift compensation.This conclusion follows the comparisons across the two experimental settings and multiple batches.
  • Target-label sensitivity: DAELM-S and DAELM-T show significant classification-accuracy gains as the number of labeled target-domain samples increases.The explored sample counts are k∈{5, 10, 15, 20, 25, 30, 35, 40, 45, 50}.
  • Target-label sensitivity: Traditional ELM shows little obvious improvement when additional labeled target samples are provided, indicating limited knowledge adaptation.DAELM is comparable to ELM on batch 2, where drift is small, and on batch 10, where drift may be strongly noisy.

V. CONCLUSION AND FUTURE WORK

DAELM addresses sensor drift in E-nose systems through two domain-adaptation algorithms that transfer knowledge between source and target domains. The framework retains ELM's analytical efficiency, supports direct recognition, and motivates future online adaptation.

  • DAELM-S leverages limited labeled target-domain samples to learn a robust classifier based on the source domain.
  • DAELM-T learns from limited labeled target-domain data while leveraging a pre-learned source-domain classifier.
  • The framework supports direct recognition from algorithm outputs without cumbersome multiclass procedures.This contrasts with SVM-based one-against-one or one-against-all classification followed by voting.
  • Training takes about several seconds and testing takes microseconds because the solutions are analytically determined without iterative learning.
  • Future work will investigate online domain adaptation through incremental learning and online dynamic classifiers for nonlinear drift behavior.
Loading 1505.06405v1…