Lung Sound Classification Using Co-tuning and Stochastic Normalization

Truc Nguyen; Franz Pernkopf

Source-linked AI summary

Lung Sound Classification Using Co-tuning and Stochastic Normalization

Truc Nguyen, Franz Pernkopf

arXiv:2108.01991v1Aug 4, 2021eess.AScs.LGcs.SD

TL;DR

The paper addresses adventitious lung sound and respiratory disease classification using limited and imbalanced recordings with device variation. It applies pre-trained ResNet backbones with vanilla fine-tuning, co-tuning, stochastic normalization, augmentation, and spectrum correction. The proposed systems mostly outperform state-of-the-art systems across both datasets.

Problem
The study targets robust adventitious lung sound and respiratory disease classification across imbalanced datasets and recordings made with different devices.
Method
Pre-trained ResNet backbones are transferred using vanilla fine-tuning, co-tuning, stochastic normalization, and their combination, with augmentation and spectrum correction.
Results
The proposed systems mostly outperform state-of-the-art systems for adventitious lung sound and respiratory disease classification on both datasets.
Takeaways & Limitations
Co-tuning performs best for adventitious lung sound classification, while stochastic normalization and its combination with co-tuning achieve higher respiratory disease classification performance.

Abstract

from arXiv · show

In this paper, we use pre-trained ResNet models as backbone architectures for classification of adventitious lung sounds and respiratory diseases. The knowledge of the pre-trained model is transferred by using vanilla fine-tuning, co-tuning, stochastic normalization and the combination of the co-tuning and stochastic normalization techniques. Furthermore, data augmentation in both time domain and time-frequency domain is used to account for the class imbalance of the ICBHI and our multi-channel lung sound dataset. Additionally, we apply spectrum correction to consider the variations of the recording device properties on the ICBHI dataset. Empirically, our proposed systems mostly outperform all state-of-the-art lung sound classification systems for the adventitious lung sounds and respiratory diseases of both datasets.

I. INTRODUCTION

The paper targets automated adventitious lung sound and respiratory disease classification using transferred knowledge from pre-trained ResNet models. It combines multiple fine-tuning and normalization strategies with augmentation and spectrum correction to improve robustness across two datasets.

Motivation: Lung sound analysis supports classification of adventitious sounds and respiratory diseases from auscultation recordings.The tasks include distinguishing normal, crackle, wheeze, and combined sounds, alongside several respiratory disease groupings.
Approach: The study uses ImageNet-pre-trained ResNet models as backbones and transfers their knowledge through vanilla fine-tuning, co-tuning, stochastic normalization, and their combination.Spectrograms are converted to three channels to match the pre-trained model input.
Transfer learning: Co-tuning collaboratively fine-tunes source- and target-domain representation and task-specific layers using learned category relationships between domains.Source probabilistic labels and target one-hot labels jointly supervise the fine-tuning process.
Normalization: Stochastic normalization replaces Batch Normalization to reduce sensitivity to distribution shifts and help avoid over-fitting on the small target datasets.It normalizes activations using either mini-batch or moving statistics and preserves prior knowledge from pre-trained moving statistics.
Contributions: The systems address dataset imbalance with time and time-frequency augmentation, apply spectrum correction for recording-device variation, and report robust classification systems for both datasets.Flipping samples doubles the target-domain training data, while spectrum correction accounts for device differences in the ICBHI dataset.

II. DATABASES

The study evaluates adventitious sound and respiratory disease classification on the ICBHI database and a multi-channel lung sound database. The proposed pipeline preprocesses recordings into fixed-size features and aggregates segment predictions to classify each cycle or recording.

Datasets: The ICBHI database contains 920 annotated recordings from 126 subjects covering healthy status and seven disease categories recorded with four stethoscopes.Recordings range from 10 to 90 seconds and use sampling rates from 4000Hz to 44100Hz.
Tasks: Adventitious lung sound classification includes four-class Normal, Crackles, Wheezes, and combined classification, plus two-class normal-versus-abnormal classification.The ICBHI official split uses 60% of data for training and 40% for testing, with different patients in each set.
Tasks: Respiratory disease classification includes three-class healthy, chronic, and non-chronic grouping and two-class healthy-versus-unhealthy classification.The unhealthy class contains the seven diseases represented in the ICBHI database.
Datasets: The multi-channel database contains lung sounds from 16 healthy subjects and 7 patients with idiopathic pulmonary fibrosis, recorded using a 16-channel device.Recordings were made over the posterior chest at two airflow rates with 3–8 respiratory cycles within 30 seconds.
Pipeline: The pipeline preprocesses respiratory cycles or recordings, transforms them into fixed-size log-mel spectrograms, and uses majority voting over 8-second segments during inference.The CNN receives the extracted features, while segment-level predictions are aggregated for the same recording.

A. Audio Pre-processing and Feature Extraction

Audio is resampled, segmented, padded, and transformed into normalized log-mel spectrograms for ResNet processing. Spectrum correction calibrates device-dependent frequency responses, while augmentation addresses severe class imbalance.

Pre-processing: Recordings are resampled to 16kHz for adventitious sound classification and 4kHz for ICBHI respiratory disease classification, then split into non-overlapping segments.Time-reversed padding creates fixed-length segments without abrupt signal changes.
Feature extraction: The pipeline computes FFT-based log-mel spectrograms, normalizes them to zero mean and unit variance, and duplicates them into three channels for pre-trained ResNet input.The FFT uses 512-sample windows with 50% overlap; the mel-bin count is 50 for ICBHI and 45 for the multi-channel dataset.
Spectrum correction: Different recording devices have different frequency responses, causing performance degradation for underrepresented devices and motivating spectrum correction.Calibration coefficients are computed from reference-device spectra and applied to device STFT magnitudes.
Spectrum correction: Spectrum correction averages device spectra, forms a reference spectrum from selected devices, and scales each device spectrum using frequency-wise coefficients.The authors report that correction in the spectrogram domain is more successful than correction in the log-mel domain.
Data augmentation: Because normal cycles and COPD recordings dominate the datasets, time-domain and time-frequency augmentation is used to balance training data and prevent over-fitting.Time stretching doubles selected ICBHI classes for adventitious sound classification and all classes for respiratory disease classification.

1) Time Domain:

The paper uses transferred knowledge from pre-trained models and augments lung-sound data through time- and frequency-domain transformations. These methods support classification while addressing limited data and dataset variation.

Transfer learning: Pre-trained models reduce the data and training requirements for lung-sound classification through transfer learning.The paper transfers knowledge from models trained on other datasets because collecting lung-sound data is time-consuming.
Transfer learning: The study evaluates ResNet18, ResNet34, ResNet50, and ResNet101 as backbone architectures.

2) Co-tuning:

Co-tuning transfers knowledge by learning relationships between source and target categories, then jointly using source probabilistic labels and target labels during fine-tuning.

Co-tuning: Co-tuning learns source-to-target category relationships before collaboratively supervising fine-tuning with translated source labels and target labels.
Collaborative supervision: The co-tuning objective combines target supervision with source supervision weighted by λ.
Inference: During inference, the source task-specific layers are removed to avoid additional computational cost.
Category relationship: The direct approach estimates p(ys|yt) by averaging pre-trained source-model predictions over samples in each target category.
Category relationship: The reverse approach learns ys → yt from source predictions and target labels, then derives p(ys|yt) using Bayes’s rule.

3) Stochastic Normalization (StochNorm):

Stochastic normalization replaces Batch Normalization with two alternative statistics branches and randomly selects between them during training, while initializing moving statistics from the pre-trained model.

Architecture: Stochastic normalization replaces Batch Normalization with parallel mini-batch-statistics and moving-statistics branches.
Regularization: Stochastic normalization is used as architecture regularization to reduce over-dependence on sample statistics during fine-tuning.
Initialization: Moving statistics are initialized from corresponding pre-trained-model parameters to exploit prior knowledge.
Normalization statistics: The mini-batch branch uses the current batch mean and variance, whereas the other branch uses moving statistics.
Training: The stochastic selection mechanism randomly chooses a normalization branch for each channel during training.

4) Combination of Co-tuning and Stochastic Normalization:

The combined system first computes category relationships for co-tuning, then replaces Batch Normalization in its ResNet backbone with stochastic-normalization modules before fine-tuning.

Combination of methods: The combined method computes the co-tuning category relationship before replacing Batch Normalization with StochNorm modules in the backbone.
Combination of methods: Fine-tuning is then performed on the resulting co-tuning architecture with stochastic normalization.

IV. EXPERIMENTS

The experiments evaluate transfer learning across ResNet architectures and datasets, with ablations for segment length, spectrum correction, and flipping augmentation. Performance is measured with task-specific sensitivity, specificity, average, harmonic, precision, recall, and F1 metrics under patient-independent splits.

The study evaluates ImageNet-pretrained ResNets on ICBHI adventitious lung sound classification and respiratory disease classification.
Ablations examine respiratory segment length, spectrum correction, and flipping data augmentation.
The multi-channel dataset is evaluated with ImageNet- and ICBHI-pretrained ResNets for crackle detection, alongside comparisons with state-of-the-art systems.
ALSC and RDC use sensitivity, specificity, average score, and harmonic score, while crackle detection additionally uses precision, recall, and F1-score.
The ICBHI ALSC experiments use official patient-independent splits, while the multi-channel dataset uses 7-fold subject-independent cross-validation.

C. Experimental Results

The results compare transfer-learning methods and preprocessing choices across ALSC and RDC tasks. Co-tuning is strongest for 4-class ALSC, whereas stochastic normalization is especially effective for 3-class RDC; 8-second segments, spectrum correction, and flipping augmentation improve the evaluated systems.

1) Transfer learning techniques for different ResNets: ResNet50 is the best backbone for 4-class ALSC, and co-tuning achieves the best performance at approximately 58%.The combined CoTuning-StochNorm method does not outperform the original techniques for this task.
1) Transfer learning techniques for different ResNets: Co-tuning and CoTuning-StochNorm produce more separated 4-class training representations than vanilla fine-tuning and stochastic normalization in the ResNet50 t-SNE visualization.
1) Transfer learning techniques for different ResNets: ResNet101 performs best for 3-class RDC, with StochNorm and CoTuning-StochNorm outperforming CoTuning and vanilla fine-tuning.ResNet50 performs better than the other architectures for vanilla fine-tuning.
2) Respiratory segment length: 8-second fixed-length segments obtain the best average score for 4-class ALSC and are used for the other evaluated tasks.
3) Spectrum correction: All-device spectrum correction improves co-tuned ResNet50 performance by 1.62% absolute over no spectrum calibration.The correction is consequently applied to both adventitious lung sound and respiratory disease classification.
4) Flipping data augmentation: Flipping augmentation improves accuracy by about 3% for vanilla fine-tuning, 2% for stochastic normalization, and 1% for co-tuning without spectrum correction.Combining spectrum calibration with flipping augmentation consistently enhances robustness.

5) Effect of pre-trained model on the multi-channel lung sound dataset:

On the multi-channel lung sound dataset, co-tuning transfers pretrained knowledge effectively for crackle detection, with ImageNet pretraining and smaller ResNets performing best. Across the broader ICBHI evaluations, the proposed systems generally outperform competing systems, especially for RDC with stochastic normalization.

5) Effect of pre-trained model on the multi-channel lung sound dataset: Co-tuning with ImageNet pretraining consistently outperforms co-tuning with ICBHI pretraining for crackle detection.
5) Effect of pre-trained model on the multi-channel lung sound dataset: ResNet34 achieves the best crackle-detection performance, followed by ResNet18, ResNet50, and ResNet101.
5) Effect of pre-trained model on the multi-channel lung sound dataset: Co-tuning with pretrained ICBHI or ImageNet models achieves better accuracy than vanilla fine-tuning with an ImageNet-pretrained model on the small lung sound dataset.
5) Effect of pre-trained model on the multi-channel lung sound dataset: Overall, co-tuning performs best for ALSC, while stochastic normalization or its combination with co-tuning performs best for RDC.ResNet34 and ResNet50 are more suitable for ALSC, whereas ResNet101 is more robust for RDC in most settings.
1) Comparison to state-of-the-art systems using the ICBHI dataset: The best ICBHI ALSC average scores are 58.29% for 4 classes and 64.74% for 2 classes using co-tuned ResNet50.
1) Comparison to state-of-the-art systems using the ICBHI dataset: Stochastic normalization with ResNet101 reaches 92.72% average score for official-split 3-class RDC and 95.73% under five-fold cross-validation.The five-fold result is reported as approximately 5% better than state-of-the-art systems.

2) Comparison for our multi-channel lung sound dataset:

The multi-channel lung sound literature includes conventional feature-based classifiers and deep learning systems, with substantial variation in datasets, splits, and evaluation protocols. The proposed transfer-learning systems outperform the previous multi-input CNN baseline on this dataset.

The study compares proposed transfer-learning systems with a previous multi-input CNN system on the multi-channel lung sound dataset.The comparison is presented in Table V.
The co-tuned ResNet34 system achieves the best performance, closely followed by stochastic normalization with pre-trained ResNet50.
2.82% improvement in best F1-score over the previous multi-input fine-tuned system is reported.
Comparisons on ICBHI are difficult because some studies use non-official data splits or different evaluation metrics.
Conventional approach: Prior studies use conventional classifiers with time- or frequency-domain features, including MFCCs, wavelets, and time-domain features.Reported methods include hidden Markov models, Gaussian mixture models, support vector machines, and RUSBoost.
Deep learning approach: Deep learning systems combine CNNs, RNNs, and hybrid architectures with augmentation, ensembles, or transfer learning to improve robustness.

2) Deep learning approach:

Deep learning approaches use CNNs, RNNs, and hybrid models with augmentation and transfer learning across adventitious sound and respiratory disease classification tasks. The paper positions its proposed ResNet-based systems within these comparisons and reports broad state-of-the-art performance advantages.

Deep learning approach: Deep learning systems use CNNs, RNNs, and hybrid architectures for lung sound classification.
Deep learning approach: Prior CNN and RNN systems report results across four-class, two-class, ternary, and respiratory disease classification tasks.Reported evaluations use different splits and cross-validation settings.
Deep learning approach: Deep learning studies use augmentation and architectural techniques such as snapshot ensembles, temporal stretching, VTLP, mixup, attention blocks, and device-specific fine-tuning.
Multi-channel classification: A multi-channel CNN-RNN framework achieved 92% on seven-fold cross-validation for healthy-versus-pathological breathing-cycle classification.
Proposed approach: The proposed systems use vanilla fine-tuning, co-tuning, stochastic normalization, and their combination across ResNet architectures, with spectrum correction and flipping augmentation.
Reported comparison: The authors report that their proposed systems outperform almost all state-of-the-art systems for adventitious sound and respiratory disease classification.