PaperlayerView exact revision on arXiv

Source-linked AI summary

Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification

Sangmin Bae, June-Woo Kim, Won-Yang Cho, Hyerim Baek, Soyoun Son, Byungjo Lee, Changwan Ha, Kyongpil Tae, Sungnyun Kim, Se-Young Yun

arXiv:2305.14032v5eess.AScs.LGcs.SD

TL;DR

Respiratory sound classification is challenged by scarce medical data, despite demand for accurate and rapid diagnosis. The paper fine-tunes pretrained AST and combines random patch mixing with latent-space contrastive learning. It reports a new ICBHI record, 4.08% above the previous best Score.

  • Problem

    Respiratory-sound classification faces scarce medical data, privacy restrictions, and specialized annotation demands, while accurate, rapid diagnosis is increasingly sought.

    Source: §1 · p. 1

  • Method

    The method fine-tunes pretrained AST and uses random cross-sample patch mixing with contrastive learning to distinguish mixed representations in latent space.

    Source: §1 · pp. 1–2

  • Results

    62.37% Score on ICBHI sets a new record, improving 4.08% over the previous best Score.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support pretrained AST with Patch-Mix Contrastive Learning as an effective approach for respiratory-sound classification on ICBHI.

    Source: §6 · p. 4

Abstract

from arXiv · show

Respiratory sound contains crucial information for the early diagnosis of fatal lung diseases. Since the COVID-19 pandemic, there has been a growing interest in contact-free medical care based on electronic stethoscopes. To this end, cutting-edge deep learning models have been developed to diagnose lung diseases; however, it is still challenging due to the scarcity of medical data. In this study, we demonstrate that the pretrained model on large-scale visual and audio datasets can be generalized to the respiratory sound classification task. In addition, we introduce a straightforward Patch-Mix augmentation, which randomly mixes patches between different samples, with Audio Spectrogram Transformer (AST). We further propose a novel and effective Patch-Mix Contrastive Learning to distinguish the mixed representations in the latent space. Our method achieves state-of-the-art performance on the ICBHI dataset, outperforming the prior leading score by an improvement of 4.08%.

1. Introduction

Respiratory sound classification could support demand for rapid, contact-free diagnosis, but data scarcity, privacy restrictions, and reliance on medical expertise complicate development. The paper applies pretrained AST to this task and introduces patch mixing with a contrastive objective, reporting a new ICBHI benchmark result.

  • Conventional in-person auscultation depends on professional expertise, and interpretation can be subjective, motivating interest in accurate, rapid diagnostic systems.Source: §1 · p. 1
  • Medical data scarcity, privacy restrictions, and the specialized expertise required for annotation constrain respiratory-sound model development.Source: §1 · p. 1
  • Fine-tuning AST pretrained on large-scale visual and audio datasets achieves state-of-the-art respiratory sound classification, especially on ICBHI.Source: §1 · p. 2
  • 62.37% Score on ICBHI sets a new record, improving 4.08% over the previous best Score.The paper attributes this result to its proposed method.Source: §1 · p. 2
  • Patch-Mix randomly mixes patches between instances, while Patch-Mix Contrastive Learning distinguishes the mixed representations in latent space irrespective of label information.The approach addresses ambiguity in assigning labels to mixtures of hierarchical respiratory sound classes.Source: §1 · p. 2

2. Related Works

The related work positions AST as a departure from conventional residual-block models in respiratory sound classification. It also introduces simple patch replacement and a mixed contrastive objective for learning from mixed representations.

2.1. Respiratory Sound Classification

  • The study applies AST rather than conventional residual-block-based models and reports state-of-the-art performance on the ICBHI lung sound dataset.Source: §2.1 · p. 2

2.2. Mixed Representation Learning

  • Patch-Mix replaces randomly selected patches with patches from other samples, and mixed contrastive loss learns how much the mixture resembles the original in latent space.The method avoids complex operations such as teacher activation maps or guided attention.Source: §2.2 · p. 2

3. Preliminaries

The experiments use the patient-disjoint ICBHI 2017 dataset, standardize recordings and spectrogram inputs, and specify training and evaluation procedures. Performance is assessed using the challenge Score and both four-class and two-class settings.

3.1. Dataset Description

  • ICBHI 2017 contains 6,898 respiratory cycles, with an official 60% train and 40% test split and no patient overlap.The four labels are normal, crackle, wheeze, and both.Source: §3.1 · p. 2

3.2. Preprocessing Details

  • Recordings were resampled to 16 kHz, set to 8 seconds, and converted to 128-dimensional log Mel filterbank features.Long cycles were truncated, while shorter cycles were duplicated with fade-in/out; features used 25 ms windows and 10 ms overlap.Source: §3.2 · p. 2

3.3. Training Details

  • AST was fine-tuned for 50 epochs with Adam, a 5e-5 learning rate, cosine scheduling, and batch size 8; results average five random runs.SpecAugment used maximum masks of 160 time frames and 48 frequency bins.Source: §3.3 · p. 2

3.4. Evaluation Metrics

  • The ICBHI Score averages specificity and sensitivity; the two-class evaluation groups crackle, wheeze, and both as abnormal.For the two-class scenario, sensitivity is computed using a classifier trained for the four-class task.Source: §3.4 · p. 2

4. Methodology

The method fine-tunes an Audio Spectrogram Transformer pretrained on ImageNet and AudioSet, then combines random patch mixing with a contrastive loss that treats mixed-source representations as positive pairs. On ICBHI, Patch-Mix Contrastive Learning improves Score over simple AST fine-tuning.

4.1. Audio Spectrogram Transformer

  • ImageNet-and-AudioSet-pretrained AST achieved state-of-the-art performance in the 4-class comparison, while training AST from scratch was challenging.The authors report improvement with broader pretraining and suggest both domains enhance AST generalization for lung-sound classification.Source: §4.1 · p. 3

4.3. Patch-Mix Contrastive Learning

  • 2.82% Score improvement over simple AST fine-tuning came from contrasting Patch-Mixed representations across all batch samples with stop-gradient targets.The caption identifies Table 2 as covering augmentation, negative-pair, and target-representation choices.Source: Table 2 · p. 3, §4.3 · p. 4

4.2. Patch-level CutMix on AST Architecture

  • Patch-Mix randomly combines patch embeddings from selected instances without teacher activation maps or guided attention.The approach builds on Transformer inputs represented as sequences of patches or tokens.Source: §4.2 · p. 3
  • Patch and T-Patch with mixing-ratio-interpolated cross-entropy did not improve over fine-tuning, and respiratory label hierarchy makes proportional labels counterintuitive.T-Patch masks patch groups within the same time domain to mimic CutMix in raw audio.Source: §4.2 · p. 3

4.3. Patch-Mix Contrastive Learning

  • Table 3 compares overall ICBHI lung-sound classification performance using the official 60-40% train-test split.The caption notes that the comparison includes previous works and marks the previous state-of-the-art Score.Source: Table 3 · p. 4
  • The contrastive loss treats mixed-source representations as positive pairs, weighting similarity to each source by the mixing ratio while including positive and negative pairs in the denominator.The mixing ratio lets the model learn how much inherent information from the original samples a mixed representation contains.Source: §4.3 · p. 4

5. ICBHI Dataset Results

Patch-Mix Contrastive Learning achieved state-of-the-art ICBHI scores in both 4-class and 2-class evaluations. Straightforward AST fine-tuning also outperformed the current state-of-the-art model.

  • Pretrained models on large-scale datasets showed relatively high performance across architectures and methods.Source: §5 · p. 4
  • 62.37% Score in the 4-class evaluation and 68.71% in the 2-class evaluation made Patch-Mix Contrastive Learning state of the art.Source: §5 · p. 4
  • 59.55% Score from straightforward AST fine-tuning outperformed the current state-of-the-art model by 1.26%.Source: §5 · p. 4
  • The authors report an approximately 3% increase over simple AST fine-tuning, compared with 1–2% increases from prior methods within their architectures.Source: §5 · p. 4

6. Conclusion

The authors demonstrate AST pretrained on visual and audio datasets for respiratory sound classification and introduce Patch-Mix augmentation and Patch-Mix Contrastive Learning. Their method achieved a new record on ICBHI, while conventional mixed loss did not improve performance, likely because of lung-sound label hierarchy.

  • AST pretrained on visual and audio datasets improves respiratory-sound classification Score.Source: §6 · p. 4
  • Patch-Mix randomly replaces patches with patches from other samples, and Patch-Mix Contrastive Learning distinguishes the resulting latent representations.Source: §6 · p. 4
  • Conventional mixed loss did not improve performance, likely due to the hierarchy of lung-sound labels.Source: §6 · p. 4
  • 62.37% Score on ICBHI set a new record and outperformed all existing methods.Source: §6 · p. 4
Loading 2305.14032v5…