PaperlayerView exact revision on arXiv

Source-linked AI summary

PAC-KD: Pathology-Aware Compositional Knowledge Distillation for Respiratory Sound Classification

Li Xiao, Weiping Tu, Youqiang Zheng, Dan Chen

arXiv:2610.22826v1cs.SD

TL;DR

Four-class respiratory sound distillation does not explicitly exploit events shared across categories, leaving transfer of event-specific knowledge less explored. PAC-KD distills crackle and wheeze knowledge from dedicated specialists into a four-class student, with stable conflict calibration. It achieves the highest mean Score among compared methods on both ICBHI and SPRSound.

  • Problem

    Existing four-class knowledge distillation does not explicitly exploit shared respiratory events, leaving transfer of crackle- and wheeze-related knowledge across individual abnormal classes and Both less explored.

    Source: §I · p. 1

  • Method

    PAC-KD transfers crackle and wheeze presence knowledge from two independent specialists to a unified four-class student's corresponding attribute marginals, with stable conflict calibration regulating guidance.

    Source: §I · p. 1, A. Pathology Specialist Learning · p. 2

  • Results

    PAC-KD records the highest mean Score on both datasets: 65.28% on ICBHI, 0.89 percentage points above BTSd (k = 15), and 89.54±0.67% on SPRSound, 0.24 percentage points above it.

    Source: A. Main Results · pp. 3–4

  • Takeaways & Limitations

    The findings support compositional knowledge transfer without increasing student inference cost.

    Source: §V · p. 4

Abstract

from arXiv · show

Respiratory sound classification distinguishes four categories: Normal, Crackle, Wheeze, and Both (crackle and wheeze). Existing knowledge distillation transfers four-class predictions without explicitly exploiting shared respiratory events, potentially limiting the transfer of event-specific knowledge across individual abnormal classes and the Both class. We propose PAC-KD, a pathology-aware compositional knowledge distillation framework that learns crackle and wheeze presence through two dedicated specialists. Their complementary knowledge is transferred to a unified four-class student, enabling it to effectively capture pathology-specific attributes while preserving class-level discrimination. Furthermore, stable conflict calibration adaptively regulates specialist guidance according to its consistency with classification supervision, mitigating potential supervision conflicts. Experiments on ICBHI and SPRSound achieve Scores of 65.28\% and 89.54\%, respectively, outperforming strong knowledge distillation baselines on both datasets.

I. INTRODUCTION

Four-class respiratory sound distillation does not explicitly exploit events shared across categories, leaving specialist-to-student transfer of crackle and wheeze knowledge less explored. PAC-KD addresses this with independent pathology specialists, compositional attribute distillation, and conflict-calibrated guidance.

I. INTRODUCTION

  • Prior respiratory sound work includes ensemble-based four-class distillation, while event relationships have also been used in data construction or target-classifier objectives.Transferring complementary event knowledge from independent specialists to a unified student remains less explored.Source: §I · p. 1
  • Four-class distillation can underemphasize shared respiratory events, potentially limiting transfer of crackle- and wheeze-related knowledge across abnormal classes and Both.The four categories compose from crackle and wheeze presence, with Both representing their co-occurrence.Source: §I · p. 1
  • PAC-KD transfers complementary crackle and wheeze knowledge from dedicated specialists to a unified four-class student while preserving class-level discrimination.Stable conflict calibration adjusts specialist guidance according to its consistency with classification supervision.Source: §I · p. 1

II. PROPOSED METHOD

  • PAC-KD comprises pathology specialist learning, compositional attribute distillation, and stable conflict-calibrated learning.The introduction situates the task in automated identification of crackles and wheezes in auscultation recordings.Source: §II · p. 1, §I · p. 1

A. Pathology Specialist Learning

PAC-KD represents each respiratory-cycle class with binary crackle and wheeze attributes, then trains separate specialists to learn each event across individual and co-occurring categories.

A. Pathology Specialist Learning

  • The four respiratory-cycle labels are Normal, Crackle, Wheeze, and Both.The class order is used for the two-attribute representation.Source: A. Pathology Specialist Learning · p. 1
  • The four classes map to crackle–wheeze attribute pairs (0, 0), (1, 0), (0, 1), and (1, 1), respectively.The mapping uses an indicator function for each event's presence.Source: A. Pathology Specialist Learning · p. 2
  • Two independent binary specialists are trained with binary cross-entropy on the corresponding crackle or wheeze attribute targets.Each specialist uses a separate network with the same backbone architecture as the four-class comparison teacher.Source: A. Pathology Specialist Learning · p. 2
  • The crackle specialist treats Crackle and Both as positive, while the wheeze specialist treats Wheeze and Both as positive.This lets each specialist learn event-presence knowledge shared across individual and co-occurring abnormal categories.Source: A. Pathology Specialist Learning · p. 2

B. Compositional Attribute Distillation

The four-class student is aligned with two binary specialists by distilling its crackle and wheeze presence marginals, allowing Both to receive both event-specific signals while retaining the baseline objective.

B. Compositional Attribute Distillation

  • The student produces a four-class distribution ordered as Normal, Crackle, Wheeze, and Both, which is marginalized into crackle and wheeze presence probabilities.These marginals align the student predictions with the binary specialists.Source: B. Compositional Attribute Distillation · p. 2
  • The corresponding crackle and wheeze Bernoulli divergences are aggregated into the attribute distillation loss.This supervision lets the Both class receive simultaneous crackle and wheeze guidance.Source: B. Compositional Attribute Distillation · p. 2
  • PAC-KD supplements the baseline training objective with compositional specialist supervision while preserving the baseline learning formulation.The divergences are defined between Bernoulli distributions and computed over the mini-batch.Source: B. Compositional Attribute Distillation · p. 2

C. Stable Conflict-Calibrated Learning

Stable conflict calibration compares classification and attribute-distillation gradients, then adjusts specialist guidance when they conflict while leaving the class-supervised term unchanged.

C. Stable Conflict-Calibrated Learning

  • Gradient compatibility is measured between the classification and attribute-distillation objectives with respect to student logits; positive values indicate compatible directions and negative values indicate conflict.The Frobenius inner product and norm are used, with a positive epsilon for numerical stability.Source: C. Stable Conflict-Calibrated Learning · p. 2
  • An exponential moving average smooths compatibility over mini-batches, and the smoothed value determines a bounded distillation coefficient.The nominal weight, correction strength, and minimum coefficient constrain the adjustment.Source: C. Stable Conflict-Calibrated Learning · pp. 2–3
  • When smoothed compatibility is nonnegative, the nominal weight is retained; otherwise specialist guidance is attenuated within [λmin, λ0] while class supervision remains unchanged.This one-sided calibration does not eliminate specialist knowledge.Source: C. Stable Conflict-Calibrated Learning · p. 3

A. Datasets and Preprocessing

The evaluation uses public ICBHI and SPRSound datasets with official patient-wise train–test splits, harmonizing SPRSound labels to the four respiratory categories.

A. Datasets and Preprocessing

B. Experimental Settings

The experiments compare PAC-KD with respiratory-sound classifiers and knowledge-distillation baselines, using repeated training runs and reporting specificity, sensitivity, and their mean Score.

B. Experimental Settings

  • The comparison includes conventional respiratory-sound classifiers and knowledge-distillation baselines, including BTSd with five mean-teacher or 15 random-teacher models.Other named comparisons include Patch-Mix CL, RepAugment, BTS, BEATs, PAFA, QLung (Audio-CLAP), and BTS-CARD.Source: B. Experimental Settings · p. 3
  • Table II reports respiratory-cycle counts for ICBHI and harmonized SPRSound splits, with an inter-patient test split for SPRSound.Source: Table II · p. 3
  • Student models use Adam, a peak learning rate of 5×10−5, batch size 32, and 100 epochs; PAC-KD and optimization-matched controls use five warmup epochs.Experiments use five fixed random seeds and report mean and standard deviation.Source: B. Experimental Settings · p. 3
  • The reported metrics are Specificity (Sp), Sensitivity (Se), and their mean Score.Source: B. Experimental Settings · p. 3
  • For each class c, Cc and Nc denote the correctly classified and total cycles, respectively.Source: B. Experimental Settings · p. 3

A. Main Results

PAC-KD leads the compared methods on ICBHI and SPRSound, while its ablations associate specialist supervision with stronger sensitivity and SCC with a more balanced sensitivity–specificity profile.

A. Main Results

  • 65.28% mean Score on ICBHI exceeds PAFA and BTSd (k = 15) by 0.44 and 0.89 percentage points, respectively; 89.54±0.67% on SPRSound surpasses BTSd by 0.24 points.These are the highest mean Scores among the compared methods on each dataset.Source: A. Main Results · pp. 3–4
  • Sensitivity rises while specificity falls on both datasets, indicating gains mainly in abnormal-cycle recognition rather than a uniform improvement.On ICBHI, sensitivity changes from 45.90% to 51.21% and specificity from 82.89% to 79.35%; on SPRSound, Se changes from 87.76% to 89.67% and Sp from 90.83% to 89.40%.Source: A. Main Results · p. 4

B. Ablation Study

  • Replacing one four-class teacher with two pathology specialists raises ICBHI Score from 64.10% to 65.02% and Se from 49.48% to 52.39%.The ablation attributes the change to transferring complementary crackle and wheeze knowledge through compositional attribute distillation.Source: B. Ablation Study · p. 4, Table III · p. 4
  • Adding SCC after specialist distillation raises Sp from 77.64% to 79.35%, while Se changes from 52.39% to 51.21%, producing the highest Score of 65.28%.The passage describes SCC as balancing classification and distillation supervision.Source: B. Ablation Study · p. 4, Table III · p. 4
  • Across CNN6, AST, and PAFA students, PAC-KD improves mean Score over CE training by 0.40, 0.37, and 0.44 percentage points, respectively.Sensitivity also increases by 7.17, 2.75, and 3.58 points, accompanied by lower specificity.Source: B. Ablation Study · p. 4, Table IV · p. 4
  • Figure 2 compares class-wise attribute errors for a four-class teacher and pathology specialists in shared crackle–wheeze space; lower errors are better.The supplied caption identifies the comparison but does not give its class-wise outcomes.Source: Figure 2 · p. 4, B. Ablation Study · p. 4

C. Teacher Knowledge Analysis.

The specialists reduce attribute errors for the three abnormal categories, including co-occurring Both, while Normal error increases. The conclusion presents PAC-KD as transferring complementary event knowledge without increasing student inference cost.

C. Teacher Knowledge Analysis.

  • Specialists reduce attribute errors for Crackle from 0.186 to 0.103, Wheeze from 0.339 to 0.236, and Both from 0.440 to 0.220, while Normal error rises from 0.124 to 0.218.The passage says the abnormal-category error reductions are consistent with higher sensitivity, while the Normal increase corresponds to lower specificity.Source: C. Teacher Knowledge Analysis · p. 4

V. CONCLUSION

  • PAC-KD transfers complementary crackle and wheeze knowledge from dedicated specialists, and only the four-class student is retained at inference without added cost.Stable conflict calibration regulates specialist guidance while preserving the original classification supervision.Source: §V · p. 4
Loading 2610.22826v1…