Source-linked AI summary
Description and Discussion on DCASE 2021 Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring under Domain Shifted Conditions
Yohei Kawaguchi, Keisuke Imoto, Yuma Koizumi, Noboru Harada, Daisuke Niizumi, Kota Dohi, Ryo Tanabe, Harsh Purohit, Takashi Endo
TL;DR
Unsupervised ASD must detect unknown anomalous sounds from normal-only training data, while domain shifts can make normal training and testing sounds acoustically different. The paper presents the DCASE 2021 domain-shift benchmark and analyzes 75 submissions from 26 teams. The TOP-5 teams used OE–IM detector ensembles or IM detection on features learned for machine identification.
Problem
Unsupervised ASD must detect unknown anomalous sounds from normal-only training data, while domain shifts can make normal training and testing sounds acoustically different.
Source: §1 · p. 1Method
The paper presents the DCASE 2021 domain-shift benchmark and analyzes 75 submissions from 26 teams.
Source: §1 · p. 1Results
The TOP-5 teams used OE–IM detector ensembles or IM detection on features learned for machine identification.
Source: §4.1 · p. 3, §5 · p. 4Takeaways & Limitations
Both approaches are promising, but domain-adaptation performance still has room for improvement.
Source: §5 · p. 4Features effective for machine identification are not guaranteed to remain effective after domain shift.
Source: §4.3 · p. 4
Abstract
from arXiv · showhide
We present the task description and discussion on the results of the DCASE 2021 Challenge Task 2. In 2020, we organized an unsupervised anomalous sound detection (ASD) task, identifying whether a given sound was normal or anomalous without anomalous training data. In 2021, we organized an advanced unsupervised ASD task under domain-shift conditions, which focuses on the inevitable problem of the practical use of ASD systems. The main challenge of this task is to detect unknown anomalous sounds where the acoustic characteristics of the training and testing samples are different, i.e., domain-shifted. This problem frequently occurs due to changes in seasons, manufactured products, and/or environmental noise. We received 75 submissions from 26 teams, and several novel approaches have been developed in this challenge. On the basis of the analysis of the evaluation results, we found that there are two types of remarkable approaches that TOP-5 winning teams adopted: 1) ensemble approaches of ``outlier exposure'' (OE)-based detectors and ``inlier modeling'' (IM)-based detectors and 2) approaches based on IM-based detection for features learned in a machine-identification task.
1. INTRODUCTION
The paper introduces the DCASE 2021 unsupervised ASD benchmark, where systems must detect unknown anomalies despite changed acoustic conditions between training and testing. It reviews the challenge and analyzes submissions to identify prominent approaches.
1. INTRODUCTION
- ASD identifies whether a sound emitted from a machine is normal or anomalous, a capability described as useful for machine condition monitoring.The paper connects prompt sound-based anomaly detection with automatic mechanical-failure detection in factory automation.Source: §1 · p. 1
- The 2020 challenge attracted 117 entries from 40 teams and yielded several new approaches.The paper presents this as evidence of participant interest in the real-world-oriented task.Source: §1 · p. 1
- Domain shift can make a normal sound from a changed operating condition appear anomalous to a system trained under the original condition.The paper illustrates this with a conveyor motor whose speed falls from 300–400 RPM in summer to 100–200 RPM in winter.Source: §1 · p. 1
- The DCASE Challenge 2021 Task 2 is presented as the first benchmark task for domain-shift problems in ASD, covering differences in operating speed, machine load, environmental noise, and more.The authors discuss notable approaches and potential problems based on analysis of all 75 submissions from 26 teams.Source: §1 · p. 1
2. UNSUPERVISED ANOMALOUS SOUND DETECTION UNDER DOMAIN-SHIFTED CONDITIONS
In domain-shifted ASD, training data come from a source condition while detection occurs in a different target condition. The task remains to identify anomalous sounds even though only normal samples are available for training.
2. UNSUPERVISED ANOMALOUS SOUND DETECTION UNDER DOMAIN-SHIFTED CONDITIONS
- A sound clip x is evaluated by an anomaly-score calculator A with parameters θ, which outputs Aθ(x).The calculator also receives file-path information, and its score is designed to be larger for anomalous sounds.Source: §2 · p. 1
- The core training difficulty is making the anomaly score large for anomalous sounds when training data contain only normal sounds.Source: §2 · p. 2
- The system must distinguish target-domain normal sounds from target-domain anomalies even though training samples come from a different source domain.Only a few normal clips from the target domain are provided in the 2021 setting.Source: §2 · p. 2
3. TASK SETUP
The setup specifies seven-machine data and its train/test organization, an AUC/pAUC evaluation, and two baseline anomaly detectors.
3.1. Dataset
- The task uses noisy, 10-second, 16 kHz single-channel recordings from seven machine types in ToyADMOS2 and MIMII DUE.Machine sounds are mixed with environmental noise recorded in several real factory environments.Source: §3.1 · p. 2
- Sections group source- and target-domain data within a machine type; development covers sections 00–02, while sections 03–05 supply additional training and unlabeled evaluation tests.Sections may contain multiple products, and products may appear across sections.Source: §3.1 · p. 2
3.2. Evaluation metrics
- AUC and pAUC are evaluated by machine type, section, and domain; pAUC focuses on FPR 0–0.1, and the official score harmonically combines both metrics.The metric calculation does not require a threshold value.Source: §3.2 · pp. 2–3
3.3. Baseline systems and results
- The AE baseline scores reconstruction error, relying on the assumption that it cannot reconstruct sounds absent from normal-only training data.Its anomaly score is the mean square reconstruction error.Source: §3.3 · p. 3
- The MobileNetV2 outlier-exposure baseline identifies machine sections from log-mel features and scores clips using the averaged negative logit for the correct section.Sections serve as classes, and the correct-section probability comes from the model’s softmax output.Source: §3.3 · p. 3
- Tables 1 and 2 report AUC and pAUC for the AE and MobileNetV2 baselines, respectively, using averages and standard deviations from five independent trials.Source: §3.3 · p. 3, Table 1 · p. 3, Table 2 · p. 3
4. CHALLENGE RESULTS AND DISCUSSION
The evaluation attracted 75 submissions, with 20 teams outperforming the baselines, and the top-ranked solutions featured parallel OE–IM ensembles or serial IM detection on machine-identification features. These approaches offer different robustness and complexity trade-offs, while domain adaptation remains an improvement area.
4.1. Results for evaluation dataset
- The study received 75 submissions from 26 teams; 20 teams achieved better performance than the baseline systems, and target-domain scores roughly correlated with the official ranking.Figure 1 compares AUC for source domains (top) and target domains (bottom).Source: §4.1 · p. 3, Figure 1 · p. 4
4.2. Parallel-type hybrid approach: ensemble of OE-based and IM-based detectors
- High-ranking teams used parallel ensembles of OE- and IM-based detectors, which can reduce OE weaknesses by leveraging IM robustness.The first-, fourth-, and fifth-place teams used this approach; the 2021 first-place team reported outstanding target-domain performance without domain adaptation.Source: §4.2 · p. 4
- The fourth- and fifth-place teams used domain adaptation, and the fifth-place team exceeded 55% AUC on every machine type in both domains.The fifth-place model was small despite the approach’s tendency to increase model complexity; improving adaptation while maintaining compactness remains a research problem.Source: §4.2 · p. 4
4.3. Serial-type hybrid approach: IM-based detection for features learned in a machine-identification task
- The second- and third-place teams used serial hybrids: machine-identification features were followed by IM detection, combining OE feature extraction with IM robustness without ensemble complexity.They adapted only the IM detector, not the feature extractor; the authors note that machine-identification features are not guaranteed to remain effective after domain shift.Source: §4.3 · p. 4
5. CONCLUSION
The DCASE 2021 Task 2 analysis identifies two promising approaches to unsupervised anomaly detection under domain shift: parallel OE–IM ensembles and serial IM detection on machine-identification features. Both leave room to improve domain adaptation, with future work differing by approach.
5. CONCLUSION
- The top-five solutions featured parallel OE–IM ensembles or serial IM detection on machine-identification features, and both approaches leave room for domain-adaptation improvement.For parallel hybrids, future work should improve adaptation while maintaining model compactness; for serial hybrids, it should test the effective range and improve adaptation performance.Source: §5 · p. 4