Source-linked AI summary
Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments
Romain Serizel, Nicolas Turpault, Hamid Eghbal-Zadeh, Ankit Parag Shah
TL;DR
Sound event detection systems need to locate event boundaries as well as classify events, but time-coded annotations are costly and the task data include weak labels. DCASE 2018 task 4 combines a small weakly labeled set with larger unlabeled data and evaluates temporal detection; its baseline benefits from a second pass, while short-event segmentation remains poor.
Problem
Time-coded sound-event annotations are tedious to obtain, while systems must detect event classes and boundaries from weak labels and exploit unlabeled data.
Source: §1 · p. 1Method
The task pairs a small weakly labeled training set with larger unlabeled data and evaluates systems on time-boundary sound event detection.
Source: §1 · p. 1Results
The second-pass baseline improves macro-average and every class except Dishes over the first pass, but its F1-score is close to zero for all short-event classes.
Source: §4.2 · p. 4Takeaways & Limitations
The dataset's varied event durations and boundary-sensitive metric put a strong focus on localizing events in time.
Source: §5 · p. 4The in-domain unlabeled subset has unverified Audioset labels that may not match the labeled-set distribution and must not be used during systems development.
Source: §2.1.2 · p. 2
Abstract
from arXiv · showhide
This paper presents DCASE 2018 task 4. The task evaluates systems for the large-scale detection of sound events using weakly labeled data (without time boundaries). The target of the systems is to provide not only the event class but also the event time boundaries given that multiple events can be present in an audio recording. Another challenge of the task is to explore the possibility to exploit a large amount of unbalanced and unlabeled training data together with a small weakly labeled training set to improve system performance. The data are Youtube video excerpts from domestic context which have many applications such as ambient assisted living. The domain was chosen due to the scientific challenges (wide variety of sounds, time-localized events.. .) and potential industrial applications .
1. INTRODUCTION
The task targets sound event detection in domestic audio, requiring systems to identify event classes and their time boundaries despite limited time-coded annotations. It investigates whether a small weakly labeled subset and larger unlabeled data can support this detection task.
1. INTRODUCTION
- Domestic sound event detection must identify both which events occur and their time boundaries, unlike audio tagging, which records presence without timing.Time-coded annotations are tedious to obtain and difficult to gather at the scale needed for complex deep-network systems.Source: §1 · p. 1
- The task explores using a large corpus with only a small labeled subset, drawing on Audioset classes related to domestic applications.The labels indicate clip-level event presence but omit time boundaries, reducing annotation effort.Source: §1 · p. 1
- Systems must exploit unlabeled data and estimate event boundaries from weak labels, while the evaluation heavily penalizes boundary errors.The paper notes that smoothing-based time consistency alone is insufficient for accurate boundaries.Source: §1 · p. 1
2. AUDIO DATASET
The dataset combines weakly labeled Audioset clips with in-domain and out-of-domain unlabeled training data, then evaluates event detection against time-coded annotations. Its varied event durations and overlapping events make temporal localization a central challenge.
2. AUDIO DATASET
- Audioset comprises 632 sound-event classes and 2 million human-labeled 10-second clips from 2 million YouTube videos; counts vary dramatically by class, and experts checking 10 clips per class found label mismatches in most cases.Source: §2 · p. 1
2.1. Training set
- Training data are split into a labeled set, an unlabeled in-domain set, and an unlabeled out-of-domain set to reflect a real-world scenario.The in-domain labels are weak clip-level labels; out-of-domain clips come from classes outside the task.Source: §2.1 · p. 2, §2.1.3 · p. 2
2.1.1. Labeled training set
- The labeled set contains 1,578 clips and 2,244 class occurrences with Audioset weak labels verified and corrected by human annotators.One-third of its clips contain at least two event classes.Source: §2.1.1 · p. 2
2.1.2. Unlabeled in domain training set
- The in-domain unlabeled set has 14,412 clips selected to approximate the labeled set's class distribution, but its Audioset labels are unverified and must not be used in development.The labels may be noisy, so the distribution may not exactly match the labeled set.Source: §2.1.2 · p. 2
2.1.3. Unlabeled out of domain training set
- The out-of-domain set contains 39,999 clips from classes excluded from the task, including almost 20,000 speech clips, and its unverified Audioset labels may be noisy.Speech was retained because removing it would also remove other sound classes and reduce the set's variability.Source: §2.1.3 · p. 2
2.2. Test set
- The test set contains 288 clips and 906 human-annotated events; event durations range from short to long, and overlapping events occur about 12% of the time.About 64% of clips contain multiple events, underscoring the need to distinguish events occurring at different times as well as overlapping events.Source: §2.2 · pp. 2–3, Figure 1 · p. 3
2.3. Evaluation set
- The evaluation set contains 880 ten-second clips with class distributions selected to be comparable to the training and test sets.Human time-boundary labels were scheduled for release after the DCASE 2018 challenge.Source: §2.3 · p. 3
3. TASK DESCRIPTION
Task 4 requires detecting sound events in 10-second web-video clips and returning their start and end timestamps. Systems are ranked by event-based macro F1, with boundary matching governed by onset and offset tolerances.
3. TASK DESCRIPTION
- Systems must detect sound events in 10-second clips and report start and end timestamps.Source: §3 · p. 3
3.1. Task evaluation
- Event matches allow a 200 ms onset collar and an offset tolerance of max(200 ms, 20% of event length).Source: §3.1 · p. 3
- Submissions are ranked by event-based F1 averaged equally across sound-event classes, regardless of each class’s event count.Macro-averaging gives each class equal importance in the unbalanced task scenario.Source: §3.1 · p. 3
4. BASELINE
The baseline uses a two-pass CRNN pipeline, moving from clip-level weak labels to frame-level event predictions. Its second pass improves macro-average performance and all classes except Dishes, while short-event segmentation remains poor.
4.1. System description
- The baseline extracts 64 log-mel-band magnitudes and feeds them to a CRNN with convolutional, recurrent, and dense layers.The model uses 40 ms frames with 50% overlap, three convolution layers, a 64-unit GRU, and a 10-unit sigmoid output.Source: §4.1 · pp. 3–4
- A first model trained on 1,578 weakly labeled clips predicts labels for 14,412 unlabeled in-domain clips, which train a second model.Source: §4.1 · p. 4
- The second model predicts frame-level events from clip-identical labels, then applies a 51-frame median filter to estimate onsets and offsets.Source: §4.1 · p. 4
4.2. System performance
- F1 is close to zero for short-event classes, and segmentation errors incur both false positives and false negatives.Performance is somewhat higher for variable-duration events and best for the long-event class; the authors identify time segmentation as a weak point.Source: §4.2 · p. 4
- The second pass improves macro-average F1 and every class except Dishes compared with the first pass.Source: Table 4 · p. 4, §4.2 · p. 4
5. CONCLUSION
The conclusion frames Task 4 as a realistic test of using small weakly labeled and larger unlabeled datasets for time-bounded sound-event detection. Its data and metric emphasize event localization across varied event lengths.
5. CONCLUSION
- Task 4 targets sound-event detection with time boundaries using a small weakly labeled set alongside a larger unlabeled set.Source: §5 · p. 4
- The setup reflects a realistic constraint because obtaining time-coded annotations is time-consuming and a principal bottleneck in training SED systems.Source: §5 · p. 4
- Variable event lengths and a metric that heavily penalizes segmentation errors focus the task on localizing events in time.Source: §5 · p. 4