Source-linked AI summary
A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling
Yun Wang, Juncheng Li, Florian Metze
TL;DR
Weakly labeled SED must infer temporal localization while training data provides only recording-level event types, making the MIL pooling function a central design choice. The paper compares five pooling functions theoretically and experimentally, finds linear softmax best among them, and uses it to build TALNet for strong tagging and localization performance.
Problem
Weak labeling supplies event types present in recordings but not temporal information, so the paper examines how MIL pooling affects sound-event localization.
Method
The paper derives gradients for five pooling functions and evaluates them experimentally on DCASE 2017 and Audio Set within MIL-based SED systems.
Results
Linear softmax performs best among the five pooling functions, and TALNet achieves strong audio tagging and localization performance simultaneously.
Takeaways & Limitations
Linear softmax offers unobstructed gradient flow, balanced localization errors, and relatively consistent recording- and frame-level predictions for MIL-based SED.
Takeaways & Limitations
Linear softmax is not necessarily optimal; adaptive or generalized weighting schemes remain possible alternatives.
Abstract
from arXiv · showhide
Sound event detection (SED) entails two subtasks: recognizing what types of sound events are present in an audio stream (audio tagging), and pinpointing their onset and offset times (localization). In the popular multiple instance learning (MIL) framework for SED with weak labeling, an important component is the pooling function. This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization. Although the attention pooling function is currently receiving the most attention, we find the linear softmax pooling function to perform the best among the five. Using this pooling function, we build a neural network called TALNet. It is the first system to reach state-of-the-art audio tagging performance on Audio Set, while exhibiting strong localization performance on the DCASE 2017 challenge at the same time.
1. INTRODUCTION
The paper frames weakly labeled sound event detection as a multiple instance learning problem in which pooling aggregates frame-level predictions into recording-level labels. It compares pooling functions theoretically and experimentally, emphasizing localization, and introduces TALNet for joint tagging and localization.
- SED identifies sound-event types and their onset and offset times, while this paper emphasizes localization alongside audio tagging.
- Weak labeling records only event types present in each recording, omitting temporal information needed for direct localization.Strong labeling specifies event type, onset, and offset for each occurrence; weakly labeled Audio Set enables larger-scale training.
- In MIL, recordings are bags and frames are instances; a pooling function aggregates frame-level event probabilities into recording-level probabilities used for training.Each sound-event type is treated as an independent binary classification problem.
- The paper derives gradients for five pooling functions and compares them with max pooling on the DCASE 2017 challenge and Audio Set, focusing on tagging and localization.
- The experiments produce TALNet, a CRNN designed to achieve strong audio tagging and localization performance simultaneously.TALNet closely matches current state-of-the-art audio tagging performance on Audio Set and achieves competitive DCASE 2017 localization without finetuning.
2. THEORETICAL COMPARISON OF THE FIVE POOLING FUNCTIONS
The five pooling functions differ in how they distribute learning signals across frame-level predictions. Theoretical gradient analysis suggests that linear softmax best balances recording-level aggregation with localized frame-level detections.
- Max pooling: Max pooling sends a non-zero gradient only to the highest-probability frame, potentially missing other event occurrences.Its single-frame error signal can cause many frame-level false negatives when an event occurs multiple times.
- Softmax pooling: Softmax pooling gives larger frame probabilities larger weights while still allowing lower-probability frames to receive error signals.Linear softmax uses yi as the weight, whereas exponential softmax uses exp(yi).
- Average pooling: Average pooling distributes the gradient evenly across frames, which can create many false-positive frames in positive recordings.Uniformly boosting every frame is inappropriate when only some frames contain the event.
- Linear softmax pooling: Linear softmax drives larger probabilities upward and smaller probabilities downward for positive recordings, producing frame probabilities near 0 and 1.For negative recordings, its dynamics push frame-level probabilities toward zero over sufficient iterations.
- Exponential softmax pooling: Exponential softmax retains a risk of false-positive frames because its gradient is always positive, although smaller probabilities receive smaller gradients.Its behavior resembles average pooling but reduces the concern for lower-probability frames.
- Attention pooling: Attention pooling learns frame weights separately, but large frame probabilities paired with small weights can produce localization false positives.The recording-level prediction may remain small while high-probability frames become false positives.
3. EXPERIMENTAL COMPARISON OF THE FIVE POOLING FUNCTIONS
The experiments compare five pooling functions on DCASE 2017 and Audio Set, evaluating both audio tagging and localization. Linear softmax offers the strongest overall balance, while TALNet combines competitive performance on both tasks.
- DCASE 2017 Challenge: The DCASE 2017 experiment evaluates five pooling functions on 17 vehicle and warning sound types using audio-tagging and localization metrics.The systems are tested on weakly labeled training data and strongly labeled test data.
- DCASE 2017 Challenge: All four alternatives outperform max pooling on F1 for audio tagging and localization, but only linear softmax slightly improves localization error rate over max pooling.Average, exponential softmax, and attention produce localization error rates above 100%.
- DCASE 2017 Challenge: Linear softmax maintains a good localization balance, whereas max pooling produces too many false negatives and the other alternatives produce too many false positives.The latter systems consequently incur high insertion rates and high error rates.
- DCASE 2017 Challenge: Average, exponential softmax, and attention generate frame-level bus false positives despite correctly producing negative recording-level predictions.These false positives are described as common throughout the data.
- TALNet: Joint Tagging and Localization on Audio Set: TALNet is evaluated on Audio Set for tagging and directly on DCASE 2017 for localization, where it is the only system reported to achieve low error rate and high F1 together.The paper contrasts this joint performance with prior systems that localize poorly or not at all.
- TALNet: Joint Tagging and Localization on Audio Set: On the full Audio Set, TALNet closely matches Yu et al. on audio-tagging performance and outperforms the other reported systems by a large margin.Audio Set contains over 2 million training recordings and 20,371 evaluation recordings.
4. CONCLUSION AND DISCUSSION
The study identifies linear softmax as the best of five pooling functions, while noting that adaptive alternatives and constrained attention remain promising directions.
- Linear softmax was the best-performing pooling function among the five compared.The authors attribute this to unobstructed gradient flow, balanced localization errors, and consistency between recording- and frame-level predictions.
- Linear softmax pooling supports TALNet’s strong performance for both audio tagging and localization.The network uses linear softmax pooling and is reported to perform strongly on both tasks simultaneously.
- Adaptive pooling could generalize exponential softmax, while a y_i^β exp(αy_i) weighting scheme could subsume both softmax variants.The proposed family is presented as a possible direction beyond linear softmax.
- Attention pooling remains attractive for learning weights dynamically, but its frame-level false positives are linked to attention on low-probability frames.The authors suggest monotonicity constraints so larger frame probabilities receive larger weights.
- Additional experimental details and the implementation resources are available in the first author’s PhD thesis and public repository.The referenced materials cover data balancing, threshold tuning, training hyperparameters, code, and acoustic features.