Source-linked AI summary
Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training
Kazuki Shimada, Yuichiro Koyama, Shusuke Takahashi, Naoya Takahashi, Emiru Tsunoo, Yuki Mitsufuji
TL;DR
Class-wise SELD outputs assign only one location to each event class, leaving the detection of the same class from multiple locations challenging. The paper extends ACCDOA to multi-ACCDOA and uses class-wise ADPIT so tracks can learn with the single-ACCDOA target despite duplicated outputs. The method detects same-class overlapping events without degradation in other cases and performs comparably to state-of-the-art SELD methods with fewer parameters.
Problem
Class-wise SELD outputs assign only one location to each event class, leaving the detection of the same class from multiple locations challenging.
Source: §1 · p. 1Method
Results
The method detects same-class overlapping events without degradation in other cases and performs comparably to state-of-the-art SELD methods with fewer parameters.
Source: §1 · p. 2Takeaways & Limitations
Multi-ACCDOA with class-wise ADPIT successfully localizes and detects same-class overlaps while maintaining performance in other cases.
Source: §4.4 · p. 4
Abstract
from arXiv · showhide
Sound event localization and detection (SELD) involves identifying the direction-of-arrival (DOA) and the event class. The SELD methods with a class-wise output format make the model predict activities of all sound event classes and corresponding locations. The class-wise methods can output activity-coupled Cartesian DOA (ACCDOA) vectors, which enable us to solve a SELD task with a single target using a single network. However, there is still a challenge in detecting the same event class from multiple locations. To overcome this problem while maintaining the advantages of the class-wise format, we extended ACCDOA to a multi one and proposed auxiliary duplicating permutation invariant training (ADPIT). The multi- ACCDOA format (a class- and track-wise output format) enables the model to solve the cases with overlaps from the same class. The class-wise ADPIT scheme enables each track of the multi-ACCDOA format to learn with the same target as the single-ACCDOA format. In evaluations with the DCASE 2021 Task 3 dataset, the model trained with the multi-ACCDOA format and with the class-wise ADPIT detects overlapping events from the same class while maintaining its performance in the other cases. Also, the proposed method performed comparably to state-of-the-art SELD methods with fewer parameters.
1. INTRODUCTION
SELD identifies sound-event classes and their directions, but class-wise outputs assign only one location per class, limiting detection of same-class overlaps. The paper extends ACCDOA to multi-ACCDOA and introduces class-wise ADPIT; evaluations report same-class overlap detection without degradation in other cases and performance comparable to state-of-the-art methods with fewer parameters.
1. INTRODUCTION
- SELD identifies sound-event types and directions of arrival and is used in applications including surveillance, biodiversity monitoring, and context-aware devices.Source: §1 · p. 1
- Class-wise outputs assign one location per event class, whereas track-wise outputs can represent the same class at multiple locations but cannot use ACCDOA's single-target, single-network advantage.Source: §1 · p. 1
- Multi-ACCDOA adds tracks to the class-wise ACCDOA format, while class-wise ADPIT uses duplicated vectors to address track permutation and preserve learning with the single-format target.Source: §1 · p. 1, Figure 1 · p. 1
- The method detected overlapping events from the same class without degradation in other cases and performed comparably to state-of-the-art methods with fewer parameters.Source: §1 · p. 2
2. RELATED WORK
Related work describes class-wise ACCDOA as a single-network SELD approach, track-wise EINV2 with permutation invariant training, and auxiliary targets in A2PIT for source separation.
2. RELATED WORK
- ACCDOA assigns activity to a Cartesian DOA vector’s length, representing active classes with unit vectors and inactive classes with zero vectors; single-ACCDOA outperforms the two-branch format on joint SELD metrics with simpler architecture.Source: §2 · p. 2
- EINV2 assigns events to separate tracks and uses permutation invariant training to handle assignments; frame-level PIT outperformed chunk-level PIT in its evaluation.Source: §2 · p. 2
- A2PIT replaces auxiliary low-energy random-noise targets with the mixture signal when source tracks outnumber targets, inspiring the paper's use of alternative auxiliary targets for SELD.Source: §2 · p. 2
3. PROPOSED METHOD
The method extends single-ACCDOA with a track dimension and uses class-wise ADPIT to train tracks for overlapping same-class events. At inference, similar active outputs are unified.
3.1. Multi-ACCDOA format
- Multi-ACCDOA extends single-ACCDOA with a track dimension, while each track predicts activities and locations for all target classes.The method uses a CRNN to estimate the multi-ACCDOA vectors.Source: §3.1 · p. 2
3.2. Class-wise auxiliary duplicated PIT
- Class-wise PIT assigns one active event per class to one track and assigns zero-vector auxiliary targets to inactive tracks, which may interfere with optimization.Class-wise ADPIT instead uses duplicated original targets for auxiliary tracks.Source: §3.2 · p. 3
- Class-wise ADPIT duplicates original targets for auxiliary tracks, so each track learns from an original or duplicated target rather than a zero vector.The PIT framework selects the best permutation for the original and duplicated targets.Source: §3.2 · p. 3
3.3. Unification of duplicated outputs during inference
- During inference, thresholding identifies active outputs, then outputs of the same class with angle differences below a threshold are averaged into one output.Source: §3.3 · p. 3
4. EXPERIMENTAL EVALUATION
The evaluation tests whether multi-ACCDOA with class-wise ADPIT handles same-class overlaps while preserving performance in other cases. On DCASE 2021 Task 3, it performs comparably to state-of-the-art SELD methods with fewer parameters.
4.1. Task setups
- The evaluation used the TAU Spatial Sound Events 2021 Ambisonic development set, with 600 one-minute recordings and up to three overlapping events.The set included 12 event classes, recordings from 15 indoor locations, and 400 training, 100 validation, and 100 testing scenes.Source: §4.1 · p. 3, §4 · p. 3
4.2. Hyper-parameters
- Evaluation used five metrics, including localization and location-dependent detection measures, plus the aggregated ESELD score.Inputs were segmented into 1.27-second features, with 0.2-second inference shifts; training used Adam, learning-rate scheduling, and stochastic weight averaging.Source: §4.1 · p. 3, §4.2 · p. 3
4.3. Experimental settings
- The study compared single- and multi-ACCDOA, three PIT variants, and track-wise EINV2, using a CRNN and DCASE 2021 challenge configurations.The multi-format track count was N = 3; comparisons also included CNN-MHSA and RD3Net, and excluded ensembles for the state-of-the-art comparison.Source: §4.3 · p. 4
4.4. Experimental results
- Class-wise ADPIT multi-ACCDOA exceeded single-ACCDOA by 7.2 LRCD points for same-class overlaps while performing comparably in other cases.The comparison found threshold choice slightly affected performance, especially LRCD.Source: §4.4 · p. 4, Table 1 · p. 4
- Multi-ACCDOA performed better than the track-wise format in ER20◦, F20◦, and ESELD, with considerably fewer parameters.The multi-ACCDOA model used CNN-MHSA, while the track-wise comparison used the original EINV2 architecture.Source: §4.4 · p. 4, Table 2 · p. 4
- The RD3Net using multi-ACCDOA trained with class-wise ADPIT performed best in ER20◦, F20◦, and ESELD on the DCASE 2021 Task 3 development set.The comparison excluded model ensembles and used a 30-degree unification threshold.Source: §4.4 · p. 4, Table 3 · p. 4
5. CONCLUSION
The paper extends ACCDOA to multi-ACCDOA and proposes class-wise ADPIT for same-class overlaps. On DCASE 2021 Task 3, the method detects those overlaps while maintaining performance in other situations and performs comparably to state-of-the-art methods with fewer parameters.
5. CONCLUSION
- Multi-ACCDOA with class-wise ADPIT detects overlapping same-class events while maintaining performance in other situations.ADPIT trains each track with original and duplicated targets, enabling each track to learn with the same target as single-ACCDOA; duplicated outputs are unified by similarity during inference.Source: §5 · p. 4