PaperlayerView exact revision on arXiv

Source-linked AI summary

A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection

Archontis Politis, Sharath Adavanne, Tuomas Virtanen

arXiv:2006.01919v2eess.AScs.SD

TL;DR

SELD systems must classify sound events, detect when they occur, and estimate their spatial directions or locations, but training and testing require realistic, diverse scenes. This paper constructs a reverberant dataset with moving sources and supplies an updated baseline and joint evaluation metrics. The baseline successfully tracks dynamic sources in a test example, while its performance varies across formats, polyphony, and unseen rooms.

  • Problem

    SELD combines sound-class prediction and temporal activity detection with spatial direction or location estimates, requiring diverse sound scenes for training and testing.

    Source: §1 · p. 1

  • Method

    The authors synthesize static and moving sound events using measured room-response trajectories, add room ambient noise, and provide an updated SELDnet baseline and joint evaluation metrics.

    Source: §2.2 · p. 2, §1 · p. 2, §6 · p. 6

  • Results

    The baseline successfully detects, localizes, and tracks dynamic sources in a test example, but performs better on FOA than MIC data and on single-event than overlapping scenes.

    Source: §5 · p. 5

  • Takeaways & Limitations

    The dataset pairs moving, reverberant scenes with joint detection-and-localization metrics for evaluating SELD systems.

    Source: §4.2 · p. 5, §6 · p. 6

    The baseline does not generalize equally well across the two unseen evaluation rooms.

    Source: §5 · p. 5

Abstract

from arXiv · show

This report presents the dataset and the evaluation setup of the Sound Event Localization & Detection (SELD) task for the DCASE 2020 Challenge. The SELD task refers to the problem of trying to simultaneously classify a known set of sound event classes, detect their temporal activations, and estimate their spatial directions or locations while they are active. To train and test SELD systems, datasets of diverse sound events occurring under realistic acoustic conditions are needed. Compared to the previous challenge, a significantly more complex dataset was created for DCASE 2020. The two key differences are a more diverse range of acoustical conditions, and dynamic conditions, i.e. moving sources. The spatial sound scenes are created using real room impulse responses captured in a continuous manner with a slowly moving excitation source. Both static and moving sound events are synthesized from them. Ambient noise recorded on location is added to complete the generation of scene recordings. A baseline SELD method accompanies the dataset, based on a convolutional recurrent neural network, to provide benchmark scores for the task. The baseline is an updated version of the one used in the previous challenge, with input features and training modifications to improve its performance.

1. INTRODUCTION

SELD combines sound-class recognition and temporal detection with spatial direction or location estimates to describe acoustic scenes. The DCASE 2020 dataset and evaluation setup extend the prior challenge with more varied rooms and moving sources, alongside an updated baseline and joint-performance metrics.

1. INTRODUCTION

  • SELD jointly predicts sound types, temporal activations, and spatial trajectories, extending temporal sound event detection with direction or location.The resulting scene description has applications including acoustic monitoring, robot navigation, and human-machine interaction.Source: §1 · p. 1
  • Before DCASE 2019, most SELD approaches treated detection and localization independently, with only joining localization outputs to event classifiers.Earlier methods included TDoA, steered-response power, acoustic intensity, and several classifier types.Source: §1 · p. 1
  • The DCASE 2020 release is the TAU-NIGENS Spatial Sound Events dataset, intended for the next SELD challenge iteration.It retains realistic reverberation, ambient noise, varied rooms, broad source positions, and two recording-array formats while addressing limitations of the previous dataset.Source: §1 · p. 1
  • The new dataset adds more rooms, diverse acoustic conditions, non-grid source positions, and moving sources for about half of active events.These changes make the task significantly more challenging and demand performance closer to real life.Source: §1 · p. 2
  • The updated SELDnet baseline incorporates common DCASE 2019 improvements, while joint SELD metrics assess whether systems detect and localize the correct events together.The joint metrics are intended to distinguish systems that perform well on both tasks from those that perform well on either independently.Source: §1 · p. 2

2. REVERBERANT DYNAMIC DATASET

The dataset combines sound events with continuously measured room responses to create reverberant scenes containing static and moving sources. Its reference directions, synthesis procedure, and two four-channel formats support evaluation across diverse acoustic conditions.

2.1. Sound events

  • The 714 event recordings span 14 classes, including alarms, animals, fire, footsteps, speech, screams, phones, and piano.Samples come from the NIGENS General Sound Events Database, which provides more examples and classes than the previous challenge database.Source: §2.1 · p. 2

2.2. Recording of multichannel RIRs

  • Ten diverse rooms were added to the five earlier rooms, and moving-source MLS recordings along varied trajectories captured a broader range of source positions and reverberation.Circular and linear paths varied in distance, elevation, and direct-to-reverberant ratio.Source: §2.2 · p. 2

2.3. Reference RIRs and positional labels

  • Reference directions were estimated acoustically from extracted RIRs using early-path windows and broadband MUSIC, avoiding reliance on planned trajectory geometry.Video tracking was stable but camera-to-array rotational misalignment could produce large direction errors; final references were selected near the geometric trajectory at approximately one-degree intervals.Source: §2.3 · pp. 2–3

2.4. Dataset Synthesis

  • Mixtures combine static or moving events at randomized times and polyphony, with moving events rendered by time-varying RIR convolution and room ambient noise added at varied SNRs.Events could move at slow, medium, or fast rates; events lasting up to two seconds were assigned static directions.Source: §2.4 · p. 3

2.5. Dataset Formats

  • The dataset provides four-channel MIC and FOA formats: MIC encodes direction with time and level differences, while space-coincident FOA provides level differences only.The different spatial encodings suit different spatial features, and participants could use either format or both.Source: §2.5 · p. 3

3. BASELINE METHOD

The baseline updates SELDnet with array-dependent acoustic features and masked DOA training while retaining a single model that jointly estimates SED and DOA without output post-processing.

3. BASELINE METHOD

  • Array-dependent features and an SED-masked DOA loss update SELDnet while a single model jointly estimates SED and DOA, with no output post-processing.The DOA branch computes mean squared error only for active sound events.Source: §3 · p. 4
  • FOA uses log-mel energies plus acoustic-intensity vectors, whereas MIC uses log-mel energies plus GCC-PHAT features.The intensity vector expresses net acoustic energy flux; GCC-PHAT is generalized cross-correlation with phase transform.Source: §3 · p. 4
  • FOA inputs have 7×T×64 dimensions, while MIC inputs have 10×T×64 dimensions, with T denoting input time frames.FOA combines four log-mel channels and three intensity channels; MIC combines four log-mel channels and six GCC-PHAT channels.Source: §3 · p. 4
  • Three convolutional layers and two GRU layers feed separate SED and Cartesian-DOA heads at 100 ms resolution.Training uses binary cross-entropy for SED and masked MSE for DOA, with DOA loss weighted 1000 times more than SED loss.Source: §3 · p. 4

4. EVALUATION

The evaluation uses validation-selected models on an unseen test split and assesses SELD with four joint detection-and-localization metrics, alongside the earlier separate SED and DOA measures.

4.1. Evaluation Setup

  • The first development split is unseen test data, the second is validation, and the remaining splits train the model; the best validation model is scored on test data.Participants use the unseen test split for comparable scores, but may choose training procedures for the evaluation dataset.Source: §4.1 · pp. 4–5

4.2. Metrics

  • The 2019 setup measured SED with one-second F-score and error rate, and DOA estimation with frame-wise angular error and frame recall.Frame recall measures the percentage of frames with the correct estimated number of DOAs.Source: §4.2 · p. 5
  • The 2020 metrics assess joint performance through location-aware ER20° and F20° with a 20° threshold, plus class-aware LECD and LRCD without a distance threshold.Ideal values are ER20° = 0, F20° = 100%, LECD = 0°, and LRCD = 100%.Source: §4.2 · p. 5
  • All four 2020 metrics are reported, with submissions ranked separately per metric and final positions determined by the cumulative minimum of ranks.Source: §4.2 · p. 5

5. RESULTS

The baseline demonstrates joint tracking of moving sources in an example recording, while its reported scores vary across data formats, polyphony levels, and unseen rooms.

5. RESULTS

  • In the illustrated unseen-test recording, SELDnet successfully jointly detects, localizes, and tracks dynamic sources.Figure 2 shows the input log-mel spectrum and model output, with each event class represented by a unique color.Source: §5 · p. 5
  • On FOA test data, LRCD was 60.7% with LECD 22.8°, while F20° was 37.4% and ER20° was 0.72.The metrics evaluate class-aware localization and location-aware detection, respectively.Source: §5 · p. 5
  • SELDnet performed better on FOA than MIC, on single-event than overlapping recordings, and in one unseen room than the other.The authors suggest FOA and MIC formats contain non-identical spectral and spatial information, and report unequal generalization across the two evaluation rooms.Source: §5 · p. 5

6. CONCLUSION

The DCASE 2020 SELD task uses a new dataset of dynamic sound scenes in reverberant environments, contrasting with the static scenes of DCASE 2019. The paper describes its synthesis, provides a fixed evaluation setup, and reports baseline scores.

  • The dataset shifts from DCASE 2019's static scenes to dynamic sound scenes in reverberant environments for the DCASE 2020 SELD task.Source: §6 · p. 5
  • Moving sound events are synthesized from impulse-response trajectories measured in 13 indoor environments, with natural ambience added at different signal-to-noise ratios.Random trajectory segments are convolved with isolated NIGENS events, which are placed at random times to simulate different polyphony.Source: §6 · p. 6
  • The dataset provides four-channel first-order Ambisonics and tetrahedral-microphone-array formats, a fixed evaluation setup, and baseline scores from an improved SELDnet.Source: §6 · p. 6, Table 2 · p. 6
Loading 2006.01919v2…