Source-linked AI summary

EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation

Julius Richter, Yi-Chiao Wu, Steven Krenn, Simon Welker, Bunlong Lay, Shinji Watanabe, Alexander Richard, Timo Gerkmann

arXiv:2406.06185v2eess.AScs.LGcs.SD

TL;DR

Existing speech datasets often lack the combined recording quality, scale, and speaking-style diversity needed for varied speech research. This paper releases EARS and benchmarks enhancement and dereverberation methods on it; SGMSE+ performs best across most enhancement metrics and is preferred in listening tests.

  • Problem

    Many existing datasets lack one or more of high recording quality, sufficient scale, and coverage beyond reading or neutral speech.

    Source: §2 · p. 1
  • Method

    The paper constructs EARS as a diverse anechoic speech dataset and benchmarks predictive and generative methods for speech enhancement and dereverberation.

    Source: §1 · p. 1
  • Results

    SGMSE+ performs best across most speech-enhancement metrics and is preferred in the listening test, closely followed by Demucs.

    Source: §4.3 · pp. 3–4
  • Takeaways & Limitations

    EARS provides a diverse speech resource and paired benchmarks intended to support future speech-enhancement research on the dataset.

    Source: §5 · p. 4

Abstract

from arXiv · show

We release the EARS (Expressive Anechoic Recordings of Speech) dataset, a high-quality speech dataset comprising 107 speakers from diverse backgrounds, totaling in 100 hours of clean, anechoic speech data. The dataset covers a large range of different speaking styles, including emotional speech, different reading styles, non-verbal sounds, and conversational freeform speech. We benchmark various methods for speech enhancement and dereverberation on the dataset and evaluate their performance through a set of instrumental metrics. In addition, we conduct a listening test with 20 participants for the speech enhancement task, where a generative method is preferred. We introduce a blind test set that allows for automatic online evaluation of uploaded data. Dataset download links and automatic evaluation server can be found online.

1. Introduction

Existing speech datasets often lack sufficient scale, recording quality, or speaker and speaking-style variety. EARS addresses these gaps with diverse anechoic speech and benchmarks for enhancement and dereverberation.

1. Introduction

  • Existing datasets such as LibriSpeech and VCTK are described as too small, low quality, or insufficiently varied in speakers and speaking styles.The paper presents these shortcomings as a barrier to open, reproducible progress in speech research.Source: §1 · p. 1
  • EARS provides 100 h of anechoic speech from over 100 English speakers, spanning reading, emotional and conversational speech, and non-verbal sounds.The collection also spans the full human speech range, from whispering to yelling and screaming.Source: §1 · p. 1, Figure 1 · p. 1
  • The paper benchmarks predictive and generative methods for speech enhancement and dereverberation on EARS.The benchmarks are intended to support comparison of model strengths and limitations.Source: §1 · p. 1

2. EARS dataset

EARS is designed to address the shortage of speech datasets combining high recording quality, scale, and broad speech-style coverage. Its anechoic recordings include diverse speakers and scripted, emotional, conversational, and non-speech material.

2. EARS dataset

  • EARS responds to a gap in datasets that combine clean 48 kHz audio, sufficient scale, and speech beyond reading or neutral styles.The paper identifies this combination as needed for research spanning speech synthesis, voice and style conversion, and speech enhancement.Source: §2 · p. 1
  • The dataset contains 100 h of clean speech from 107 demographically diverse speakers, recorded in an anechoic chamber at 48 kHz.Two microphones capture subtle speech and high-energy speech; the lower-sensitivity recording replaces clipped high-sensitivity audio in about 5% of cases.Source: §2 · p. 1
  • Table 1 contrasts speech datasets by recording quality, scale, and diversity; EARS includes freeform and emotional speech alongside seven reading styles.The table also notes reading transcriptions and speaker metadata for EARS.Source: Table 1 · p. 2
  • Speakers range from age 18 to 75 and include African American, Caucasian, Hispanic, and Asian participants; 44% are male, 53% female, and 3% non-binary.Source: §2 · p. 2
  • The recording script covers seven reading styles, 22 emotional styles, conversational freeform speech, and non-speech sounds such as laughter and coughing.A trained operator monitors recordings and requests re-recording of faulty segments.Source: §2 · p. 2

3. Benchmarks

The benchmark defines speaker-separated training, validation, and test splits, then constructs noisy and reverberant speech tasks from EARS recordings. EARS-WHAM uses real noise, while EARS-Reverb uses room impulse responses.

3. Benchmarks

  • Training uses speakers p001–p099, validation uses p100 and p101, and testing uses p102–p107, with 32,485 training, 632 validation, and 886 test files.The respective split durations are 86.8 h, 1.7 h, and 3.7 h.Source: §3 · p. 2
  • Table 3 reports baseline methods’ publication dates, parameter counts, MACs for a four-second input, and average processing time per one-second input length.Source: Table 3 · p. 2

3.1. EARS-WHAM

  • EARS-WHAM mixes EARS speech with WHAM! real noise at randomly sampled SNRs from −2.5 to 17.5 dB, and provides a blind online test set.The blind set contains 743 files from six speakers outside EARS, with noisy audio but no published clean ground truth.Source: §3.1 · p. 2

3.2. EARS-Reverb

  • EARS-Reverb creates reverberant speech by convolving clean speech with fullband room impulse responses and restricting RT60 to no more than 2 s.The RIRs come from multiple public datasets; the direct-path delay is removed and reverberant loudness is normalized to the clean speech using LKFS.Source: §3.2 · pp. 2–3

4. Baselines and Evaluation

This section specifies the baseline methods and evaluation measures, then reports results for speech enhancement, input SNR, speaking styles and emotions, and dereverberation.

4.1. Baselines

  • Baselines comprised predictive Conv-TasNet (120-sample kernels, 60-sample stride at 48 kHz) and Demucs v4, and generative CDiffuSE and SGMSE+ [9], with the latter two adapted for 48 kHz.Source: §4.1 · p. 3

4.2. Metrics

  • Evaluation combines intrusive reference-based quality, intelligibility, and distortion measures with non-intrusive quality estimators and downstream ASR word error rate.The measures include POLQA, PESQ, ESTOI, SI-SDR, SIGMOS, DNSMOS, and WER.Source: §4.2 · p. 3

4.3. Evaluation

  • SGMSE+ [9] performs best across most speech-enhancement metrics, while Demucs is strong but slightly behind; listeners also prefer SGMSE+, closely followed by Demucs.The listening test involved 20 participants.Source: §4.3 · pp. 3–4, Figure 2 · p. 4
  • Performance tends to improve at higher input SNR, with smaller standard deviations than on the full test set.Table 6 reports POLQA, SI-SDR, ESTOI, and WER by input-SNR range; 0 dB denotes [−2.5, 2.5] dB, with successive ranges in 5 dB increments.Source: §4.3 · p. 4, Table 6 · p. 4
  • Whispered speech performs worse, while all considered approaches trained on EARS-WHAM generalize well to emotional speech.Tables 8 and 9 report POLQA by speaking style and emotion, respectively.Source: §4.3 · p. 4, Table 8 · p. 4, Table 9 · p. 4
  • The dereverberation evaluation uses diffusion-based SGMSE+ [9] on the EARS-Reverb test set.Table 7 reports intrusive and non-intrusive metrics and WER, with means and standard deviations.Source: §4.3 · p. 4, Table 7 · p. 4

5. Conclusion

EARS combines diverse, full-range speech recordings with enhancement and dereverberation benchmarks comparing predictive and generative baselines. The authors present the benchmark as a standard for future speech-enhancement work on EARS.

  • EARS has high speaker and speaking-style diversity spanning the full range of human speech.Source: §5 · p. 4
  • The paper benchmarks speech enhancement and dereverberation on EARS by comparing predictive and generative baselines.Source: §5 · p. 4
  • The authors hope EARS will serve as a community resource for tackling new frontiers in speech processing.Source: §5 · p. 4
Loading 2406.06185v2…