Source-linked AI summary
MUSAN: A Music, Speech, and Noise Corpus
David Snyder, Guoguo Chen, Daniel Povey
TL;DR
The paper addresses the need for data to train music/speech discrimination and VAD systems. It introduces a corpus from public-domain and Creative Commons sources and demonstrates simple GMM-based systems on Broadcast news and speaker recognition, with improved speaker-recognition performance from GMM-based VAD.
Problem
Practical speech technologies require audio classification and suitable data for music/speech discrimination and voice activity detection.
Method
The paper constructs a corpus of speech, music, and noise and trains simple GMM systems using MFCC-based features for discrimination and VAD experiments.
Results
GMM-based VAD improves speaker recognition across all tested speech amounts, with greater benefit when less speech is available, while MUSAN and GTZAN achieve similar discrimination performance.
Takeaways & Limitations
The corpus supports training simple classifiers for music/speech discrimination and frame-level VAD.
Abstract
from arXiv · showhide
This report introduces a new corpus of music, speech, and noise. This dataset is suitable for training models for voice activity detection (VAD) and music/speech discrimination. Our corpus is released under a flexible Creative Commons license. The dataset consists of music from several genres, speech from twelve languages, and a wide assortment of technical and non-technical noises. We demonstrate use of this corpus for music/speech discrimination on Broadcast news and VAD for speaker identification.
1 Introduction
Audio classification into speech, music, and nonspeech supports practical tasks including VAD, speech technologies, and removal of on-hold music. The paper uses simple GMM-based systems while focusing on providing data rather than complex modeling.
- Audio classification into speech, music, and nonspeech supports VAD, speech recognition, speaker diarization, speaker verification, and on-hold music removal.
- GMMs are popular simple models for VAD and music/speech discrimination, and prior work found an unsupervised GMM VAD effective on NIST SRE 2008.
- The paper uses GMM-based systems to demonstrate the corpus while deliberately avoiding complex models and sophisticated features.
2 The corpus
The corpus combines annotated speech, music, and noise from public-domain and Creative Commons sources. Its contents span multiple languages, musical genres, and technical and environmental sounds.
- Corpus directories commonly include ANNOTATIONS files with metadata such as music vocals and genres, while READMEs describe annotation details.
- Speech: The speech portion contains about 60 hours, including Librivox read speech in twelve languages and English-language US government recordings.Librivox contributes 20 hours and 21 minutes; government hearings, committees, and debates contribute 40 hours and 1 minute.
- Music: The 42-hour-31-minute music portion covers Western art and popular genres, with annotations for genre, artist, vocals, and sometimes composer.All music files were released under some form of Creative Commons license, with care taken to support commercial use.
- Noise: The noise portion contains 929 files totaling about 6 hours, spanning technical noises and ambient sounds without intelligible speech.
3 Experiments
The experiments train simple GMM systems for music/speech discrimination and VAD, evaluating them on Broadcast news and speaker recognition. MUSAN performs similarly to GTZAN for discrimination, while GMM-based VAD improves speaker verification, especially with less speech.
- Experiments: The experiments train simple systems for music/speech discrimination and VAD using the Kaldi ASR toolkit.
- Music/speech discrimination: For music/speech discrimination, two GMMs use normalized MFCCs with first- through fourth-order deltas and are compared with an identical GTZAN-trained system.
- Music/speech discrimination: Broadcast news evaluation separates speech and music segments, excludes overlaps, classifies whole segments by majority frame decisions, and reports EER.
- Music/speech discrimination results: GMM size has little overall effect, and MUSAN- and GTZAN-trained models achieve similar performance across tested component counts.Table 1 evaluates EER(%) with K ∈ {4, 8, 16, 32, 64, 128} components.
- Speaker recognition: The speaker recognition system is i-vector-based, using a GMM UBM, PLDA backend, MFCC features, normalization, deltas, acceleration, and VAD-based removal of unvoiced frames.
- Speaker recognition: The modified NIST SRE 2010 evaluation restricts test-time speech to the first n seconds to expose VAD effects in rapid recognition settings.
- GMM and Energy VAD: The VAD pipeline applies energy VAD first, refines speech frames with three eight-component full-covariance GMMs, and maps music or noise to nonspeech.The GMMs are trained on speech, music without vocals, and Librivox speech using the discrimination features.
- VAD Results: Adding GMM-based VAD improves speaker recognition for every tested speech amount, with greater benefit when less speech is available.
5 Conclusion
The paper introduces a corpus for training music/speech discrimination and VAD systems, collected from US Public Domain and Creative Commons sources. Experiments show that it supports simple classifiers for both tasks.
- The corpus provides raw audio for training music/speech discrimination and voice activity detection systems.It was collected entirely from US Public Domain and Creative Commons sources.
- Experiments demonstrate that the dataset can train simple classifiers for music/speech discrimination and frame-level VAD.