Source-linked AI summary
Hi-Fi Multi-Speaker English TTS Dataset
Evelina Bakhturina, Vitaly Lavrukhin, Boris Ginsburg, Yang Zhang
TL;DR
Public multi-speaker TTS datasets often provide limited speech per speaker or lack detailed audio-quality analysis. This paper constructs Hi-Fi TTS from LibriVox and Gutenberg materials using audio-quality screening and ASR-based validation, producing a 10-speaker corpus with at least 17 hours per speaker at 44.1 kHz. The dataset is intended to support broader, cleaner, and more expressive TTS models, although LibriVox audio remains below professional recording quality.
Problem
Public multi-speaker TTS datasets often provide limited speech per speaker or lack detailed audio-quality analysis.
Method
Hi-Fi TTS combines LibriVox audiobooks and Project Gutenberg texts, selecting speakers and samples with bandwidth, SNR, spectrogram, and ASR-based checks.
Results
Hi-Fi TTS contains 10 speakers with at least 17 hours each, sampled at 44.1 kHz, and accepts only samples with zero WER.
Takeaways & Limitations
The dataset is intended to support TTS models with broader pitch and voice variation, fewer mispronunciations, and less noise and fewer artifacts.
Takeaways & Limitations
LibriVox recordings still lag professional audio quality, with possible future processing aimed at hissing and metallic sounds.
Abstract
from arXiv · showhide
This paper introduces a new multi-speaker English dataset for training text-to-speech models. The dataset is based on LibriVox audiobooks and Project Gutenberg texts, both in the public domain. The new dataset contains about 292 hours of speech from 10 speakers with at least 17 hours per speaker sampled at 44.1 kHz. To select speech samples with high quality, we considered audio recordings with a signal bandwidth of at least 13 kHz and a signal-to-noise ratio (SNR) of at least 32 dB. The dataset is publicly released at http://www.openslr.org/109/ .
1. Introduction
Existing public English TTS datasets provide substantial data but are limited by low sampling rates, insufficient audio-quality analysis, or too little speech per speaker. Hi-Fi TTS addresses these gaps with a 10-speaker dataset designed around high-quality audio and verified text-audio alignment.
- Motivation: Existing datasets are constrained by low sampling rates, limited audio-quality analysis, or small amounts of speech per speaker.LJSpeech has about 24 hours from one speaker at 22.05 kHz, while M-AILABS has about 102 hours for three English speakers.
- Dataset goals and contribution: The dataset is built from LibriVox audiobooks and Project Gutenberg texts, with speaker and book selection guided by audio quality.The authors identify bandwidth and SNR requirements as selection goals.
- Dataset goals and contribution: Hi-Fi TTS provides 10 speakers with at least 17 hours per speaker, while targeting a minimum of 44.1 kHz sampling frequency and high-quality reference texts.The speakers include 6 female and 4 male voices, with 17.7–58.0 hours per speaker.
- Dataset goals and contribution: Reference-text quality is checked with ASR inference, and only samples with zero Word Error Rate are included.This verification is intended to ensure accurate text-audio matches.
- Expected impact: The authors expect the dataset to support models with broader pitch ranges, fewer mispronunciations, higher overall quality, and greater voice variation.These are stated expectations rather than reported experimental results.
- Ethical consideration: Use of a speaker’s voice for a TTS application was conditioned on obtaining the speaker’s written consent despite the public-domain source material.The authors encourage other developers to seek consent for synthetic voice replication.
2. The dataset construction
The dataset construction pipeline selects LibriVox readers and books using bandwidth and SNR analysis, then preprocesses, segments, aligns, and filters their text-audio pairs. Spectrogram inspection and alignment confidence checks supplement automated screening.
- Audio-quality selection: Bandwidth is estimated from the mean power spectrogram, while SNR is computed from energy-based VAD, non-speech noise power, and speech-segment power.The SNR procedure assumes stationary noise and works fairly well when SNR is not very low.
- Audio-quality selection: Readers and books are filtered using bandwidth, SNR, spectrogram analysis, and manual listening, reducing the candidate pool from 250 readers to 10.Books with low noise and high bandwidth enter the clean subset, while noisier or narrower-bandwidth books enter the other subset.
- Text preprocessing: The pipeline downloads reference texts, splits them into chapters using ASR transcripts and chapter titles, removes bracketed material, and normalizes abbreviations and non-standard words.PDF-based references are handled with OCR when needed.
- Segmentation and alignment: Long reference texts and audio files are divided into sentence-based fragments of up to roughly 20 seconds for TTS training.Long chunks are further split at semicolons, dashes, or colons, and audio is aligned with corresponding text using CTC-based segmentation.
- Segmentation and alignment: CTC-Segmentation uses forward character probabilities and a backward path recovery step to locate text segments within audio.The method permits alignment to begin after untranscribed LibriVox preambles, while cutting off 3 seconds can mitigate repeated preamble utterances.
- Alignment validation: Alignment confidence filtering removes poorly aligned utterances, but the final pipeline additionally requires zero WER from QuartzNet and Citrinet ASR inference.This addresses residual mismatches caused by changed reference-book versions or reader modifications.
3. Dataset splits and statistics
The Hi-Fi TTS corpus provides 44.1 kHz speech from 10 speakers, divided into clean and other subsets according to audio quality, with verified text-audio alignment and dev/test splits.
- 10 speakers provide at least 17 hours of 44.1 kHz speech each, with all samples verified at zero WER.The dataset is distributed as WAV audio and contains six female and four male speakers.
- The clean subset contains audiobooks with SNR of at least 40 dB, while the other set contains books with SNR of at least 32 dB.Some readers occur in both subsets because audio quality varies across books.
- Figure 3 uses violin plots to show audio-clip duration distributions for each speaker in the clean and other sets.The figure separates the clean set and other set into panels a and b.
- Dev and test splits support model comparison, alongside original and normalized text versions with letter capitalization preserved.The dataset did not overlap with M-AILABS or LJSpeech at the time of writing, although LibriTTS may share some speakers.
4. Conclusions
The paper presents a high-quality multi-speaker TTS dataset built from LibriVox and Gutenberg materials, using acoustic analysis and ASR validation for selection. It acknowledges that LibriVox audio remains below professional recording quality and suggests future sound enhancement.
- The dataset combines LibriVox audiobooks and Gutenberg texts, selecting readers with SNR and spectrogram analysis and accepting only samples with zero WER.ASR predictions validate both text-audio alignment and reference-text accuracy.
- The authors identify LibriVox audio quality as still substantially below professional recordings despite careful speaker and book selection.They suggest removing hissing and metallic sound from samples with boosted high frequencies as a possible next step.
- Normalization, segmentation, and interactive data-analysis code is publicly available through the NeMo toolkit [34].