Source-linked AI summary
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, Yonghui Wu
TL;DR
LibriSpeech contains properties that make it less suitable for TTS, including low sampling rate and lost prosodic text and sentence-boundary information. The paper introduces LibriTTS by processing LibriSpeech audio and text into a filtered, sentence-aligned corpus with 24 kHz audio and contextual text access. LibriTTS contains 585 hours from 2,456 speakers, and models trained on it achieved the best subjective scores across evaluated speakers, while speaker-duration imbalance and lower male-speaker scores remain limitations.
Problem
LibriSpeech has properties that limit its suitability for TTS, including 16 kHz audio, silence-based segmentation, removed punctuation and capitalization, and discarded sentence context.
Method
The paper automatically aligns audiobook audio and text, segments recordings at sentence boundaries, normalizes text, preserves original and normalized forms, and filters noisy or mismatched utterances.
Results
LibriTTS contains 585 hours of 24 kHz speech from 2,456 speakers, and LibriTTS (24kHz) achieved the best subjective scores with all speakers.
Takeaways & Limitations
LibriTTS is released freely for public use as a TTS-specific corpus, and the authors report that Tacotron models trained on it produced naturally sounding speech.
Takeaways & Limitations
Strict filtering produces per-speaker duration imbalance, while male-speaker subjective scores are lower and require further experiments to understand.
Abstract
from arXiv · showhide
This paper introduces a new speech corpus called "LibriTTS" designed for text-to-speech use. It is derived from the original audio and text materials of the LibriSpeech corpus, which has been used for training and evaluating automatic speech recognition systems. The new corpus inherits desired properties of the LibriSpeech corpus while addressing a number of issues which make LibriSpeech less than ideal for text-to-speech work. The released corpus consists of 585 hours of speech data at 24kHz sampling rate from 2,456 speakers and the corresponding texts. Experimental results show that neural end-to-end TTS models trained from the LibriTTS corpus achieved above 4.0 in mean opinion scores in naturalness in five out of six evaluation speakers. The corpus is freely available for download from http://www.openslr.org/60/.
1. Introduction
LibriSpeech offers broad, openly licensed speech resources but contains properties that limit its suitability for high-quality TTS. LibriTTS addresses these issues through higher sampling rate, sentence-level segmentation, preserved text features, contextual access, and noise filtering.
- LibriSpeech contains 982 hours of speech from 2,484 speakers and is released under a non-restrictive license.
- 16 kHz audio is adequate for ASR but considered too low for high-quality TTS compared with modern production sampling rates.
- Silence-based segmentation, removed punctuation and capitalization, and discarded sentence positions limit access to sentence- and inter-sentence prosody.
- LibriTTS uses 24 kHz audio, sentence-boundary segmentation, original and normalized texts, neighboring-sentence access, and SNR-based noise filtering.
2. Related work
Existing publicly available English corpora support different TTS settings, but their sizes, speaker coverage, recording conditions, licensing, or scale constrain particular uses.
- Appropriate publicly available corpora facilitate exploration of new TTS tasks and research ideas.
- CMU ARCTIC is too small for training neural end-to-end TTS models.
- VCTK provides studio-quality multispeaker speech, while BC2011 offers relatively large single-speaker reading data under a non-commercial license.
3. The data processing pipeline
The LibriTTS pipeline extracts chapter text, normalizes and aligns it to long-form audio at sentence level, then filters problematic utterances before producing paired original and normalized texts.
- The pipeline aligns long-form recordings with corresponding texts and splits them into sentence-level segments.
- Book-level text is split into paragraphs and sentences, then non-standard words and semiotic classes are normalized with a WFST-based text normalizer.
- Chapter-level text is extracted from book-level text by matching ASR transcriptions to the book text.
- Audio is aligned to normalized sentences using a miniature trigram language model, a bidirectional LSTM acoustic model, edit-distance matching, and decoded sentence timings.
- Post-processing removes suspicious long sentences, mismatched utterances, long silences, polarity irregularities, and low-SNR audio.
- The final corpus contains paired original and normalized audio-text data after post-processing.
4. Statistics
Strict alignment and SNR filtering reduce LibriTTS yield but produce sentence-length distributions closer to the original LibriVox material. The filtering also creates greater per-speaker duration imbalance.
- About 60% of LibriSpeech’s audio yield remains because LibriTTS applies stricter alignment requirements and SNR-based filtering.
- About 25% of sentences are filtered from the clean subset using the 20 dB SNR threshold.
- LibriTTS sentence-length distributions resemble original LibriVox materials more closely than LibriSpeech distributions.
- Filtering heuristics give LibriTTS a shorter sentence-length tail than the original LibriVox materials.
- LibriTTS has a lower median per-speaker duration of about 900 seconds, wider variance, and greater speaker-duration diversity than LibriSpeech.
5. Experiments
The experiments evaluate GMVAE-Tacotron models trained on LibriSpeech and LibriTTS using subjective naturalness ratings. LibriTTS at 24kHz achieved the best scores, although performance varied by speaker and remained below natural speech.
- Experimental conditions: GMVAE-Tacotron models trained on LibriSpeech and LibriTTS served as TTS experimental baselines.The models used latent attribute representations and speaker-identity conditioning, then synthesized waveforms with WaveRNN.
- Experimental conditions: Six readers—three female and three male—were evaluated using 620 randomly selected test sentences.The evaluation set was included in the corpus release.
- Experimental conditions: Mean opinion scores measured synthesized-speech naturalness on a five-point Likert scale using crowd-sourced native-speaker ratings.Each sample was rated by one listener, and 16kHz down-sampled outputs were included to assess sampling-rate effects.
- Results: LibriTTS (24kHz) achieved the best subjective scores for all speakers.The MOS gaps between LibriTTS at 16kHz and 24kHz were 0.175 for female speakers and 0.133 for male speakers.
- Results: The MOS gaps between LibriSpeech (16kHz) and LibriTTS (16kHz) were 0.15 for female speakers and -0.03 for male speakers.The authors caution that the apparent role of capitalization and punctuation is not fully confirmed because the LibriTTS corpus is smaller.
- Results: Female-speaker scores exceeded male-speaker scores by 0.48 for LibriTTS (24kHz) and 0.26 for LibriSpeech (16kHz).The authors attribute this pattern to potentially suboptimal model configurations and call for further experiments.
- Results: MOS gaps between natural and synthesized speech were 0.16 for female speakers and 0.61 for male speakers.The authors identify improving synthesized-speech naturalness as further work.
6. Conclusions
The paper introduces LibriTTS as a TTS-specific corpus derived from LibriSpeech materials and reports its scale, public availability, and baseline modeling results. It identifies several directions for extending the corpus and studying data and representation effects.
- Conclusions: LibriTTS contains 585 hours of 24kHz speech from 2,456 speakers with corresponding texts.The corpus was created by aligning audiobooks and texts, segmenting utterances, and filtering noisy transcripts and recordings.
- Conclusions: Tacotron models trained on LibriTTS produced naturally sounding speech.The corpus is freely available online for public use.
- Future work: Future work includes studying speaker imbalance, punctuation and capitalization, training-data quantity, and synthesized-speech naturalness.The authors also plan to add more speakers and languages.
A. Character coverage
The LibriTTS training subsets provide good character coverage, including capitalization and punctuation.
- Character coverage: LibriTTS has good coverage of characters in its training subsets.The coverage includes capitalization and punctuation.
- Character coverage: Character frequencies are reported across all training subsets.The frequency analysis is presented in Table 5.
- Character coverage: Capitalization and punctuation are represented among the covered character types.These features are explicitly included in the corpus coverage analysis.
B. Subjective naturalness ratings
Figure 3 presents subjective naturalness ratings separately for female and male readers. The main comparison is between the distributions shown for the two reader groups.
- Subjective naturalness ratings: Figure 3 uses letter-value plots to display subjective naturalness ratings.The plot summarizes the rating distributions for the evaluated readers.
- Subjective naturalness ratings: Female-reader ratings appear above the male-reader ratings in Figure 3.The vertical arrangement separates the two reader groups.
- Subjective naturalness ratings: Compare the female and male reader rating distributions for differences in subjective naturalness.The figure caption identifies reader sex and plot placement but does not state the comparison outcome.