LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus

1. Introduction

• A key barrier to high-quality TTS research is the scarcity of large, high-quality public datasets, while studies have used over 100 hours of unavailable studio-recorded data [11].Source: §1 · p. 1
• The authors propose applying speech restoration to public datasets, motivated by models that can convert reverberated lecture and historical speech to studio-recorded quality [18].Source: §1 · p. 1
• Subjective experiments showed that a TTS model trained with LibriTTS-R had greatly improved speech naturalness over one trained with LibriTTS, and naturalness comparable to ground-truth speech.Source: §1 · p. 1
• The authors created LibriTTS-R by cleaning LibriTTS with text-informed Miipher [20], using a w2v-BERT feature cleaner and WaveFit neural vocoder.Source: §1 · p. 1
• LibriTTS-R is publicly available under the same non-restrictive license as LibriTTS, with ground-truth and generated audio samples on a demo page.The corpus is available at http://www.openslr.org/141/.Source: §1 · p. 1

• LibriTTS contains 585 hours of 24 kHz speech from 2,456 speakers, with corresponding texts.Source: §2 · p. 1
• Its samples were re-derived from LibriSpeech materials to address sampling-rate and text-normalization issues, using LibriVox audio and Project Gutenberg texts.Source: §2 · p. 1
• Glow-TTS scored 3.45 MOS on LibriTTS versus 4.22 for ground-truth-mel vocoder speech, illustrating the corpus’s quality limitation.The passage also reports LJspeech scores of 4.01 for generated speech and 4.19 for ground-truth speech.Source: §2 · p. 1

3. Data processing pipeline

• To balance the usefulness of noisy TTS data with the need for high-quality resources, the authors apply speech restoration to LibriTTS.Source: §3 · p. 1

3.1. Speech restoration model overview

• Miipher [20] targets phoneme masking and deletion using w2v-BERT features and transcript-derived PnG-BERT conditioning.The model’s feature-cleaner predicts clean w2v-BERT features, which WaveFit uses to synthesize restored audio; Miipher uses w2v-BERT rather than conventional log-mel features.Source: §3.1 · p. 2, Figure 1 · p. 2

3.2. Speech restoration model training

• The authors trained Miipher using 2,680 hours of noisy and studio-quality speech pairs.The target set comprised 670 hours of studio-recorded English speech at 24 kHz; noisy samples were mixed at 5–30 dB SNR with simulated reverberation and codec artifacts. Training pre-trained the feature-cleaner for 150k steps and WaveFit for 1M, then fine-tuned WaveFit for 350k steps.Source: §3.2 · p. 2

3.3. Speech restoration pipeline

• The pipeline derives PnG-BERT features from a transcript and speaker embeddings from the original 24 kHz waveform, repeating sub-2-second samples; it then synthesizes restored 24 kHz audio with WaveFit.For samples shorter than two seconds, the speaker embedding was calculated after waveform repetition. The LibriTTS sample was downsampled for w2v-BERT features because the model was trained on 16 kHz waveforms.Source: §3.3 · p. 2

4.1.1. Experimental setups

• Ground-truth quality was evaluated on 620 randomly selected samples each from test-clean and test-other, using MOS and SxS ratings from over 100 paid native English-speaking listeners.Each stimulus was rated by one subject; MOS used a 1–5 scale in 0.5 increments and SxS a −3 to 3 scale. Listeners used headphones in a quiet room.Source: §4.1.1 · p. 2

4.1.2. Results

• LibriTTS-R ground-truth samples had better MOS in both splits, without significant differences, while SxS tests found significant sound-quality differences.Spectrograms showed LibriTTS degradation from down-sampling, noise, reverberation, and non-linear enhancement, while restoration produced high-quality samples.Source: §4.1.2 · pp. 2–3, Figure 2 · p. 3
• WER was 3.4 and 5.1 for LibriTTS versus 3.2 and 5.1 for LibriTTS-R on test-clean and test-other; same-speaker cosine similarity was 0.784/0.755 versus 0.762/0.745.The authors concluded text content was unchanged and speaker characteristics remained consistent; different-speaker similarity in LibriTTS was 0.302. WER used Pre-trained Conformer XXL, and speaker embeddings used.Source: §4.1.2 · p. 3
• Two of 640 LibriTTS-R samples were distorted by restoration failures, and uninspected training splits may also contain a small number of distorted samples.The authors had not manually checked every sample.Source: §4.1.2 · p. 3

4.2.1. Experimental setups

• TTS systems used the same architecture and hyperparameters for LibriTTS and LibriTTS-R, with NAT-FVAE acoustic models and WaveRNN vocoders trained on Train-460 or Train-960.Evaluation used 620 test-clean sentences and six speakers, with MOS and SxS listening tests; Train-960 adds train-other-500 to Train-460.Source: §4.2.1 · p. 3

4.2.2. Results

• LibriTTS-R Train-960 significantly outscored Train-460 on MOS for speakers other than ID 19, improved SxS sound quality, and reached human-speech MOS parity.For speaker ID 19, Train-460 had the highest MOS, not significantly different from Train-960. LibriTTS Train-960 sometimes reduced MOS, whereas LibriTTS-R Train-960 benefited from adding restored train-other-500 data.Source: §4.2.2 · p. 4, Table 2 · p. 4, Table 3 · p. 4

• TTS trained on LibriTTS-R improved speech naturalness over TTS trained on LibriTTS and was comparable with ground-truth speech in subjective experiments.Source: §5 · p. 4
• LibriTTS-R is a sound-quality-improved version of LibriTTS, produced by cleaning its speech samples with an SR model [20].Source: §5 · p. 4
• The corpus is freely available online for download.The authors hope its release will accelerate TTS research.Source: §5 · p. 4