Source-linked AI summary
LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus
Yuma Koizumi, Heiga Zen, Shigeki Karita, Yifan Ding, Kohei Yatabe, Nobuyuki Morioka, Michiel Bacchiani, Yu Zhang, Wei Han, Ankur Bapna
TL;DR
High-quality public TTS data are limited, while LibriTTS speech quality constrains models trained on it. The paper applies text-informed speech restoration to LibriTTS to create and release LibriTTS-R, and reports improved TTS naturalness comparable to ground-truth speech.
Problem
Large, high-quality public TTS datasets are scarce, and LibriTTS speech quality is inadequate for training high-quality TTS models.
Method
The paper applies the text-informed Miipher speech-restoration model to LibriTTS, using cleaned speech features and neural waveform synthesis.
Results
TTS models trained with LibriTTS-R show improved speech naturalness compared with models trained with LibriTTS and are comparable with ground-truth speech.
Takeaways & Limitations
LibriTTS-R provides a publicly available, quality-improved version of LibriTTS for TTS research.
Takeaways & Limitations
A small number of distorted samples may remain because not all LibriTTS-R speech samples were manually checked.
Abstract
from arXiv · showhide
This paper introduces a new speech dataset called ``LibriTTS-R'' designed for text-to-speech (TTS) use. It is derived by applying speech restoration to the LibriTTS corpus, which consists of 585 hours of speech data at 24 kHz sampling rate from 2,456 speakers and the corresponding texts. The constituent samples of LibriTTS-R are identical to those of LibriTTS, with only the sound quality improved. Experimental results show that the LibriTTS-R ground-truth samples showed significantly improved sound quality compared to those in LibriTTS. In addition, neural end-to-end TTS trained with LibriTTS-R achieved speech naturalness on par with that of the ground-truth samples. The corpus is freely available for download from \url{http://www.openslr.org/141/}.
1 Google, Japan, 2 Tokyo University of Agriculture & Technology, Japan, 3 Google, USA
The passage lists three email addresses using the google.com domain.
- Three email addresses are listed in the passage.
- All listed addresses use the google.com domain.
- The usernames are koizumiyuma, heigazen, and karita.
1. Introduction
High-quality public TTS data remains limited, motivating the restoration of public speech corpora. The paper addresses this barrier by releasing LibriTTS-R, whose trained TTS models show improved naturalness comparable to ground-truth speech.
- Large, high-quality public TTS datasets remain scarce, making results from private studio-recorded datasets difficult to reproduce.High-quality TTS training often uses over 100 hours of studio-recorded data that are not publicly available.
- Speech restoration models can convert reverberated lecture and historical speech into studio-recorded quality, motivating restoration of public datasets.
- LibriTTS-R is a quality-improved version of LibriTTS produced with the text-informed Miipher restoration model.Miipher uses a w2v-BERT feature cleaner and WaveFit neural vocoder.
- LibriTTS-R is publicly available under the same non-restrictive license, with ground-truth and TTS-generated audio available through a demo page.
2. The LibriTTS corpus
LibriTTS is a large, multi-speaker TTS corpus, but its speech quality limits the quality of models trained on it compared with higher-quality data.
- LibriTTS contains 585 hours of 24 kHz speech from 2,456 speakers with corresponding texts.The materials were derived from LibriSpeech and re-derived for TTS-related sampling-rate and text-normalization issues.
- LibriTTS is one of the largest multi-speaker speech datasets designed for TTS use.
- LibriTTS sound quality is below that of smaller, higher-quality TTS datasets, affecting the quality of generated speech.
- 3.45 MOS for Glow-TTS on LibriTTS contrasted with 4.22 for vocoder speech from ground-truth mel-spectrograms.On LJspeech, the corresponding generated and ground-truth MOSs were 4.01 and 4.19.
3. Data processing pipeline
The pipeline applies speech restoration to LibriTTS using Miipher, designed to reduce failed recoveries and address masking and deletion degradations before synthesizing restored waveforms.
- LibriTTS is processed with a speech-restoration model to provide a public large-scale, high-quality TTS dataset.
- Robust restoration is required because artifacts in many restored samples would adversely affect subsequent TTS training.
- Miipher extracts w2v-BERT features, cleans them with a DF-Conformer feature-cleaner, and synthesizes the restored waveform with WaveFit-5.
- Miipher targets phoneme masking and deletion using w2v-BERT features and linguistic-feature conditioning.These degradations can arise from noise, reverberation, nonlinear processing, or down-sampling.
- Miipher was trained with 2,680 hours of noisy and studio-quality speech pairs, including 670 hours of studio-recorded English speech.
- The pipeline computes linguistic features from transcripts and speaker embeddings, extracts features from down-sampled audio, and synthesizes restored 24 kHz waveforms.
4. Experiments
Experiments evaluated LibriTTS-R against LibriTTS for ground-truth quality, content and speaker preservation, and multi-speaker TTS performance. LibriTTS-R improved sound quality and TTS naturalness while preserving text content and speaker characteristics, though a few restoration failures remained.
- Ground-truth evaluation: 620 samples from each “test-clean” and “test-other” subset were evaluated using MOS and side-by-side preference tests.MOS measured naturalness on a 5-point scale, while SxS measured which sound quality was better on a 7-point scale.
- Ground-truth evaluation: 4.36 and 3.94 were the LibriTTS MOSs on “test-clean” and “test-other”, respectively, while LibriTTS-R scored better on both splits without a significant naturalness difference.The small naturalness difference was attributed to both datasets containing real human speech.
- Ground-truth evaluation: Significant sound-quality differences favored LibriTTS-R on both evaluated splits in the SxS tests.Spectrograms showed restoration of degradations including down-sampling, environmental noise, reverberation, and non-linear speech enhancement.
- Content and speaker preservation: WER was 3.4 versus 3.2 on “test-clean” and 5.1 versus 5.1 on “test-other” for LibriTTS and LibriTTS-R, respectively, indicating preserved text content.Speaker cosine similarities were 0.784 and 0.755 for LibriTTS, versus 0.762 and 0.745 for LibriTTS-R across the two splits; the authors considered speaker characteristics consistent.
- TTS evaluation: LibriTTS-R Train-960 achieved the highest MOS for all speaker IDs except 19, and its MOS was on a par with human-spoken LibriTTS samples.Using LibriTTS Train-960 sometimes decreased MOS, whereas LibriTTS-R Train-960 benefited from restored “train-other-500” data.
- TTS evaluation: LibriTTS-R improved both naturalness and sound quality of TTS outputs, with reduced harmonic degradation and background noise in the illustrated examples.The multi-speaker models used the same architecture and hyper-parameters, allowing the training corpus to be compared directly.
5. Conclusions
LibriTTS-R is a sound-quality-improved version of LibriTTS created by applying speech restoration. Subjective experiments found improved TTS naturalness comparable to ground-truth speech, and the corpus is freely available.
- LibriTTS-R improves the sound quality of the LibriTTS corpus through speech restoration.
- Subjective experiments showed that TTS trained with LibriTTS-R had improved speech naturalness compared with TTS trained with LibriTTS.
- TTS trained with LibriTTS-R achieved speech naturalness comparable to ground-truth samples.
- The LibriTTS-R corpus is freely available for download online.