Source-linked AI summary
CSS10: A Collection of Single Speaker Speech Datasets for 10 Languages
Kyubyong Park, Thomas Mulc
TL;DR
Public TTS datasets are limited, making results difficult to reproduce and models difficult to compare, while non-English languages have received less attention. CSS10 constructs single-speaker, aligned-text datasets for ten languages from LibriVox and validates them with two TTS models and MOS tests. Models were trained successfully on the datasets, though Tacotron training failed for Greek and some Japanese samples included irrelevant trailing utterances.
Problem
The lack of public benchmark datasets makes TTS results difficult to reproduce and models hard to compare, while freely available non-English speech data is scarce.
Source: §1 · p. 1Method
CSS10 creates ten single-speaker datasets with aligned text and evaluates Tacotron and DCTTS using MOS tests on synthesized samples.
Source: §1 · p. 1Results
Models trained successfully on the datasets except Tacotron on Greek; DCTTS and Tacotron had similar pronunciation accuracy, while DCTTS was statistically more natural for German, French, and Spanish.
Source: §4.4 · p. 3Takeaways & Limitations
The authors release the datasets, source code, pre-trained models, and evaluation data publicly.
Source: §5 · p. 4In a few Japanese samples, irrelevant utterances appeared at the end, which the authors suspect may stem from imperfect automatic phonetic transcription in training data.
Source: §4.4 · p. 4
Abstract
from arXiv · showhide
We describe our development of CSS10, a collection of single speaker speech datasets for ten languages. It is composed of short audio clips from LibriVox audiobooks and their aligned texts. To validate its quality we train two neural text-to-speech models on each dataset. Subsequently, we conduct Mean Opinion Score tests on the synthesized speech samples. We make our datasets, pre-trained models, and test resources publicly available. We hope they will be used for future speech tasks.
1. Introduction
The lack of public TTS datasets makes results difficult to reproduce and models hard to compare, while research has largely focused on English. CSS10 addresses these gaps with aligned single-speaker datasets for ten languages, model evaluations, and public resources.
1. Introduction
- The shortage of public benchmark datasets makes TTS results difficult to reproduce and different models hard to compare.Most neural TTS models were evaluated on internal datasets.Source: §1 · p. 1
- Because freely available non-English speech data is scarce, TTS research has mostly focused on English.The authors argue that other languages also deserve research attention.Source: §1 · p. 1
- CSS10 provides single-speaker speech datasets with aligned text for ten languages.Source: §1 · p. 1
- The authors train two TTS models on each dataset and evaluate synthesized speech with Mean Opinion Scores.Source: §1 · p. 1
- The datasets and other project resources are publicly available through the authors’ repository.Source: §1 · p. 1
2. Related work
Prior resources differ in language coverage, speaker setup, recording conditions, and suitability for regular single-speaker TTS. CSS10 is positioned among audiobook-derived multilingual resources while addressing constraints found in existing datasets.
2. Related work
- LJ Speech offers more than 20 hours of single-speaker audiobook audio and has been verified for neural TTS, while WEB shares similar properties but is sampled at 12 kHz.Other English corpora listed include Blizzard, VCTK, LibriSpeech, TED-LIUM, VoxForge, and Common Voice; these are multi-speaker and thus considered unideal for single-speaker TTS.Source: §2 · p. 1
- JSUT is a roughly 10-hour Japanese single-speaker dataset with aligned text, recorded in a controlled environment, but lacks phonetic transcription and follow-up experiments.Source: §2 · p. 1
- Pavoque provides over 12 hours of German multi-style speech with phoneme-level annotations, but is better suited to speech-style tasks than regular TTS.Source: §2 · p. 1
- The Spoken WP Corpus Collection has hundreds of hours of aligned text and audio in Dutch, German, and English, but its minute-long clips complicate neural TTS use.Much of each language’s audio comes from a single speaker.Source: §2 · p. 1
- Tundra and M-AILABS cover 14 and 9 audiobook-based languages, respectively, but Tundra uses one audiobook per language and M-AILABS uses multiple speakers within each language.M-AILABS focuses on European languages and has nearly one thousand total audio hours.Source: §2 · p. 1
3. Datasets
CSS10 builds multilingual single-speaker datasets from LibriVox audiobooks by selecting suitable recordings, segmenting audio, and aligning and processing text. The resulting datasets pair audio with original and processed text.
3. Datasets
- The authors choose LibriVox audiobooks because they come with text, are often performed by one speaker, and tend toward a regular speaking speed.Source: §3 · pp. 1–2
3.1. Selection of audiobooks
- Recordings are excluded when a language has under four hours of solo audio, text is unavailable, or audio has noticeable noise.The four-hour threshold reflects prior success training DCTTS on an audiobook of that duration.Source: §3.1 · p. 2, §3.3 · p. 2
- Selection yields Chinese, Dutch, French, Finnish, German, Greek, Hungarian, Japanese, Russian, and Spanish datasets, all sampled at 22 kHz.Source: §3.1 · p. 2
3.2. Audio processing
- Audio is split at silences longer than 0.5 seconds, or 0.25 seconds for Spanish, then neighboring clips are adjusted to around 10 seconds.The authors report improved computational efficiency; about 85% of Spanish clips last 5–11 seconds, with similar distributions in other languages.Source: §3.2 · p. 2
3.3. Text processing
- Experts align text to each segmented audio clip after the authors reject a forced aligner because it does not guarantee correct, language-independent alignments.Source: §3.3 · p. 2
3.3.1. Text normalization
- Text normalization expands abbreviations and spells out numerals, retains case, and removes infrequent non-punctuation symbols.Source: §3.3.1 · p. 2
3.3.2. Phonetic transcription
- Chinese text is converted to pinyin and Japanese text to Romanized pronunciations, with native-speaker pronunciation entry when MeCab fails.The datasets contain audio, original text, and processed text triplets.Source: §3.3.2 · p. 2
4. Experiments
The experiments validate the datasets by training Tacotron and DCTTS and evaluating synthesized speech with MOS tests. DCTTS generally scored better for naturalness and produced cleaner-sounding samples, while pronunciation accuracy was similar; Greek was an exception to successful Tacotron training.
4.1. Models
- Tacotron and DCTTS, both attention-based sequence-to-sequence neural TTS models, were trained on each dataset to validate it.Tacotron uses a CBHG backbone and end-to-end training; DCTTS uses convolution layers and two independently trained networks.Source: §4 · p. 2, §4.1 · p. 2
4.2. Training
- Both models were trained for 400k steps with mostly paper-recommended hyperparameters and the same settings across languages.Training took around ten days for Tacotron and three days for DCTTS.Source: §4.2 · p. 3
4.3.1. Test sentences
- Evaluation used 20 Tatoeba sentences per language, selected to maximize coverage of vocabulary letters; Chinese and Japanese used phonetic transcriptions.Some very rare letters were left out.Source: §4.3.1 · p. 3
4.3.2. Mean Opinion Scores
- Workers used a 1-to-5 absolute category rating to score speech naturalness and pronunciation accuracy, with language-specific transcriptions used to screen for truthful responses.Naturalness followed a published rubric, while the authors designed a pronunciation-accuracy rubric.Source: §4.3.2 · p. 3
4.4. Results
- Tacotron training did not succeed for Greek, and Greek DCTTS samples were much worse than those from other languages.The authors believe Greek’s roughly four hours of audio may explain the Tacotron failure.Source: §4.4 · p. 3
- DCTTS was statistically more natural than Tacotron for German, French, and Spanish, while the models were somewhat similar in pronunciation accuracy.MOS results are reported in Table 3 and Figures 2 and 3.Source: §4.4 · p. 3, Figure 2 · p. 3, Figure 3 · p. 4
- DCTTS score distributions were more skewed toward higher ratings for both measures, especially naturalness, though averaging Likert scores may be inappropriate.The authors also report that DCTTS samples sounded better and fairly clean, while Tacotron outputs consistently had noise.Source: §4.4 · pp. 3–4
- Both models sometimes produced end-of-sample mumbling that could be removed by trimming; a few Japanese samples ended with utterances unrelated to their input text.The authors suspect the Japanese issue may stem from imperfect automatic phonetic transcription in the training data.Source: §4.4 · p. 4
5. Conclusions
CSS10 comprises single-speaker speech datasets for ten languages, and the authors successfully trained TTS models on them despite language-dependent performance differences. They publicly release project resources.
5. Conclusions
- Models trained successfully on CSS10 datasets despite performance differences across languages.The work uses the collection for a TTS task.Source: §5 · p. 4
- The authors release CSS10 project resources, including source code, datasets, pre-trained models, and evaluation data.Source: §5 · p. 4
- CSS10 contains single-speaker speech datasets for ten languages.Source: §5 · p. 4
6. Future work
The authors hope CSS10 will serve as a benchmark for future non-English TTS research and note possible future extensions, including Korean recordings and other speech tasks.
6. Future work
- The authors hope CSS10 and their experiments will serve as benchmarks for future non-English TTS research.Source: §6 · p. 4
- The authors report errors in some automatic phonetic transcriptions for Japanese and Chinese.Source: §6 · p. 4
- Figure 4 presents percentages of naturalness scores by model and language.Source: Figure 4 · p. 4
- Figure 5 presents percentages of pronunciation accuracy scores by model and language.Source: Figure 5 · p. 4
- The authors plan to add a Korean dataset, potentially recording speech themselves because LibriVox has too few Korean audiobooks.Source: §6 · p. 4
- CSS10 may also be used for other speech tasks, including multilingual speech recognition.Source: §6 · p. 4