YODAS v3: Over 1 Million Hours of High-Bandwidth, Stereophonic, Multilingual Speech

Motivation

• Open speech data has a scale gap: nearly all open corpora together totaled only a little over one million hours as of late 2024, compared with proprietary systems exceeding 10 million hours.A footnote reports that open-corpus totals later doubled to roughly 2 million hours after another release, still far below proprietary corpora.Source: §1 · p. 1
• Most open speech datasets use mono audio at 16 or 24 kHz, limiting research that requires high-resolution spatial audio.The cited application areas include noisy multi-speaker ASR, audio codec modeling, expressive TTS, spatial audio processing, and stereo speech enhancement.Source: §1 · p. 1

Contribution

• YODAS v3 contains more than one million hours of multilingual speech in 48 kHz multi-channel OPUS, with 22 languages exceeding 10K hours and 73 exceeding 5K hours.The paper also describes the corpus as weakly supervised, with language tags, recognition transcripts, and English translations.Source: §1 · p. 1

2.1. Keyword Generation

• Language-specific Wikipedia keywords are filtered for form and then ranked by unigram likelihood to produce manageable search lists.The filters exclude punctuation- or number-only terms, terms beginning with punctuation or a digit, terms outside 3–30 characters, and specified file types.Source: §2.1 · pp. 1–2

2. Data Collection

• YODAS v3 is the first public dataset to exceed one million hours while supporting high-bandwidth stereo data.This is the scope of the comparison with other open large-scale speech datasets.Source: Table 1 · p. 2

2.2. Video Search

• Searching Creative Commons videos while prioritizing newer uploads helps discover new videos and avoids duplicating IDs already included in YODAS v2.Two searches revealed newly uploaded videos absent from the earlier search, and the final list excludes IDs found in YODAS v2.Source: §2.2 · p. 2

2.3. Downloading

• Audio is downloaded at the highest available quality, 48 kHz multi-channel OPUS, alongside original-language subtitles and English subtitles for non-English videos.Manual subtitles are used when available; otherwise, automatically generated original-language subtitles are downloaded. Transcripts and translations are timestamped at the utterance level, and video descriptions are also provided.Source: §2.3 · p. 2

3.1. Language Distribution

• 22 languages exceed 10K hours and 73 exceed 5K, while English accounts for less than 2% of the dataset.Language identity is proxied by each video’s YouTube-provided locale.Source: §3.1 · p. 2, §3.2 · p. 2, Figure 2 · p. 3

3.2. Length Distribution

• 1.7 million videos exceed five minutes and 1.2 million exceed ten minutes, while more than 24% of videos are under one minute.The authors attribute the short-video tail to YouTube Shorts and report that many 60+ minute videos are livestream recordings.Source: §3.2 · p. 3, Figure 3 · p. 3

3.3. True Bandwith Estimation

• 67.3% of recordings have 44.1-kHz effective bandwidth, and 92.5% exceed 32 kHz.The authors report bandwidth mismatches in about 100% of CommonVoice and DNS5 LibriVox samples and about 25% of LibriTTS samples.Source: Figure 4 · p. 3, §3.3 · p. 3

3.4. Audio Channels

• Over 71% of audio files are genuinely multichannel, with most of that data in stereo.The analysis compares channel pairs using normalized residual RMS and treats values below 1e-3 as effectively identical; some files contain 3–4 effective channels from 5.1 audio.Source: §3.4 · p. 4, Figure 5 · p. 4

4.1. Speech Recognition

• The ASR probe used 4,500 hours for English and roughly 100 hours for each of six other languages, with CTC segmentation and language- and score-based filtering.Models started from OWSM v4 base 102M, were trained for 40K steps, and were evaluated on CommonVoice using WER or Japanese CER.Source: §4.1 · p. 4

4.2. Neural Audio Codecs

• Codec models used a random 1,200-hour subset of YODAS v3 at 16, 24, and 48 kHz, all with the DAC architecture.They were trained using official ESPnet-Codec configurations.Source: §4.2 · p. 4

4.1. Speech Recognition

• WER almost always improves across all seven languages as θCTC decreases, with the best models for every language except English trained without score-based filtering.The authors report this contrasts with YODAS v2, which required significant filtering to avoid failed training runs with 100+ WER.Source: Table 3 · p. 4, §4.1 · p. 4

4.2. Neural Audio Codecs

• The 48 kHz DAC trained on YODAS v3 leads with STOI 0.97 / 0.94 and speaker similarity 0.84 / 0.90 on in-domain / out-of-domain tests.It outperforms models trained at lower sample rates and models trained on AMUSE’s 33K hours.Source: Table 4 · p. 4, §4.2 · p. 4

• YODAS v3 contains 1.1 million hours of high-bandwidth stereo speech across 147 languages.It is weakly supervised and was collected using a language-balanced keyword search approach.Source: §5 · p. 4
• The collection has over 10K hours in 22 languages and over 5K hours in 73 languages.Source: §5 · p. 4
• The paper analyzes audio properties and trains ASR and neural codec models, reporting clean transcripts and codecs competitive with the state of the art.It also reports that previous approaches fail to generalize to the complex audio scenes in web-scale data.Source: §5 · p. 4

• Generative AI was used to refine manuscript wording and prose.Source: §7 · p. 5
• Generative AI also helped write and scale code for large-scale crawling and data analyses.Source: §7 · p. 5
• Humans created and verified the final versions of artifacts produced with AI assistance.Source: §7 · p. 5