Source-linked AI summary
GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio
Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Yujun Wang, Zhao You, Zhiyong Yan
TL;DR
English speech recognition needs larger and more diverse corpora than many established resources provide. GigaSpeech builds an evolving corpus from multiple sources and speaking styles using a segmentation and validation pipeline, with baselines and training subsets for system development. It provides 10,000 hours of labeled audio, while its validation choices trade transcription strictness against scale for the XL subset.
Problem
Established English speech corpora are often old, smaller than 1,000 hours, or concentrated in limited speaking styles and domains.
Method
The paper constructs GigaSpeech from audiobook, podcast, and YouTube audio using forced alignment, segmentation, normalization, and transcription-quality filtering.
Results
10,000 hours of validated labeled audio are provided, with maximum WER set at 4% for XL training data and 0% for smaller subsets.
Takeaways & Limitations
GigaSpeech offers multi-source, multi-style, multi-topic training data with baseline systems and multiple subset sizes for speech recognition experiments.
Takeaways & Limitations
The evaluation sets do not cover audiobooks.
Abstract
from arXiv · showhide
This paper introduces GigaSpeech, an evolving, multi-domain English speech recognition corpus with 10,000 hours of high quality labeled audio suitable for supervised training, and 40,000 hours of total audio suitable for semi-supervised and unsupervised training. Around 40,000 hours of transcribed audio is first collected from audiobooks, podcasts and YouTube, covering both read and spontaneous speaking styles, and a variety of topics, such as arts, science, sports, etc. A new forced alignment and segmentation pipeline is proposed to create sentence segments suitable for speech recognition training, and to filter out segments with low-quality transcription. For system training, GigaSpeech provides five subsets of different sizes, 10h, 250h, 1000h, 2500h, and 10000h. For our 10,000-hour XL training subset, we cap the word error rate at 4% during the filtering/validation stage, and for all our other smaller training subsets, we cap it at 0%. The DEV and TEST evaluation sets, on the other hand, are re-processed by professional human transcribers to ensure high transcription quality. Baseline systems are provided for popular speech recognition toolkits, namely Athena, ESPnet, Kaldi and Pika.
1. Introduction
GigaSpeech addresses the limited scale, age, and domain coverage of existing English speech corpora with an evolving 10,000-hour, multi-domain resource and a scalable corpus-generation pipeline.
- GigaSpeech combines audiobook, podcast, and YouTube sources to broaden coverage beyond existing domain-focused corpora.Existing resources include audiobook-heavy, telephone-conversation, TED-talk, and business-domain data.
- The corpus covers both read and spontaneous speech, alongside topics including arts, science, and sports.
- Original and normalized transcription pairs support end-to-end systems with post-processing such as punctuation and case or date/time normalization.
- 10,000 hours of labeled audio are released in an evolving, multi-domain English speech recognition corpus.
- The authors provide a scalable, reliable pipeline for generating speech recognition corpora in addition to releasing GigaSpeech.
2. GigaSpeech Corpus
GigaSpeech organizes its corpus through version-controlled metadata, five training subsets, and separate DEV/TEST evaluation sets. Audio is compressed to reduce storage, with only a small measured WER impact.
- Metadata: GigaSpeech stores corpus metadata in the version-controlled GigaSpeech.json file, which users follow to locate audio, segments, transcripts, and supplementary entries.Future releases plan to add speaker information and more diverse sources.
- Training subsets: Five training subsets—XS, S, M, L, and XL—are provided in increasing order of audio hours.The supplied passage does not include the detailed hour breakdown from Table 1.
- Evaluation sets: 12.5 hours of audio comprise DEV and 40.3 hours comprise TEST; neither evaluation set covers audiobooks.LibriSpeech evaluation files are excluded from the corpus and can also be used for evaluation.
- Audio format: 32 kbps Opus compression is applied after conversion to 16 kHz, single-channel, 16-bit audio, achieving an 8-fold compression ratio.The compression is intended to reduce corpus file size.
- Audio format: 0.1–0.2% WER degradation results when training data uses 32 kbps Opus compression on DEV and TEST evaluation.The comparison uses Kaldi systems trained on the M (1000h) subset, without recurrent neural network language-model rescoring.
3. GigaSpeech Creation Pipeline
The corpus-creation section presents a detailed pipeline for building GigaSpeech and indicates that the same pipeline can support other data-generation tasks.
- The GigaSpeech creation pipeline is presented as a detailed procedure that can also be applied to other data-generation tasks.
3.1. Stage 1: Audio Collection
Audio collection begins by defining topical categories and then selecting podcast, YouTube, and audiobook sources according to their available transcripts or captions.
- The collection process manually defines 24 topical categories spanning areas such as arts, business, education, science, sports, and travel.
- Podcasts are selected when episodes have manual transcriptions, while YouTube videos are selected using category seed keywords and human-generated closed captions.
- Audiobook collection does not enforce the manually defined topical categories.
- After source lists are assembled, tools download the audio files together with their corresponding transcripts.
3.2. Stage 2: Text Normalization
The pipeline normalizes heterogeneous transcripts and segments long recordings into short training units, while avoiding unreliable YouTube caption timestamps.
- Text normalization: Text normalization addresses differing transcription standards by normalizing case, removing special symbols, and rewriting numbers, dates, and times as words.
- Segmentation: Audiobook and podcast transcripts are split from episode- or book-level text into training segments shorter than 20 seconds.
- Segmentation: Because YouTube caption timestamps are unreliable for segmentation, the captions are spliced together before further processing.
3.3. Stage 3: Forced Alignment
The forced-alignment stage aligns audio and transcripts through chunking, biased-LM decoding, timestamped hypotheses, and TF-IDF matching. An in-domain acoustic model then re-aligns the corpus.
- Audio and transcripts are uniformly chunked before alignment.
- Biased-LM decoding generates hypotheses with timestamps for each audio chunk.
- Each timestamped hypothesis is matched to one transcript segment using TF-IDF similarity.
- A close-domain acoustic model first creates about 3,000 hours of segments, enabling an in-domain model to align the full corpus.
3.4. Stage 4: Audio Segmentation
The segmentation stage converts alignment information into sentence-level audio segments using silence and punctuation boundaries, while removing poorly aligned or excessively long segments.
- Segmentation begins by extracting audio segments from the alignment information.
- Splits are allowed after silence longer than 1 second or punctuation followed by more than 0.2 seconds of duration.
- Segments with alignment WER ≥75% or length ≥20 seconds are removed.
- Silence at segment boundaries is truncated to 0.15 seconds.
- Four punctuation types are retained and mapped to special words, supporting sentence-boundary splitting, punctuation tagging, and endpoint detection.
3.5. Stage 5: Segment Validation
Segment validation filters transcription errors with a flexible alignment graph and decoding pass. Reference rewriting preserves some spontaneous-speech diversity in the XL subset.
- Candidate segments are validated because segmentation can produce segments with high transcription error rates.
- The alignment graph uses leaky arcs, a null state, and a 1,000-word garbage loop to accommodate audio-transcript discrepancies.
- The validation decoder detects omitted words and typographical errors in transcripts.
- The XL podcast and YouTube data cap WER at 4%, while the XL audiobook data and smaller subsets cap it at 0%.
- Validated segments commonly omit fillers, conjunctions, or disfluencies.
- A filler loop and disfluency detector can rewrite references, with rewriting applied only to the 10,000-hour XL subset.
3.6. Stage 6: Evaluation
Evaluation uses professionally transcribed sets as ground truth to measure frame-level segmentation quality. Working points are selected to balance validated audio volume against WER limits.
- Professional human transcriptions provide ground truth for evaluating frame-level segmentation precision and recall.
- Recall measures retrieved frames, while precision measures retrieved frames that match human labels.
- The XL subset selects a working point yielding 10,000 hours of validated audio while keeping maximum WER below 4%.
- Other training subsets use the leftmost working point that keeps maximum WER at 0%.
4. Experiments
The experiments provide baseline systems across four speech-recognition toolkits and evaluate segmentation quality, while showing that larger GigaSpeech training subsets generally improve Kaldi performance. These baselines are toolkit-specific reference systems, not cross-toolkit performance comparisons.
- Baseline systems: Four toolkit baselines are provided: Athena, ESPnet, Kaldi, and Pika.The systems are intended to support baseline experiments for each toolkit.
- Athena: Athena uses an encoder-decoder Transformer with joint CTC training, beam-search decoding, and RNN language-model scoring.The decoder, CTC, and language-model scores are combined during decoding.
- ESPnet: ESPnet uses a Conformer encoder with a Transformer decoder and 5k BPE tokens generated by SentencePiece.The encoder has 12 Conformer blocks, while the decoder has 6 Transformer blocks.
- Kaldi: Kaldi uses a chain model with GMM-HMM alignments, augmentation, i-vectors, and neural-network training with cross-entropy and LF-MMI criteria.Decoding applies a 4-gram language model followed by RNNLM rescoring.
- Experimental results: The reported toolkit baselines are not state-of-the-art results and should not be used to compare performance across toolkits.Table 4 provides these results solely as baseline systems for each toolkit.
- Experimental results: As the Kaldi training subset grows, performance generally improves; the 10-hour XS subset is intended for system building and debugging rather than strong performance.Table 5 reports Kaldi baselines across four training subsets.