Source-linked AI summary
AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale
Jiayu Du, Xingyu Na, Xuechen Liu, Hui Bu
TL;DR
Mandarin ASR research lacks large-scale, high-quality free data that reliably supports industrial-scale development. AISHELL-2 addresses this gap with a 1000-hour open corpus, a self-contained Kaldi recipe, and multi-channel development and test data. The release is intended as a practical baseline and research resource for scalable Mandarin speech recognition.
Problem
Mandarin ASR research has limited access to real-world application data, while a large-scale, high-quality free corpus is still needed.
Method
The paper releases AISHELL-2 data and a self-contained Kaldi recipe covering corpus preparation, lexicon and word segmentation, language modeling, acoustic-model training, and evaluation.
Results
AISHELL-2 provides 1000 hours of clean iOS speech for academic research, plus development and test data from iOS, Android, and Mic channels.
Takeaways & Limitations
The corpus and recipe provide a research baseline for exploring more scalable and practical Mandarin ASR solutions for industrial scenarios.
Takeaways & Limitations
GMM training stops at the speaker-independent stage, with speaker-dependent processing deferred to later DNN training.
Abstract
from arXiv · showhide
AISHELL-1 is by far the largest open-source speech corpus available for Mandarin speech recognition research. It was released with a baseline system containing solid training and testing pipelines for Mandarin ASR. In AISHELL-2, 1000 hours of clean read-speech data from iOS is published, which is free for academic usage. On top of AISHELL-2 corpus, an improved recipe is developed and released, containing key components for industrial applications, such as Chinese word segmentation, flexible vocabulary expension and phone set transformation etc. Pipelines support various state-of-the-art techniques, such as time-delayed neural networks and Lattic-Free MMI objective funciton. In addition, we also release dev and test data from other channels(Android and Mic). For research community, we hope that AISHELL-2 corpus can be a solid resource for topics like transfer learning and robust ASR. For industry, we hope AISHELL-2 recipe can be a helpful reference for building meaningful industrial systems and products.
1. Introduction
AISHELL-2 addresses the limited availability of large-scale, high-quality open Mandarin ASR data and the difficulty of translating research advances to industrial scenarios. It releases a 1000-hour corpus with a self-contained Kaldi baseline and multi-channel evaluation resources.
- Motivation: Mandarin ASR research still lacks a large-scale, high-quality free corpus comparable to major open datasets in computer vision.Existing Mandarin resources such as thchs30 and hkust do not fully meet this need.
- Background: AISHELL-1 provided 170 hours of high-quality Mandarin speech with human transcriptions and Kaldi-based training and evaluation recipes.It was presented as a fully open-sourced Mandarin ASR system and dataset.
- Contribution: AISHELL-2 introduces an open-sourced, self-contained baseline for industrial-scale Mandarin ASR research.The release combines data, recipes, and evaluation resources intended to support more scalable and practical research.
- Contribution: The release includes 1000 hours of iPhone-recorded speech, Kaldi recipes with Chinese word segmentation and customizable lexicon support, and development and test data from iOS, Android, and Mic channels.These components extend the corpus beyond training data and support evaluation across acoustic conditions.
2. AISHELL-2 corpus
AISHELL-2 provides 1000 hours of clean read speech recorded through parallel acoustic channels, with the iOS channel open-sourced for academic research. Its development and test sets add balanced, multi-speaker evaluation data spanning the released recording setup.
- Corpus design: AISHELL-2 contains 1000 hours of clean read-speech data recorded in parallel through Mic, Android, and iOS channels, with iOS data open-sourced.The relative placement of speakers and devices is documented in the recording setup.
- Speaker coverage: The corpus includes 1991 speakers, comprising 845 male and 1146 female participants with ages ranging from 11 to over 40.Speakers include Northern, Southern, and other Mandarin accent varieties.
- Recording environment: Recordings were collected from 1347 studio participants and additional speakers in living rooms with natural reverberation.This setup includes both controlled and naturally reverberant environments.
- Speech content: Speech content covers eight topics, including voice commands, entertainment, finance, technology, sports, and free speaking, across around half a million prompts.The topics include both practical command-oriented speech and broader conversational content.
- Evaluation sets: The development set contains 2500 utterances from five speakers, while the test set contains 5000 utterances from ten speakers.Each speaker contributed approximately half an hour covering 500 prompts, and speaker gender is balanced.
3. AISHELL-2 recipe
The AISHELL-2 recipe provides a complete Kaldi pipeline for Mandarin ASR, addressing segmentation, lexicon customization, acoustic modeling, and language modeling. Its design separates vocabulary expansion from phone-set adaptation and includes speaker-aware neural training.
- The released Kaldi recipes cover data and lexicon preparation, language-model training, GMM and neural-network training, and test-set evaluation.
- Lexicon and word segmentation: DaCiDian separates word-to-Pinyin mapping from Pinyin-to-phoneme mapping, enabling vocabulary expansion and phone-set adaptation independently.New words can be added to the first layer, while developers can redefine the second layer for their phone set.
- Lexicon and word segmentation: Jieba supplies trie-tree-based word segmentation with vocabulary customization for AISHELL-2 transcriptions and language-model text.
- Acoustic model: Acoustic training uses maximum-likelihood GMM-HMM modeling followed by hybrid DNN-HMM state estimation in Kaldi.GMM training stops at the speaker-independent stage, with speaker-dependent information incorporated later through neural-network training.
- Acoustic model: The neural baseline uses an eight-layer TDNN with 1280 hidden units per layer, 43-dimensional features, 100-dimensional i-vectors, and LFMMI training.The i-vector extractor is trained on the corpus, encoding speaker information at the neural-network stage.
- Language model: A trigram language model is trained on 5.7 million-word transcripts using Kneser-Ney smoothing, with OOV words mapped to <UNK>.
4. Experiment and evaluations
The baseline is evaluated with Character Error Rate across three acoustic channels after training only on open-sourced iOS data. iOS performs better than Android and Mic, consistent with closer channel-condition matching.
- Character Error Rate (CER) is the evaluation metric for the baseline system.
- The models are trained only on open-sourced iOS data and evaluated on Android, Mic, and iOS development and test sets.
- iOS performance exceeds Android and Mic performance, which the paper attributes to better acoustic-channel condition matching.
5. Conclusions
AISHELL-2 contributes a freely available 1000-hour Mandarin ASR corpus and a self-contained Kaldi research baseline. The project is intended to support research on scalable, practical Mandarin speech-recognition solutions for industrial scenarios.
- AISHELL-2 is a 1000-hour Mandarin ASR corpus freely available to the research community.
- The paper presents a self-contained recipe in the Kaldi toolkit as a research baseline.
- The project aims to provide ingredients for exploring scalable and practical Mandarin speech-recognition solutions in industrial scenarios.