Source-linked AI summary
WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit
Binbin Zhang, Di Wu, Zhendong Peng, Xingchen Song, Zhuoyuan Yao, Hang Lv, Lei Xie, Chao Yang, Fuping Pan, Jianwei Niu
TL;DR
Production deployment calls for stable, efficient E2E ASR, while prominent toolkits are described as research-oriented. WeNet 2.0 adds bidirectional decoding, language-model, contextual-biasing, and data-I/O updates, and reports substantial ASR improvement over original WeNet.
Problem
Production deployment needs stable, efficient E2E ASR, while prominent E2E toolkits are characterized as research-oriented.
Source: §1 · p. 1Method
WeNet 2.0 adds bidirectional U2++, an optional n-gram LM with WFST decoding, contextual biasing, and unified IO for varied dataset scales.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The toolkit offers production-oriented features and supports training from a few hours to millions of hours of data.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Recently, we made available WeNet, a production-oriented end-to-end speech recognition toolkit, which introduces a unified two-pass (U2) framework and a built-in runtime to address the streaming and non-streaming decoding modes in a single model. To further improve ASR performance and facilitate various production requirements, in this paper, we present WeNet 2.0 with four important updates. (1) We propose U2++, a unified two-pass framework with bidirectional attention decoders, which includes the future contextual information by a right-to-left attention decoder to improve the representative ability of the shared encoder and the performance during the rescoring stage. (2) We introduce an n-gram based language model and a WFST-based decoder into WeNet 2.0, promoting the use of rich text data in production scenarios. (3) We design a unified contextual biasing framework, which leverages user-specific context (e.g., contact lists) to provide rapid adaptation ability for production and improves ASR accuracy in both with-LM and without-LM scenarios. (4) We design a unified IO to support large-scale data for effective model training. In summary, the brand-new WeNet 2.0 achieves up to 10\% relative recognition performance improvement over the original WeNet on various corpora and makes available several important production-oriented features.
1. Introduction
Production deployment calls for stable, efficient E2E ASR, yet prominent E2E toolkits are described as research-oriented. WeNet 2.0 adds production-oriented features and reports substantial ASR improvement over the original WeNet.
- U2++ uses left-to-right and right-to-left context during training, then combines forward and backward predictions during decoding.Experiments report up to 10% relative error-rate reduction over original U2.Source: §1 · p. 1
- An optional n-gram language model uses production-accumulated text in a WFST decoding graph during streaming CTC decoding, with up to 8% relative performance improvement.Source: §1 · p. 1
- Contextual biasing uses user-specific information during streaming decoding with or without an LM, improving accuracy and providing rapid adaptation.Examples include contact lists, dialog state, conversation topic, and location.Source: §1 · p. 1
- Unified IO provides a common interface across storage types and dataset scales, using sample-level access for small datasets and shard-level access for large ones.It supports training from a few hours to millions of hours of data.Source: §1 · pp. 1–2
- WeNet 2.0 achieves substantial ASR improvement over original WeNet; U2++ achieves up to 10% relative reduction over original U2 in error rate.Source: §1 · pp. 1–2
2. WeNet 2.0
WeNet 2.0 updates its production-oriented ASR toolkit with U2++, language-model integration, contextual biasing, and unified IO. The updates address streaming and non-streaming decoding and support production data and user-specific context.
2.1. U2++
- U2++ adds a right-to-left decoder to model future context alongside the left-to-right decoder, improving shared-encoder representation and rescoring performance.CTC decodes in a streaming first pass; the attention decoders rescore in a non-streaming second pass.Source: §2.1 · p. 2
- Larger chunks yield better results with higher latency, while dynamic chunking lets decoding tune chunk size to balance accuracy and latency.Training samples chunk sizes dynamically so the model learns to predict with arbitrary chunk sizes.Source: §2.1 · p. 2
2.2. Language Model
- The unified language-model framework supports decoding with or without an n-gram LM, using CTC prefix beam search without an LM and a WFST graph when one is supplied.With an LM, the graph combines the n-gram model, lexicon, and CTC topology; attention rescoring selects among n-best candidates.Source: §2.2 · p. 2
2.3. Contextual Biasing
- Contextual biasing builds a WFST graph on the fly from known phrases, using boosted arcs and failure arcs to remove boosts when only part of a phrase matches.During streaming decoding, matching biasing units receive a boosted score; units are characters without an LM and vocabulary words with one.Source: §2.3 · pp. 2–3
2.4. UIO
- Production-scale datasets can contain tens of thousands of hours in many small files, creating memory and random-access speed problems that unified IO is designed to address.For large datasets, samples and metadata are packed into shards; small datasets can load samples directly.Source: §2.4 · p. 3
- Unified IO supports shard-level random access for large datasets and sample-level random access for small datasets, scaling from a few hours to millions of hours of training data.Large-dataset shards can be loaded from local or distributed storage.Source: §2.4 · p. 3
3. Experiments
Experiments evaluate U2++, production n-gram language modeling, contextual biasing, and unified IO across varied ASR corpora. The reported results show gains from U2++ and language modeling, controllable contextual-biasing benefits, and shard-mode speed or accuracy comparisons.
3. Experiments
- The experiments use five corpora spanning English and Mandarin, clean and noisy recordings, and corpus sizes from 100 to 10,000 hours.The U2++ evaluation uses 80-dimensional log Mel-filter-bank features and a 12-layer Conformer encoder.Source: §3 · p. 3, §3.1 · p. 3
3.1. U2++
- Up to 10% relative improvement over U2 is reported for U2++ across most evaluated corpora.The evaluation reports CER, WER, or MER depending on the corpus.Source: Table 1 · p. 4, §3.1 · p. 4
3.2. N-gram Language Model
- 5% gain on AISHELL-1; results are comparable on AISHELL-2 and LibriSpeech test-clean, while LibriSpeech test-other improves 8.42%.The paper concludes that production n-gram language modeling can use production-accumulated text data.Source: Table 2 · p. 4, §3.2 · p. 4
3.3. Contextual Biasing
- With boosted scores varied from 0 to 10, contextual biasing reduces positive-set errors; a proper setting avoids negative-set performance degradation.The test sets comprise positive utterances with relevant names supplied as context and negative utterances without those phrases.Source: §3.3 · p. 4, Table 3 · p. 4
3.4. UIO
- About 9.27% training-speed improvement is reported for AISHELL-1 shard mode, with close raw/shard accuracy; WenetSpeech shard results are comparable to ESPnet.WenetSpeech raw-mode training was too slow for a training-speed result to be shown.Source: Table 4 · p. 4, §3.4 · p. 4
4. Conclusion and Future Work
WeNet 2.0 is presented as a more productive end-to-end speech-recognition toolkit with production-oriented features and substantial ASR improvement. The authors say WeNet 3.0 work will focus on unsupervised self-learning, on-device exploration and optimization, and other production-level characteristics.
4. Conclusion and Future Work
- WeNet 2.0 introduces production-oriented features and achieves substantial ASR performance improvement.Source: §4 · p. 4
- Planned WeNet 3.0 work focuses on unsupervised self-learning and on-device model exploration and optimization.Source: §4 · p. 4
- WeNet 3.0 work also targets other characteristics for production-level ASR.Source: §4 · p. 4