Source-linked AI summary
wav2letter++: The Fastest Open-source Speech Recognition System
Vineel Pratap, Awni Hannun, Qiantong Xu, Jeff Cai, Jacob Kahn, Gabriel Synnaeve, Vitaliy Liptchinsky, Ronan Collobert
TL;DR
Open-source ASR frameworks need efficient, scalable, and easy-to-extend systems for end-to-end speech recognition. wav2letter++ addresses this with a fully C++ architecture built around ArrayFire, flexible models, and optimized decoding, and reports faster training and decoding than comparable systems. Its comparisons include a criterion and optimization mismatch for Kaldi, while the authors note that further optimization remains possible.
Problem
Open-source ASR needs systems that support efficient large-scale training, simple model experimentation, and straightforward deployment.
Method
wav2letter++ combines a fully C++ framework with ArrayFire tensor operations, extensible end-to-end models, on-the-fly features, and an optimized beam-search decoder.
Results
wav2letter++ outperforms similar decoders by more than an order of magnitude and is more than 15% faster than the next-best system for smaller training models on one GPU.
Takeaways & Limitations
The simple, extensible interface makes wav2letter++ suited to rapid research in end-to-end speech recognition.
Takeaways & Limitations
The Kaldi comparison is not easily comparable because its LF-MMI criterion and optimization algorithm differ from the alternatives.
Abstract
from arXiv · showhide
This paper introduces wav2letter++, the fastest open-source deep learning speech recognition framework. wav2letter++ is written entirely in C++, and uses the ArrayFire tensor library for maximum efficiency. Here we explain the architecture and design of the wav2letter++ system and compare it to other major open-source speech recognition systems. In some cases wav2letter++ is more than 2x faster than other optimized frameworks for training end-to-end neural networks for speech recognition. We also show that wav2letter++'s training times scale linearly to 64 GPUs, the highest we tested, for models with 100 million parameters. High-performance frameworks enable fast iteration, which is often a crucial factor in successful research and model tuning on new datasets and tasks.
1. INTRODUCTION
wav2letter++ addresses the need for efficient, scalable, and research-friendly open-source ASR by providing a fully C++ speech recognition system. It focuses on training and decoding speed while retaining a simple development interface.
- End-to-end neural ASR systems are increasingly replacing HMM/GMM approaches because they are simpler and their accuracy gap is closing.
- wav2letter++ is introduced as the first open-source speech recognition system written completely in C++.
- The system targets training and decoding speed, scalability, and efficient development for end-to-end speech recognition.
- The paper motivates modern C++ as a way to maintain programming ease while enabling efficient and scalable software.
2. DESIGN
wav2letter++ is designed as a flexible, efficient C++ toolkit for end-to-end speech recognition, combining ArrayFire-based tensor operations with extensible models, on-the-fly features, training modes, and optimized decoding.
- 2. DESIGN: The design prioritizes efficient large-scale training, simple integration of new operations, and straightforward deployment with minimal new code.
- 2.1. ArrayFire Tensor Library: ArrayFire provides multi-backend tensor execution, just-in-time kernel fusion, reduced memory use, and a concise C++ interface.
- 2.2. Feature Extraction: Feature extraction supports multiple audio formats and feature types, computes features on the fly, and facilitates augmentation and single-binary deployment.
- 2.3. Models: The model abstraction separates networks from criteria and supports CTC, ASG, attention-based S2S models, and diverse architectures.
- 2.4. Training: Training supports flat-start, checkpoint continuation, and forking modes, plus standard optimizers and synchronous data-parallel SGD with NCCL2.
- 2.5. Decoding: Decoding uses an optimized beam search incorporating language-model constraints and a word-insertion term.
3. RELATED WORK
The paper situates wav2letter++ among major open-source ASR systems, contrasting their architectures, implementation languages, and development interfaces. It also highlights training-loop timing as a basis for performance comparison.
- 3. RELATED WORK: Kaldi, ESPNet, and OpenSeq2Seq represent major open-source ASR systems considered alongside wav2letter++.
- Kaldi: Kaldi provides stand-alone tools and supports HMM/GMM, hybrid HMM/NN, and phone-based modeling approaches.
- ESPNet: ESPNet integrates with Kaldi for preprocessing and uses Chainer or PyTorch back-ends for CTC and attention-based models.
- Training-loop analysis: Figure 3 measures the milliseconds spent in major training-loop steps, averaged per batch over a complete epoch.
- Implementation trade-offs: The comparison emphasizes the trade-off between scripting-language prototyping and statically typed, object-oriented development for large-scale systems.
4. EXPERIMENTS
The experiments compare wav2letter++ with major open-source ASR systems on training and decoding, using recurrent and convolutional networks and shared evaluation settings. wav2letter++ is faster in training and decoding, while some comparisons have explicit methodological limits.
- Comparative setup: The study evaluates ESPNet, Kaldi, OpenSeq2Seq, and wav2letter++ on WSJ training time and utterance decoding latency.Training uses machines with eight NVIDIA Tesla V100 GPUs per node.
- Training: Training comparisons use 30-million-parameter recurrent and 100-million-parameter convolutional networks with 40-dimensional log-mel inputs and CTC.OpenSeq2Seq is evaluated with float32 and mixed-precision float16 training.
- Training: More than 15% faster than the next-best system, wav2letter++ retains an advantage for 30-million-parameter models even on one GPU.Its advantage increases as computation scales out, including multi-node experiments at 16, 32, and 64 GPUs.
- Training: More than 1.5x lower epoch time is achieved by mixed-precision OpenSeq2Seq training in some cases, an optimization wav2letter++ could also use.Kaldi’s LF-MMI setup is not directly comparable and was omitted from Figure 4.
- Decoding: More than an order of magnitude faster decoding and considerably lower memory usage are reported for wav2letter++ at matched decoder inputs.The comparison targets 5.0% WER and each framework’s best-obtainable WER on LibriSpeech dev-clean.
5. CONCLUSION
The conclusion presents wav2letter++ as a fast, simple, entirely C++ framework for end-to-end speech recognition. It reports promising comparative performance and positions the system as a platform for rapid research while acknowledging room for further optimization.
- Conclusion: wav2letter++ is a fast and simple system for developing end-to-end speech recognizers, written entirely in C++.The authors connect the C++ implementation with efficient training and real-time decoding.
- Conclusion: Its simple and extensible interface makes wav2letter++ suited to rapid research in end-to-end speech recognition.The conclusion also leaves open further optimization of the framework and possible narrowing of the gap with Python-based systems.