PaperlayerView exact revision on arXiv

Source-linked AI summary

wav2letter++: The Fastest Open-source Speech Recognition System

Vineel Pratap, Awni Hannun, Qiantong Xu, Jeff Cai, Jacob Kahn, Gabriel Synnaeve, Vitaliy Liptchinsky, Ronan Collobert

arXiv:1812.07625v1cs.CL

TL;DR

Open-source ASR frameworks need efficient, scalable, and easy-to-extend systems for end-to-end speech recognition. wav2letter++ addresses this with a fully C++ architecture built around ArrayFire, flexible models, and optimized decoding, and reports faster training and decoding than comparable systems. Its comparisons include a criterion and optimization mismatch for Kaldi, while the authors note that further optimization remains possible.

  • Problem

    Open-source ASR needs systems that support efficient large-scale training, simple model experimentation, and straightforward deployment.

    Source: §1 · p. 1, §2 · p. 1

  • Method

    wav2letter++ combines a fully C++ framework with ArrayFire tensor operations, extensible end-to-end models, on-the-fly features, and an optimized beam-search decoder.

    Source: §1 · p. 1, §2.1 · p. 1, §2.3 · p. 2

  • Results

    wav2letter++ outperforms similar decoders by more than an order of magnitude and is more than 15% faster than the next-best system for smaller training models on one GPU.

    Source: §4.2 · p. 4, §4.1 · p. 4

  • Takeaways & Limitations

    The simple, extensible interface makes wav2letter++ suited to rapid research in end-to-end speech recognition.

    Source: §5 · p. 4

    The Kaldi comparison is not easily comparable because its LF-MMI criterion and optimization algorithm differ from the alternatives.

    Source: §4.1 · p. 4

Abstract

from arXiv · show

This paper introduces wav2letter++, the fastest open-source deep learning speech recognition framework. wav2letter++ is written entirely in C++, and uses the ArrayFire tensor library for maximum efficiency. Here we explain the architecture and design of the wav2letter++ system and compare it to other major open-source speech recognition systems. In some cases wav2letter++ is more than 2x faster than other optimized frameworks for training end-to-end neural networks for speech recognition. We also show that wav2letter++'s training times scale linearly to 64 GPUs, the highest we tested, for models with 100 million parameters. High-performance frameworks enable fast iteration, which is often a crucial factor in successful research and model tuning on new datasets and tasks.

1. INTRODUCTION

wav2letter++ addresses the need for efficient, scalable, and research-friendly open-source ASR by providing a fully C++ speech recognition system. It focuses on training and decoding speed while retaining a simple development interface.

  • End-to-end neural ASR systems are increasingly replacing HMM/GMM approaches because they are simpler and their accuracy gap is closing.Source: §1 · p. 1
  • wav2letter++ is introduced as the first open-source speech recognition system written completely in C++.Source: §1 · p. 1
  • The system targets training and decoding speed, scalability, and efficient development for end-to-end speech recognition.Source: §1 · p. 1
  • The paper motivates modern C++ as a way to maintain programming ease while enabling efficient and scalable software.Source: §1 · p. 1

2. DESIGN

wav2letter++ is designed as a flexible, efficient C++ toolkit for end-to-end speech recognition, combining ArrayFire-based tensor operations with extensible models, on-the-fly features, training modes, and optimized decoding.

2. DESIGN

  • The design prioritizes efficient large-scale training, simple integration of new operations, and straightforward deployment with minimal new code.Source: §2 · p. 1

2.1. ArrayFire Tensor Library

  • ArrayFire provides multi-backend tensor execution, just-in-time kernel fusion, reduced memory use, and a concise C++ interface.Source: §2.1 · p. 1

2.2. Feature Extraction

  • Feature extraction supports multiple audio formats and feature types, computes features on the fly, and facilitates augmentation and single-binary deployment.Source: §2.2 · p. 2

2.3. Models

  • The model abstraction separates networks from criteria and supports CTC, ASG, attention-based S2S models, and diverse architectures.Source: §2.3 · p. 2

2.4. Training

  • Training supports flat-start, checkpoint continuation, and forking modes, plus standard optimizers and synchronous data-parallel SGD with NCCL2.Source: §2.4 · p. 2

2.5. Decoding

  • Decoding uses an optimized beam search incorporating language-model constraints and a word-insertion term.Source: §2.5 · p. 2

3. RELATED WORK

The paper situates wav2letter++ among major open-source ASR systems, contrasting their architectures, implementation languages, and development interfaces. It also highlights training-loop timing as a basis for performance comparison.

3. RELATED WORK

  • Kaldi, ESPNet, and OpenSeq2Seq represent major open-source ASR systems considered alongside wav2letter++.Source: §3 · p. 3

Kaldi

  • Kaldi provides stand-alone tools and supports HMM/GMM, hybrid HMM/NN, and phone-based modeling approaches.Source: §3 · p. 3

ESPNet

  • ESPNet integrates with Kaldi for preprocessing and uses Chainer or PyTorch back-ends for CTC and attention-based models.Source: §3 · p. 3

Training-loop analysis

  • Figure 3 measures the milliseconds spent in major training-loop steps, averaged per batch over a complete epoch.Source: Figure 3 · p. 3

Implementation trade-offs

  • The comparison emphasizes the trade-off between scripting-language prototyping and statically typed, object-oriented development for large-scale systems.Source: §3 · p. 3

4. EXPERIMENTS

The experiments compare wav2letter++ with major open-source ASR systems on training and decoding, using recurrent and convolutional networks and shared evaluation settings. wav2letter++ is faster in training and decoding, while some comparisons have explicit methodological limits.

Comparative setup

  • The study evaluates ESPNet, Kaldi, OpenSeq2Seq, and wav2letter++ on WSJ training time and utterance decoding latency.Training uses machines with eight NVIDIA Tesla V100 GPUs per node.Source: §4 · p. 3

Training

  • Training comparisons use 30-million-parameter recurrent and 100-million-parameter convolutional networks with 40-dimensional log-mel inputs and CTC.OpenSeq2Seq is evaluated with float32 and mixed-precision float16 training.Source: §4.1 · p. 3, Figure 4 · p. 4
  • More than 15% faster than the next-best system, wav2letter++ retains an advantage for 30-million-parameter models even on one GPU.Its advantage increases as computation scales out, including multi-node experiments at 16, 32, and 64 GPUs.Source: §4.1 · p. 4
  • More than 1.5x lower epoch time is achieved by mixed-precision OpenSeq2Seq training in some cases, an optimization wav2letter++ could also use.Kaldi’s LF-MMI setup is not directly comparable and was omitted from Figure 4.Source: §4.1 · p. 4

Decoding

  • More than an order of magnitude faster decoding and considerably lower memory usage are reported for wav2letter++ at matched decoder inputs.The comparison targets 5.0% WER and each framework’s best-obtainable WER on LibriSpeech dev-clean.Source: §4.2 · p. 4, Table 2 · p. 4

5. CONCLUSION

The conclusion presents wav2letter++ as a fast, simple, entirely C++ framework for end-to-end speech recognition. It reports promising comparative performance and positions the system as a platform for rapid research while acknowledging room for further optimization.

Conclusion

  • wav2letter++ is a fast and simple system for developing end-to-end speech recognizers, written entirely in C++.The authors connect the C++ implementation with efficient training and real-time decoding.Source: §5 · p. 4
  • Its simple and extensible interface makes wav2letter++ suited to rapid research in end-to-end speech recognition.The conclusion also leaves open further optimization of the framework and possible narrowing of the gap with Python-based systems.Source: §5 · p. 4

Related papers

Loading 1812.07625v1…