Source-linked AI summary
Jasper: An End-to-End Convolutional Neural Acoustic Model
Jason Li, Vitaly Lavrukhin, Boris Ginsburg, Ryan Leary, Oleksii Kuchaiev, Jonathan M. Cohen, Huyen Nguyen, Ravi Teja Gadde
TL;DR
End-to-end ASR seeks to reduce the complexity of conventional systems built from independently learned components. The paper presents Jasper, a deep convolutional acoustic model with NovoGrad, and reports state-of-the-art LibriSpeech results alongside competitive performance on other benchmarks. Its results support using a scalable architecture built from standard components, while performance remains weaker on harder conversational tasks.
Problem
Conventional ASR systems use several independently learned components, motivating end-to-end approaches that reduce overall system complexity.
Method
Jasper is a deep, scalable end-to-end convolutional acoustic model using 1D convolutions, batch normalization, ReLU, dropout, residual connections, and the layer-wise NovoGrad optimizer.
Results
2.95% WER on LibriSpeech test-clean is achieved using beam search with a Transformer-XL language model, while greedy decoding achieves 3.86% WER.
Takeaways & Limitations
Jasper provides an efficient baseline built from standard components for exploring deeper models, regularization, augmentation, loss functions, language models, and optimization strategies.
Takeaways & Limitations
The conversational results are good for SWB but require further improvement on harder tasks such as CHM.
Abstract
from arXiv · showhide
In this paper, we report state-of-the-art results on LibriSpeech among end-to-end speech recognition models without any external training data. Our model, Jasper, uses only 1D convolutions, batch normalization, ReLU, dropout, and residual connections. To improve training, we further introduce a new layer-wise optimizer called NovoGrad. Through experiments, we demonstrate that the proposed deep architecture performs as well or better than more complex choices. Our deepest Jasper variant uses 54 convolutional layers. With this architecture, we achieve 2.95% WER using a beam-search decoder with an external neural language model and 3.86% WER with a greedy decoder on LibriSpeech test-clean. We also report competitive results on the Wall Street Journal and the Hub5'00 conversational evaluation datasets.
1. Introduction
Jasper is a deep, computationally efficient end-to-end convolutional acoustic model designed to reduce ASR system complexity and match or outperform non-end-to-end systems. The paper combines streamlined architecture choices, residual connections, and NovoGrad, achieving state-of-the-art LibriSpeech results.
- The architecture uses 1D convolutions with ReLU, batch normalization, dropout, and residual connections, using operators optimized for GPU training and inference.
- The largest Jasper version uses 54 convolutional layers and 333M parameters, while the smaller version uses 34 layers and 201M parameters.
- 2.95% WER on LibriSpeech test-clean is achieved with the best acoustic model integrated with a Transformer-XL language model.
- Jasper is a computationally efficient end-to-end convolutional neural network acoustic model.
- ReLU and batch normalization outperform tested alternatives, while residual connections are necessary for deeper models to converge.
- NovoGrad is introduced as an Adam variant with a smaller memory footprint.
2. Jasper Architecture
Jasper is a deep, scalable end-to-end speech-recognition architecture built from convolutional sub-blocks, residual connections, and selected normalization and activation components. The section also describes Dense Residual connections, language-model decoding, and NovoGrad optimization.
- Jasper Architecture: Jasper converts mel-filterbank features into per-frame character probabilities using stacked 1D convolutions, batch normalization, ReLU, and dropout.Its BxR structure contains B blocks with R sub-blocks, each using the same sequence of operations.
- Residual Connections: Dense Residual adds each convolution block’s output to the inputs of all following blocks, using addition rather than concatenation.This topology is presented as a Jasper variant inspired by DenseNet and DenseRNet.
- Normalization and Activation: Batch normalization with ReLU outperformed the other tested normalization and activation choices on larger Jasper models.The experiments compared batch, weight, and layer normalization with ReLU, clipped ReLU, leaky ReLU, GLU, and GAU variants.
- Residual Connections: Residual connections were consistently necessary for deeper-than-Jasper-5x3 models to converge.The study compared simple, dense, DenseNet, and DenseRNet residual variants using addition or concatenation.
- NovoGrad: NovoGrad computes second moments per layer rather than per weight, reducing memory consumption and improving numerical stability relative to Adam.Replacing momentum SGD with NovoGrad reduced dev-clean WER from 4.00% to 3.64% for Jasper DR 10x5, a relative improvement of 9%.
3. Results
Jasper was evaluated on read and conversational speech benchmarks, achieving state-of-the-art results on LibriSpeech and competitive performance elsewhere. The conversational evaluation showed good SWB results but remaining room for improvement on harder CHM speech.
- Evaluation setup: 3-fold speed perturbation with fixed +/-10% was used for LibriSpeech training, while WSJ and Hub5’00 used random speed factors between [-10%, 10%].All models used dropout and weight decay as regularization.
- LibriSpeech: Jasper DR 10x5 achieved state-of-the-art performance on LibriSpeech test-clean and among end-to-end models on test-other.The model used NovoGrad and was trained for 400 epochs.
- Wall Street Journal: Jasper 10x3 was trained for 400 epochs with SGD momentum on an 80-hour combined WSJ0 and WSJ1 dataset, with results reported in Table 6.
- Conversational speech: Hub5’00 evaluation covered Switchboard and Callhome using models trained on 2000 hours of Fisher+Switchboard data.Jasper DR 10x5 used SGD with momentum for 50 epochs, and results were reported in Table 7.
- Conversational speech: Good results were obtained for SWB, while WER on the harder CHM task still requires improvement.
4. Conclusions
Jasper is presented as a deep, scalable end-to-end speech-recognition architecture built from standard components and designed for efficient training and inference. The authors position it as a baseline for exploring more sophisticated modeling strategies and larger-scale systems.
- Jasper combines a deep convolutional architecture with residual topology, regularization, and a strong optimizer to achieve state-of-the-art LibriSpeech results.
- The architecture is described as highly efficient for both training and inference.
- Jasper provides a baseline for exploring more sophisticated regularization, data augmentation, loss functions, language models, and optimization strategies.
- The authors identify continued scaling to deeper models and larger datasets as an open direction.