Source-linked AI summary
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
Ronan Collobert, Christian Puhrsch, Gabriel Synnaeve
TL;DR
Speech recognition traditionally relies on phonetic annotation and force alignment, while recurrent end-to-end approaches are computationally expensive. This paper combines a convolutional acoustic model, an alignment-free sequence criterion, and graph decoding, achieving competitive LibriSpeech results with MFCC features and promising results from raw speech.
Problem
Existing speech-recognition systems require phonetic annotation and force alignment, while recurrent end-to-end alternatives are computationally expensive.
Method
The system uses a convolutional acoustic model trained on graphemes with an alternative to CTC and coupled to a beam-search graph decoder.
Results
7.2% WER was achieved on LibriSpeech with MFCC features, compared with 9.4% using power spectrum and 10.1% using raw speech.
Takeaways & Limitations
The approach avoids HMM/GMM pre-training and force alignment while providing competitive LibriSpeech recognition and a sequence criterion as accurate as CTC.
Abstract
from arXiv · showhide
This paper presents a simple end-to-end model for speech recognition, combining a convolutional network based acoustic model and a graph decoding. It is trained to output letters, with transcribed speech, without the need for force alignment of phonemes. We introduce an automatic segmentation criterion for training from sequence annotation without alignment that is on par with CTC while being simpler. We show competitive results in word error rate on the Librispeech corpus with MFCC features, and promising results from raw waveform.
1 Introduction
The paper presents an end-to-end speech-recognition system that maps speech features directly to grapheme transcriptions, eliminating intermediate phonetic transcription and force alignment. It uses a simpler convolutional architecture and introduces an automatic segmentation criterion, with later experiments comparing features, criteria, and LibriSpeech word error rates.
- 1 Introduction: The system maps speech signals—including MFCCs, power spectra, or raw waveforms—directly to letter transcriptions without intermediate phonetic transcription.Training on graphemes removes the need for human or automatically generated phonetic transcriptions.
- 1 Introduction: The proposed end-to-end approach avoids the classical HMM/GMM pipeline and its required force alignment, unlike prevailing LibriSpeech systems and a recent GMM-free approach.An RNN phoneme-transcription approach [7] also moved beyond the HMM/GMM pipeline, while other end-to-end approaches remained relevant comparisons.
- 1 Introduction: The model uses convolutional networks and trains directly on graphemes, with a 23-million-parameter best model compared with 100 million parameters in [1].This is presented as a simpler architecture than sequence-criterion approaches that train from speech signals to graphemes [13].
- 1 Introduction: The paper next presents the convolutional acoustic model and automatic segmentation criterion, then evaluates features, the criterion, and LibriSpeech word error rates.The planned experiments compare different input features and report the authors’ current best word error rates.
2 Architecture
The system combines a standard 1D convolutional acoustic model with varied speech features, the ASG training criterion, and beam-search decoding. ASG simplifies CTC by removing blank labels and using globally normalized, un-normalized scores.
- System overview: The recognizer uses a standard convolutional neural network with alternative CTC training and a simple beam-search decoder.
- Input features: MFCC, power-spectrum, and raw-wave inputs cover conventional compressed, deep-learning, and minimally processed speech representations, [1], [15].
- Acoustic model: The acoustic models are standard 1D ConvNets using strided convolutions instead of pooling to expand receptive context without increasing parameter count.
- Acoustic model: Feature-specific designs omit the first convolution for MFCC and power-spectrum inputs, while the raw-wave architecture uses initial striding and final 1×1 convolutions equivalent to fully connected layers.MFCC networks require less striding, whereas power-spectrum and raw-wave networks place strided convolutions near the input because overall stride mattered more than placement.
- Auto Segmentation Criterion: ASG removes CTC blank labels, uses un-normalized node and optional transition scores, and applies global rather than per-frame normalization.Repetition character labels can represent repeated letters, while the simpler graph also facilitates decoder design and external language-model integration.
- Auto Segmentation Criterion: ASG scores correct-transcription paths against all possible letter sequences using transition models, with both terms efficiently computed and differentiated through the Forward algorithm.
3 Experiments
Experiments on LibriSpeech compare ASG with CTC, evaluate training-data scale and augmentation, and report best 1000-hour results across feature types. ASG matches CTC in LER and is faster on long sequences, while MFCCs slightly outperform power-spectrum and raw features in the final comparison.
- Experimental setup: The experiments use LibriSpeech’s standard train, validation, and test splits with about 1000 hours of audio, 16 KHz sampling, and a 30-grapheme vocabulary.Architecture and decoder hyperparameters were tuned on the validation set; reported metrics are LER or WER, with WER decoded using the authors’ decoder and LibriSpeech’s standard 4-gram language model.
- CTC vs ASG: ASG achieves the same LER as CTC and appears faster on long sequences despite running only on CPU.The comparison uses CPU ASG with SSE and OpenMP batching against Baidu’s GPU CTC implementation; timings cover forward and backward passes with 8 CPU threads.
- Training size and augmentation: Data augmentation helps with small training sets, but its benefit vanishes when sufficient training data are available, while MFCC and power-spectrum features converge in performance.The study varies training-set size from 10h to 1000h and tunes architecture size to avoid over-fitting; Figure 4 compares MFCC and power-spectrum systems, including augmentation.
- Final results: On 1000 hours of LibriSpeech, the best MFCC system slightly outperforms systems using power-spectrum or raw features in LER/WER.The final comparison uses architectures with overall stride 320, producing one label every 20 ms; results use a single extra 10 ms input shift, which improves performance by about 1%.
4 Conclusion
The paper introduces a simple end-to-end speech recognition system combining convolutional modeling, alignment-free sequence training, and beam-search decoding. It achieves competitive LibriSpeech performance with MFCCs and promising results from power-spectrum and raw-speech inputs, while AutoSegCriterion is faster than CTC and equally accurate.
- 4 Conclusion: 7.2% WER is achieved on LibriSpeech with MFCC features, compared with 9.4% using power-spectrum features and 10.1% using raw speech.These decoding results are described as competitive with MFCC features and promising for power-spectrum and raw-speech inputs.
- 4 Conclusion: The system combines a standard 1D convolutional neural network, a sequence criterion that infers segmentation, and a simple beam-search decoder.This provides an end-to-end automatic speech recognition system trained without requiring explicit segmentation inference at decoding time.
- 4 Conclusion: AutoSegCriterion is faster than CTC while achieving the same accuracy.The paper reports this comparison in table 1.