Source-linked AI summary
MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition
Somshubra Majumdar, Boris Ginsburg
TL;DR
Speech-command recognition on resource-limited devices calls for a compact model that remains accurate and robust to background noise. MatchboxNet uses residual 1D time-channel separable convolutions and noise augmentation; it reports state-of-the-art accuracy with fewer parameters and scalability across model sizes.
Problem
Speech-command recognition is targeted at devices with limited computational and memory resources, while continuous audio also requires distinguishing commands from background speech or noise.
Source: §1 · p. 1, §5.2 · p. 3Method
Results
MatchboxNet achieves state-of-the-art accuracy on Google Speech Commands v1 and close-to-state-of-the-art accuracy on v2 while using significantly fewer parameters than other models.
Source: §1 · p. 1, §4.2 · p. 2Takeaways & Limitations
The model scales across memory and compute capabilities, and intensive auxiliary-noise augmentation can make it robust to background noise.
Source: §1 · p. 1, §6 · p. 4Training rebalanced classes by duplicating random samples; the paper notes class-weighted cross-entropy as an alternative.
Source: §4.1 · p. 2
Abstract
from arXiv · showhide
We present an MatchboxNet - an end-to-end neural network for speech command recognition. MatchboxNet is a deep residual network composed from blocks of 1D time-channel separable convolution, batch-normalization, ReLU and dropout layers. MatchboxNet reaches state-of-the-art accuracy on the Google Speech Commands dataset while having significantly fewer parameters than similar models. The small footprint of MatchboxNet makes it an attractive candidate for devices with limited computational resources. The model is highly scalable, so model accuracy can be improved with modest additional memory and compute. Finally, we show how intensive data augmentation using an auxiliary noise dataset improves robustness in the presence of background noise.
1. Introduction
MatchboxNet is an end-to-end speech-command model designed for devices with limited compute and memory. It combines residual blocks with 1D time-channel separable convolutions and is reported to achieve strong accuracy with fewer parameters, scale with model size, and gain noise robustness through augmentation.
1. Introduction
- MatchboxNet is an end-to-end speech-command model built from residual blocks of 1D time-channel separable convolutions, batch normalization, ReLU, and dropout.The architecture builds on QuartzNet [1] and uses residual connections [2].Source: §1 · p. 1
- MatchboxNet achieves state-of-the-art accuracy on Google Speech Commands while requiring significantly fewer parameters than similarly accurate models [5].Source: §1 · p. 1
- The model scales well as its parameter count increases.Source: §1 · p. 1
- The paper presents a methodology for improving robustness to background speech and noise.Source: §1 · p. 1
2. Related Work
Related work traces speech recognition from TDNN and hybrid neural-network systems toward end-to-end keyword spotting. The Google Speech Command dataset accelerated work on diverse neural architectures for keyword spotting.
2. Related Work
- TDNNs were used for isolated-word recognition, while TDNNs and RNNs also served as phonetic classifiers in hybrid systems with HMMs.Source: §2 · p. 1
- Rapid progress in deep-learning ASR triggered end-to-end neural-network KWS research, including a small-footprint CNN, ResNets applied to ASR, and a convolutional-RNN [16].Source: §2 · p. 1
- The Google Speech Command dataset [5] accelerated keyword-spotting research, with models including deep residual networks [18], weight-shared RNNs, an attention-based RNN-Transducer, and dilated, gated CNNs.Source: §2 · p. 1
3. MatchboxNet Architecture
MatchboxNet adapts QuartzNet-style 1D time-channel separable convolutions for a compact residual architecture. Its BxRxC notation specifies the number of blocks, sub-blocks per block, and channels.
3. MatchboxNet Architecture
- MatchboxNet builds on QuartzNet [1] and uses 1D time-channel separable convolutions to reduce model size versus regular 1D convolutions.Source: §3 · p. 1
- In a BxRxC model, B is the number of residual blocks, R the sub-blocks per block, and C the shared output channels within each block.Source: §3 · pp. 1–2
- A basic sub-block combines a 1D time-channel separable convolution, 1x1 pointwise convolutions, batch normalization, ReLU, and dropout.Source: §3 · p. 2
- MatchboxNet-3x2x64 has three blocks with two sub-blocks each and 64 channels, plus Conv1 before the blocks and Conv2–Conv4 before softmax.Source: Table 1 · p. 2, §3 · p. 2
4. Experiments
MatchboxNet is evaluated on the Google Speech Commands datasets, with class-balanced training and audio augmentation. The experiments report state-of-the-art v1 accuracy, near-state-of-the-art v2 accuracy with fewer parameters, and improved accuracy as model size increases.
4. Experiments
- The experiments use Google Speech Commands v1 with 65,000 one-second utterances across 30 classes and v2 with 105,000 utterances across 35 classes, rebalanced by duplicating random samples.Source: §4 · p. 2
4.1. Training Methodology
- Inputs are 64 MFCCs padded to 128 frames, with time shifts, white noise, SpecAugment, and SpecCutout applied during training.Source: §4.1 · p. 2
- Training uses NovoGrad and a warmup-hold-decay schedule, runs for 200 epochs on two V100 GPUs, and notes class-weighted cross-entropy as an alternative to duplication-based rebalancing.Source: §4.1 · p. 2
4.2. Results
- MatchboxNet-3x1x64 and MatchboxNet-3x2x64 achieve state-of-the-art accuracy on Google Speech Commands v1 and near-state-of-the-art accuracy on v2 with significantly fewer parameters than other models.The reported comparisons are against published results; Tables 2 and 3 cover v1 and v2, respectively.Source: §4.2 · p. 2, Table 2 · p. 2, Table 3 · p. 2
4.3. Model Scaling
- On Speech Commands v2, increasing either depth B×R or channel count C raises accuracy with model size until approximately 97.6%.MatchboxNet-3x2x64 is the scaling baseline; Table 4 reports the scaling study.Source: §4.3 · p. 3, Table 4 · p. 3
5. Model Robustness to Noise
MatchboxNet is trained with background noise augmentation to improve robustness, including in continuous audio with background speech and noise. The paper also finds that a larger model consistently outperforms a smaller one under noisy test conditions.
5. Model Robustness to Noise
- The noise-augmentation training uses about 55,000 one-second noise segments, with randomly sampled segments added at SNRs from 0 to 50 dB.When a noise segment is shorter than the training sample, a sub-segment of the sample is randomly augmented.Source: §5 · p. 3, §5.1 · p. 3
5.1. Training with Noise Augmentation
- On clean data, MatchboxNet-3x1x64 has accuracy similar to the baseline trained with basic augmentation alone.Table 5 reports accuracy averaged over five trials with 95% confidence intervals.Source: §5.1 · p. 3, Table 5 · p. 3
- Background noise augmentation makes MatchboxNet significantly more robust, even when test noise has much higher amplitude than the training noise.The comparison evaluates accuracy across signal-to-noise ratios.Source: Figure 2 · p. 3, §5.1 · p. 3
5.2. Speech Commands Recognition with Background Speech and Noise Detection
- Adding background speech and noise samples and two background classes significantly improves noise robustness for continuous-audio command recognition.The training set adds roughly 3,500 environmental-noise samples and a similar number of background-speech samples.Source: Figure 3 · p. 3, Table 6 · p. 3, §5.2 · p. 3
5.3. Robustness To Noise With Model Scaling
- MatchboxNet-6x2x64 consistently outperforms MatchboxNet-3x1x64 on unseen test samples perturbed by random noise.The evaluation uses 10 random noise samples per test sample.Source: Figure 4 · p. 4, Table 7 · p. 4, §5.3 · p. 4
6. Conclusions
MatchboxNet is an end-to-end residual network for efficient speech-command recognition on resource-limited devices, combining few parameters with strong accuracy and noise robustness.
6. Conclusions
- MatchboxNet achieves state-of-the-art accuracy on Google Speech Commands v1 with significantly fewer parameters than similarly accurate models, and noise augmentation makes it highly robust to background noise.The model is scalable for devices with different memory and compute capabilities.Source: §6 · p. 4