Hello Edge: Keyword Spotting on Microcontrollers

Yundong Zhang; Naveen Suda; Liangzhen Lai; Vikas Chandra

Source-linked AI summary

Hello Edge: Keyword Spotting on Microcontrollers

Yundong Zhang, Naveen Suda, Liangzhen Lai, Vikas Chandra

arXiv:1711.07128v3Nov 20, 2017cs.SDcs.CLcs.LGcs.NEeess.AS

TL;DR

Always-on keyword spotting must deliver accurate, low-latency speech interaction within the tight memory, compute, and energy limits of microcontrollers. The paper evaluates existing architectures and introduces a depthwise-separable-convolution model, finding that it outperforms prior models across accuracy, model size, and operations while supporting microcontroller deployment.

Problem
Always-on keyword spotting requires high accuracy and low latency but must operate within microcontrollers' limited memory, compute, and energy budgets.
Method
The authors train published KWS architectures on the Google speech commands dataset, compare accuracy, memory footprint, and inference operations, and implement a depthwise-separable-convolution model.
Results
The depthwise-separable-convolution model outperforms prior models in accuracy, model size, and operations, while representative models can be quantized to 8-bit without accuracy loss.
Takeaways & Limitations
Resource-constrained architecture exploration can identify KWS models suitable for microcontrollers without sacrificing accuracy.
Takeaways & Limitations
Accuracy comparisons are not directly fair because literature models were optimized for different datasets and use different memory and compute resources.

Abstract

from arXiv · show

Keyword spotting (KWS) is a critical component for enabling speech based user interactions on smart devices. It requires real-time response and high accuracy for good user experience. Recently, neural networks have become an attractive choice for KWS architecture because of their superior accuracy compared to traditional speech processing algorithms. Due to its always-on nature, KWS application has highly constrained power budget and typically runs on tiny microcontrollers with limited memory and compute capability. The design of neural network architecture for KWS must consider these constraints. In this work, we perform neural network architecture evaluation and exploration for running KWS on resource-constrained microcontrollers. We train various neural network architectures for keyword spotting published in literature to compare their accuracy and memory/compute requirements. We show that it is possible to optimize these neural network architectures to fit within the memory and compute constraints of microcontrollers without sacrificing accuracy. We further explore the depthwise separable convolutional neural network (DS-CNN) and compare it against other neural network architectures. DS-CNN achieves an accuracy of 95.4%, which is ~10% higher than the DNN model with similar number of parameters.

1 Introduction

Keyword spotting enables speech interaction while avoiding the latency, energy, network, and privacy costs of continuously sending audio to the cloud. This work explores neural-network architectures that meet microcontroller memory, compute, accuracy, and latency constraints.

Motivation: Always-on speech recognition can increase energy use, network congestion, latency, and privacy concerns when audio is continuously transmitted to the cloud.Keyword spotting detects predefined words before activating full-scale speech recognition on-device or in the cloud.
Microcontroller constraints: Microcontrollers offer low-power deployment for always-on KWS but typically provide only tens to a few hundred KB of memory.The complete model, including inputs, outputs, weights, and activations, must fit within this budget.
Microcontroller constraints: Real-time KWS limits the total number of neural-network operations per inference because the system must remain always on.
Research focus: The work evaluates lean neural-network structures under microcontroller resource constraints while targeting high accuracy and low latency.
Evaluation: The authors train literature KWS models on the Google Speech Commands dataset and compare accuracy, memory footprint, and operations per inference.
Model exploration: A depthwise-separable and pointwise-convolution KWS model outperforms prior models in accuracy, model size, and operations.The model is inspired by MobileNet’s resource-efficient architecture.
Study scope: The study presents a comprehensive comparison of architectures within typical microcontroller compute and memory constraints.Code, model definitions, and pretrained models are made available.

2 Background

KWS combines speech-feature extraction with neural classification, but prior models are difficult to compare fairly and may overlook hardware compute limits. Microcontroller deployment is constrained by both on-chip memory and neural-network execution throughput.

Keyword spotting system: A typical KWS system extracts speech features from framed audio and passes them to a neural-network classifier.The pipeline uses overlapping frames and can apply posterior handling to average class probabilities over time.
Prior approaches: Prior KWS approaches include HMMs, discriminative models, RNNs, and DNNs, with differing accuracy, training difficulty, compute cost, and detection latency.
Comparison gap: Prior neural-network models are difficult to compare fairly because they use different datasets, speech features, and audio durations.Earlier work also emphasized accuracy and memory footprint without explicitly constraining operations per inference.
Microcontroller systems: Microcontrollers combine a processor core with on-chip SRAM and embedded flash, so deployed programs and data must fit within limited on-chip memory.
Microcontroller systems: Operations per second constrain neural-network deployment because low-cost, energy-efficient microcontrollers lack high throughput for compute-intensive workloads.Some Cortex-M4 and Cortex-M7 devices provide SIMD and MAC instructions for low-precision neural-network computation.

3 Neural Network Architectures for KWS

The paper evaluates DNN, CNN, RNN, CRNN, and DS-CNN architectures for keyword spotting, emphasizing how each models speech-feature structure. DS-CNN uses efficient convolutions to reduce parameters and operations while enabling deeper and wider models on microcontrollers.

3.1 Deep Neural Network (DNN): DNNs stack fully connected and nonlinear activation layers, mapping flattened feature matrices to keyword probabilities.The output uses a linear layer followed by softmax for the k keywords.
3.2 Convolutional Neural Network (CNN): CNNs model local temporal and spectral correlations by treating speech features as images and applying two-dimensional convolutions.Convolution layers may be followed by normalization, ReLU activations, and pooling to reduce feature dimensionality.
3.3 Recurrent Neural Network (RNN): RNNs process spectral feature vectors across T time steps and capture temporal relations and long-term dependencies through gating.At each step, the current spectral vector is combined with the previous output.
3.4 Convolutional Recurrent Neural Network (CRNN): CRNNs combine convolutional layers for local temporal and spatial correlations with recurrent layers for global temporal dependencies.The architecture starts with convolution, uses a bidirectional recurrent layer, and employs GRU cells because they use fewer parameters than LSTMs and converged better in the experiments.
3.5 Depthwise Separable Convolutional Neural Network (DS-CNN): DS-CNN decomposes standard three-dimensional convolutions into depthwise two-dimensional and pointwise 1x1 convolutions, reducing parameters and operations.Average pooling followed by a fully connected layer provides global interaction and reduces parameters in the final layer.

4 Experiments and Results

The experiments compare KWS architectures under microcontroller memory and compute constraints, using standardized speech-command evaluation and resource accounting. DS-CNN provides the strongest accuracy-resource trade-off and remains scalable to highly constrained devices.

Training setup: 65K one-second clips from the Google Speech Commands dataset were classified into 12 output classes using an 80:10:10 train-validation-test split.The inputs comprise 10 target keywords plus silence and unknown-word classes.
Training setup: 40 MFCC features per 40ms frame with 20ms stride produced 1960 features per second, while memory used 8-bit weights and activations.Activation memory was estimated using reuse across layers, taking the maximum of two consecutive layers.
Architecture comparisons: DNNs used fewer operations but were memory-intensive, whereas CNNs achieved higher accuracy at the cost of larger operation counts and/or memory requirements.LSTMs and CRNNs balanced memory, operations, and accuracy more effectively than these extremes.
Resource-constrained evaluation: Most prior models exceeded the small 80KB/6M-operations resource box; CNN-2, CRNN, and DNN fit medium or large boxes but had lower accuracy than CNN-1.CNN-1 did not fit any defined resource box, while LSTM was the exception among the compared models.
Resource-constrained evaluation: Exhaustive feature and model hyperparameter search showed that DS-CNN achieved the best accuracies and good scalability across memory and compute budgets.The study explored feature extraction and architecture parameters before manually narrowing the search space.
Resource-constrained evaluation: Scaled-down DS-CNN models exceeded similarly operational DNNs in accuracy while reducing memory requirements by more than 10×, reaching footprints below 8KB.Figure 7 targets devices with fewer than 500K operations and less than 8KB memory.

5 Conclusions

The study evaluates KWS architectures under microcontroller memory and compute constraints, finding DS-CNN consistently achieves the best accuracy across three constraint sets.

DS-CNN achieves the best accuracy across all three memory/compute constraint sets.The reported accuracies are 94.4%, 94.9% and 95.4%.
The architecture exploration targets maximum accuracy within resource constraints typical of microcontroller systems.The study derives three memory/compute constraint sets and searches for the best networks under each.
The evaluated architectures are compared using accuracy, memory requirements and operations per inference.

A Appendix: Neural Network Hyperparameters

The appendix summarizes the best models’ hyperparameters and reports how the architecture searches relate accuracy, operations, and model size.

Neural Network Hyperparameters: Table 7 summarizes the hyperparameters of the best neural networks described in Table 5.It also includes memory, operation counts, and training, validation, and test accuracy.
Neural Network Hyperparameters: All models use 10 MFCC features with a 40 ms frame length.The table reports the frame stride separately for each model.
Neural Network Hyperparameters: Figures 9(a)–9(d) plot model accuracy against operations for DNN, basic LSTM, LSTM, and CRNN architectures.Circle size represents the number of model parameters.