Source-linked AI summary
Trainable Frontend For Robust and Far-Field Keyword Spotting
Yuxuan Wang, Pascal Getreuer, Thad Hughes, Richard F. Lyon, Rif A. Saurous
TL;DR
Robust keyword spotting needs to handle noise, loudness variation, and distant speech while operating under embedded-device constraints. The paper introduces PCEN, an AGC-based dynamic-compression frontend, and makes its parameters jointly trainable with the acoustic model. PCEN significantly outperforms log-mel in noisy and far-field conditions, while trained PCEN provides further improvements without increasing inference-time complexity.
Problem
Keyword spotting must remain robust to noise and loudness variation for reliable hands-free interaction, while embedded deployment limits model size and resources.
Method
PCEN replaces static log or root compression with AGC-based dynamic compression, and differentiable PCEN parameters are jointly optimized with the keyword spotting acoustic model.
Results
PCEN significantly outperforms the widely used log-mel frontend in noisy and far-field conditions, while trained PCEN provides significant further improvements without increasing inference-time complexity.
Takeaways & Limitations
A computationally cheap, trainable PCEN frontend improves robust keyword spotting while preserving the inference-time complexity of the deployed model.
Takeaways & Limitations
The trainable parameters are data-independent and frozen after training, avoiding added inference cost but excluding input-conditioned parameter learning.
Abstract
from arXiv · showhide
Robust and far-field speech recognition is critical to enable true hands-free communication. In far-field conditions, signals are attenuated due to distance. To improve robustness to loudness variation, we introduce a novel frontend called per-channel energy normalization (PCEN). The key ingredient of PCEN is the use of an automatic gain control based dynamic compression to replace the widely used static (such as log or root) compression. We evaluate PCEN on the keyword spotting task. On our large rerecorded noisy and far-field eval sets, we show that PCEN significantly improves recognition performance. Furthermore, we model PCEN as neural network layers and optimize high-dimensional PCEN parameters jointly with the keyword spotting acoustic model. The trained PCEN frontend demonstrates significant further improvements without increasing model complexity or inference-time cost.
1. Introduction
Keyword spotting must remain reliable across noise and loudness variation so users can interact with devices from different distances. The paper introduces PCEN to replace problematic static compression and improve robustness to distant speech.
- Keyword spotting supports wake-word and conversational-assistant interactions, making reliable recognition across acoustic environments important.
- Current neural keyword spotters remain vulnerable to noise and loudness variation, which limits hands-free use across distances.
- On-device keyword spotters require small memory footprints and low power consumption, constraining their representation power.
- Log-mel frontends use static log compression whose offset is ad hoc, emphasizes low-level silence, and varies with loudness.
- PCEN replaces static log or root compression with differentiable automatic-gain-control-based dynamic compression and is jointly optimized with the acoustic model.
2. Per-Channel Energy Normalization
PCEN replaces static compression with causal, per-channel dynamic normalization based on smoothed filterbank energy. This design reduces irrelevant low-level variation while enhancing speech transitions and supporting trainable frontend integration.
- PCEN replaces static log or root compression with dynamic compression based on automatic gain control.
- The frontend computes filterbank energy E(t, f) and smooths it with a first-order IIR filter to obtain M(t, f).
- The ratio E(t, f)/(ϵ + M(t, f))^α normalizes recent loudness, with α controlling gain-normalization strength and s controlling smoothing.
- PCEN then applies stabilized root compression using offset δ and exponent r; α and s depend on the data's loudness distribution.
- PCEN preserves prominent speech patterns while keeping low-level, uninformative intervals relatively flat instead of amplifying them as log-mel does.
- PCEN enhances speech onsets, which the paper connects to improved noise and reverberation robustness through transition emphasis.
3. Trainable PCEN Frontend
The trainable PCEN frontend replaces manual scalar tuning with gradient-based optimization of differentiable, potentially frequency-dependent parameters. It also learns smoothing combinations while keeping parameters data-independent and inference costs unchanged.
- Manual PCEN parameter tuning is labor intensive, inherently suboptimal, and can become infeasible as parameter combinations multiply.The paper motivates automatic optimization to reduce human effort.
- Differentiable PCEN operations can be represented as neural-network layers and jointly optimized with the original KWS acoustic model.The PCEN outputs serve as features for the acoustic model during joint optimization.
- Neural-network PCEN generalizes scalar parameters to frequency- or time-frequency-dependent values optimized automatically with SGD.This extends beyond automatic parameter finding to higher-dimensional parameterizations.
- The frontend learns a frequency-dependent convex combination of outputs from several predetermined smoothers.The chosen approach learns smoother combinations rather than modeling the smoother as a recurrent neural network.
- All trainable parameters are data-independent and frozen after training, avoiding additional network weights and inference-time computational cost.The parameters are not conditioned on input features and remain unchanged across signals.
4. Experiments
Experiments compare PCEN with log-mel and evaluate trainable PCEN on noisy, far-field, and clean conditions. The results show robust gains from PCEN, including trained configurations that preserve performance while reducing inference-time complexity.
- Experimental setup: The experiments use multi-condition and multi-loudness training with a CNN keyword-spotting model and evaluate ROC curves using false rejection and false alarm rates.Multi-loudness training scales utterances from −45 dBFS to −15 dBFS, and the target operating region is no more than 0.5 false alarms per hour.
- PCEN versus log-mel: At 0.1 false alarms per hour, fixed PCEN reduces false rejection by about 14% absolute versus multi-loudness-trained log-mel.The comparison uses a rerecorded far-field car evaluation set.
- Trainable PCEN: Trained PCEN consistently outperforms fixed PCEN on rerecorded far-field evaluation sets at 0.8m and 5m talking distances.Compared with multi-loudness-trained log-mel, improvements are especially significant in low-false-alarm regions.
- Trainable PCEN: Both fixed and trained PCEN outperform log-mel on a large clean and near-field evaluation set.
- Learned smoother combination weights: Learned smoother weights alternate across frequency channels: even channels prefer s = 0.015, while odd channels prefer s = 0.08.The authors associate this pattern with neighboring-channel redundancy and feature discrimination.
- Learned smoother combination weights: A single smoother with alternating smoothing coefficients achieves performance similar to the full trained model without increasing inference-time complexity.The full architecture increases computation because it first computes multiple smoothers.
5. Discussions and Conclusions
The paper introduces PCEN as a computationally cheap, AGC-based dynamic-compression frontend that outperforms log-mel in noisy and far-field conditions. Formulating PCEN as neural-network layers enables further improvements without increasing inference-time complexity and frames signal processing components as structural regularizations.
- PCEN replaces static log or root compression with AGC-based dynamic compression and significantly outperforms log-mel in noisy and far-field conditions.The authors describe PCEN as conceptually simple, computationally cheap, and easy to implement.
- Neural-network-layer PCEN enables end-to-end training and frequency- or time-frequency-dependent parameters.The formulation supports jointly optimizing PCEN parameters with the keyword-spotting acoustic model.
- The resulting trained frontend provides significant further improvements without increasing inference-time complexity.
- Embedding signal-processing components into general-purpose neural networks presents them as structural regularizations.The authors identify this integration as a promising direction for future research.