Source-linked AI summary
Low-Power End-to-End Cochlear Implant Speech Denoising with Spiking Neural Networks
Ludovic Boulanger, Sean U. N. Wood
TL;DR
Cochlear implant users struggle with speech in noise, while DNN-based enhancement is energy-intensive for low-power CI processors. The paper proposes Spiking Deep ACE, an SNN integrating speech denoising and CI coding, and reports competitive VSTOI and SNRi with more than six times lower energy consumption than Deep ACE.
Problem
CI users struggle with speech in noisy environments, while DNN enhancement requires substantial energy for low-power CI processors.
Method
The paper proposes Spiking Deep ACE, an SNN inspired by Deep ACE that integrates speech denoising into the CI processing pipeline.
Results
Spiking Deep ACE achieves competitive VSTOI and SNRi performance with Deep ACE while consuming more than 6 times less energy.
Takeaways & Limitations
The results highlight Spiking Deep ACE’s potential for low-power CI processors.
Takeaways & Limitations
The energy evaluation excludes data-transfer costs to and from memory, which account for most energy in ANNs and SNNs.
Abstract
from arXiv · showhide
Cochlear implants (CI) restore hearing for individuals with severe to profound hearing loss. However, CI users often struggle to understand speech in noisy environments. Deep neural networks (DNN) have shown promise in enhancing speech for CI users, yet their high energy demands make them non-ideal for low-power CI processors. Spiking neural networks (SNN), on the other hand, offer comparable performance with significantly lower energy consumption. Hence, we propose a novel SNN inspired by the Deep ACE architecture that simultaneously performs speech enhancement and CI coding. Our model achieves competitive vocoded short-time objective intelligibility (VSTOI) and signal-to-noise ratio improvement (SNRi) scores compared to Deep ACE, while achieving more than a sixfold reduction in energy consumption.
1. INTRODUCTION
Cochlear implant users often struggle with speech in noise, while conventional enhancement methods have limitations and DNNs are too energy-intensive for low-power CI processors. The paper therefore proposes Spiking Deep ACE, an SNN that integrates speech denoising into the CI processing pipeline.
- CI users struggle to understand speech in noisy environments because implants lack temporal fine structure and introduce spectral smearing.
- Traditional single-channel enhancement methods perform well in stationary noise but degrade with non-stationary noise.
- DNNs enhance speech masked by non-stationary noise but require substantial energy, limiting their suitability for low-power CI processors.
- SNNs process sparse, binary, event-based information and can consume orders of magnitude less energy than equivalent DNNs on specialized neuromorphic hardware.
- Spiking Deep ACE is an SNN inspired by Deep ACE that integrates speech denoising into the CI processing pipeline.
2. SPIKING DEEP ACE
Spiking Deep ACE adapts the Deep ACE CI coding architecture with spiking neurons and simplified convolutional processing. Its design includes ParaLIF-Threshold neurons, reduced convolutional complexity, and upsampled intermediate outputs for denoising.
- Deep ACE: Deep ACE integrates speech denoising into the ACE sound-coding strategy and predicts CI stimulation patterns while maintaining 2 ms algorithmic latency.
- Architecture overview: The processing pipeline and convolutional blocks are presented for both Spiking Deep ACE and Deep ACE.
- Spiking neurons: ParaLIF separates membrane dynamics from spike emission by removing the reset mechanism, enabling parallel processing over time.
- Architecture changes: Spiking Deep ACE reduces convolutional-block complexity by removing normalization and activation functions that provide no benefit or are replaced by ParaLIF spiking nonlinearity.
- Architecture changes: Spiking Deep ACE upsamples separator and deep-envelope-detector outputs by a factor of P before convolutional decoding.
3. EXPERIMENTAL SETUP
The study evaluates Deep ACE and Spiking Deep ACE using controlled noisy-speech datasets, objective intelligibility and denoising metrics, and an operation-based energy estimate. Training, testing, and energy calculations are designed to compare the two models across noise types and signal-to-noise ratios.
- 3.1. Datasets: Approximately 10 hours of noisy speech from 28 speakers were used for training, with clean-speech counterparts at 0, 5, 10, and 15 dB SNR.Data were resampled to 16 kHz and divided into two-second segments, with speakers split into training and validation sets by sex.
- 3.1. Datasets: Testing used ICRA static Gaussian noise and ICRA babble noise mixed with clean speech at −5, 0, 5, and 10 dB SNR.The ICRA babble condition used synthesized six-speaker babble noise, while ICRA static used unmodulated random Gaussian noise.
- 3.2. Model Training: Spiking Deep ACE was hyperparameter-optimized for validation performance while maintaining a parameter count less than or equal to Deep ACE.Deep ACE used its published training hyperparameters, whereas Spiking Deep ACE accounted for additional ParaLIF parameters and neurons.
- 3.2. Model Training: Spiking Deep ACE used only MSE loss, while Deep ACE combined MSE and BCE; both models used Adam with an initial learning rate of 10^-3.A scheduler reduced the learning rate when performance failed to improve by a factor of 10^-4 for five epochs.
- 3.3. Performance Evaluation: SNRi measures electrodogram-domain denoising improvement, while VSTOI estimates intelligibility from speech re-synthesized from predicted electrodograms.SNRi compares noisy and denoised electrodogram SNR; VSTOI computes STOI against clean unprocessed speech after re-synthesis.
- 3.4. Energy Evaluation: Energy consumption was estimated by summing ANN and SNN layer costs separately across layers processing floating-point values and spikes.The estimate uses operation counts, MAC costs for ANNs, and ParaLIF-related spike and neuron-state costs for SNNs.
4. RESULTS AND DISCUSSION
Spiking Deep ACE achieves competitive denoising and intelligibility relative to Deep ACE across tested noise conditions and SNR levels, while substantially reducing energy consumption. Its sparse spiking activity also supports additional memory-related energy gains.
- Spiking Deep ACE achieves competitive SNRi across ICRA static and babble noises and tested SNR levels, outperforming Deep ACE at lower static-noise SNRs.
- Spiking Deep ACE’s mean SNRi remains within 0.5 dB of Deep ACE’s SNRi in the other tested cases.
- Spiking Deep ACE’s VSTOI remains competitive with Deep ACE across all tested SNRs and noise types, staying within 2% at every SNR level.
- More than 6 times less energy is consumed by Spiking Deep ACE while maintaining competitive performance with Deep ACE.
- At a 1 kHz sampling rate, a mean spike rate of 250 spikes / second corresponds to one spike every 4 time steps.
- Spiking Deep ACE’s intrinsic sparsity and fewer parameters require less memory transfers than Deep ACE, allowing additional energy gains.
5. CONCLUSION
The paper proposes an end-to-end spiking model for speech denoising and cochlear-implant coding. Compared with Deep ACE, it achieves competitive evaluation results while consuming considerably less energy, supporting its potential for low-power CI processors.
- The proposed Spiking Deep ACE is an SNN capable of end-to-end speech denoising and CI coding, evaluated against Deep ACE using VSTOI, SNRi, and energy consumption.