Source-linked AI summary
Improving noise robust automatic speech recognition with single-channel time-domain enhancement network
Keisuke Kinoshita, Tsubasa Ochiai, Marc Delcroix, Tomohiro Nakatani
TL;DR
Distant-microphone ASR remains difficult, and the ASR benefit of high-performing time-domain enhancement had not been established. This paper adapts Conv-TasNet for single-channel denoising and reports more than 30% relative WER reduction on real CHiME-4 recordings with a strong MCT-trained back-end.
Problem
Whether the high enhancement performance of time-domain approaches can translate into ASR gains remained insufficiently investigated, while many single-channel enhancement methods had brought limited ASR gains.
Source: §1 · p. 1Method
The paper adapts the Conv-TasNet time-domain network for single-channel noise reduction as Denoising-TasNet.
Source: §1 · p. 1Results
More than 30% relative WER reduction was reported on real CHiME-4 recordings with a strong ASR back-end trained on MCT data.
Source: §1 · p. 1Takeaways & Limitations
The findings show that single-channel noise reduction can still improve ASR performance.
Source: §1 · p. 1The background-noise setting excludes obvious interfering speakers.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
With the advent of deep learning, research on noise-robust automatic speech recognition (ASR) has progressed rapidly. However, ASR performance in noisy conditions of single-channel systems remains unsatisfactory. Indeed, most single-channel speech enhancement (SE) methods (denoising) have brought only limited performance gains over state-of-the-art ASR back-end trained on multi-condition training data. Recently, there has been much research on neural network-based SE methods working in the time-domain showing levels of performance never attained before. However, it has not been established whether the high enhancement performance achieved by such time-domain approaches could be translated into ASR. In this paper, we show that a single-channel time-domain denoising approach can significantly improve ASR performance, providing more than 30 % relative word error reduction over a strong ASR back-end on the real evaluation data of the single-channel track of the CHiME-4 dataset. These positive results demonstrate that single-channel noise reduction can still improve ASR performance, which should open the door to more research in that direction.
1. INTRODUCTION
Distant-microphone ASR remains challenging, and single-channel enhancement has often failed to improve recognition despite better enhancement metrics. This study tests whether time-domain denoising can improve ASR, adapting Conv-TasNet for the task.
1. INTRODUCTION
- Single-channel ASR remains far behind microphone-array performance, motivating effective single-channel enhancement front-ends.Multi-channel enhancement with an MCT-trained back-end can greatly improve ASR, but many settings have only one microphone.Source: §1 · p. 1
- Frequency-domain enhancement can improve SDR without improving ASR, consistent with distortions that mismatch the ASR back-end.Earlier masking and conventional single-channel techniques did not improve recognition with a state-of-the-art back-end.Source: §1 · p. 1
- The study adapts Conv-TasNet into Denoising-TasNet, comparing enhanced-speech-only output with speech-and-noise outputs for multi-task training.The figure depicts the investigated denoising networks and marks optional branches for multi-task training.Source: §1 · p. 1, Figure 1 · p. 2
- On real CHiME-4 recordings, Denoising-TasNet achieved more than 30% relative WER reduction with a strong ASR back-end trained on MCT data.The paper also reports improvement without ASR-back-end retraining, gains from enlarging SE training data for both Denoising-TasNet and frequency-domain BLSTM masking, and generalization to AURORA-4.Source: §1 · p. 1
2. DENOISING NETWORK
The network recovers target speech from a single-channel noisy signal, with background noise defined to exclude obvious interfering speakers. Its denoising pipeline encodes the microphone signal, estimates masks, and decodes enhanced speech, using time- or frequency-domain components and several training-loss options.
2.1. Problem formulation
- The task is to recover target speech from a single-channel microphone signal modeled as speech plus background noise, excluding obvious interfering speakers.Source: §2.1 · p. 2
2.2. Time-domain and frequency-domain denoising networks
- Denoising encodes the microphone signal, estimates speech or speech-and-noise masks, then decodes the masked representation into enhanced speech.Time-domain encoding uses a learnable 1D convolution and ReLU; decoding uses a learnable 1D deconvolution, while frequency-domain encoding and decoding use STFT and inverse STFT.Source: §2.2 · p. 2
2.3. Training losses
- The time-domain loss is negative SNR between clean and enhanced speech; preserving signal scale avoids a scaling requirement when passing enhanced speech to ASR.The authors also report slightly better preliminary speech-enhancement performance with SNR than with SiSNR.Source: §2.3 · p. 2
- Frequency-domain loss uses log-MSE on the amplitude spectrum before inverse STFT, while a multi-task loss can add noise reconstruction to the speech loss.The authors suggest predicting noise may act as a regularizer for training more robust models; fixed speech/noise output order avoids permutation-invariant training.Source: §2.3 · pp. 2–3
3. RELATED WORKS
Prior single-channel enhancement approaches had mixed ASR results: some did not improve recognition alone, while other methods reported gains on CHiME-4. This paper focuses on comparing frequency- and time-domain enhancement, with tighter integration left for future work.
3. RELATED WORKS
- GAN-based single-channel enhancement improved objective and subjective enhancement criteria but did not improve ASR in the cited work.Source: §3 · p. 3
- Legacy enhancement front ends did not improve ASR individually, though a multi-branch architecture reduced CHiME-4 eval-real WER from 22.6% to 19.4% in an online system.The authors note that this study's baseline was much worse than theirs.Source: §3 · p. 3
- A parametric Wiener filter front end with acoustically optimized hyperparameters reduced CHiME-4 eval-real WER from 11.6% to 10.6% with a strong ASR back end.Source: §3 · p. 3
- The study compares frequency- and time-domain enhancement, while tighter integration of time-domain enhancement with ASR is identified as future work.Source: §3 · p. 3
4. EXPERIMENTS
Experiments compare single-channel enhancement systems on CHiME-4 and test data augmentation and generalization on Aurora-4. Denoising-TasNet improves ASR on real CHiME-4 recordings, while the results also show that SDR gains alone do not ensure WER gains.
4. EXPERIMENTS
- The experiments assessed SDR and WER on 16-kHz speech, using CHiME-4 evaluation recordings from four environments at an average SNR of about 5 dB.The evaluation set contains 1,320 simulated and 1,320 real noisy recordings.Source: §4 · p. 3
4.1. SE systems: network configurations and training data
- The SE training set was synthesized with reverberation and CHiME-4 noise, producing 35,690 utterances with randomly selected SNRs from 0 to 5 dB; the ASR back-end used Org-AM or Enh-AM configurations.Enh-AM training included enhanced versions of the 35k set, while Org-AM did not.Source: §4.1 · p. 3, §4.2 · p. 3
4.3.1. Effect of NR-loss
- NR-loss did not significantly improve SDR but consistently improved ASR across tested SE front ends, motivating its use in subsequent experiments.Table 1 reports the NR-loss comparison for two representative front ends.Source: §4.3.1 · p. 4, Table 1 · p. 4
4.3.2. Effect of different SE systems
- FD-BLSTM-FDL improved SDR but not WER over unprocessed noisy input; FD-BLSTM-TDL and FD-Conv-FDL further improved SDR while ASR performance dropped significantly.With Enh-AM, FD-BLSTM-FDL achieved up to 10% relative WER reduction.Source: §4.3.2 · p. 4, Table 2 · p. 4
- More than 30% relative WER reduction on real CHiME-4 recordings was achieved by Denoising-TasNet with Enh-AM, and it also improved ASR with Org-AM without back-end retraining.Denoising-TasNet achieved the highest SDR among the compared systems.Source: §4.3.2 · p. 4, Table 2 · p. 4
4.3.3. Effect of data augmentation on SE systems
- Training SE systems on 100k rather than 35k utterances significantly improved ASR performance while having little impact on SDR; adding the same 100k set to AM training did not improve baseline ASR.The Denoising-TasNet advantage over FD-BLSTM remained, although FD-BLSTM also improved ASR with augmented training data.Source: §4.3.3 · p. 4, Table 3 · p. 4
4.3.4. Generalization to a different task
- Both FD-BLSTM and Denoising-TasNet generalized to Aurora-4, where Denoising-TasNet remained superior and its set-A degradation was not significant.The Aurora-4 test used systems trained on the 100k set.Source: §4.3.4 · p. 4, Table 4 · p. 4
5. CONCLUSION
Experiments across frequency- and time-domain denoising networks found that Denoising-TasNet improved ASR by more than 30% on real recordings with a strong back-end. The results also indicate that single-channel denoising can improve ASR when the enhancement systems are trained on a larger dataset.
5. CONCLUSION
- More than 30% ASR improvement was achieved by Denoising-TasNet on real recordings with a strong back-end.The conclusion describes the improvement as significant but does not specify the metric for this figure.Source: §5 · p. 4
- With the 100k training set, both Denoising-TasNet and FD-BLSTM significantly improved ASR performance.Source: §5 · p. 4
- The results show that single-channel denoising still has potential to improve ASR noise robustness.Source: §5 · p. 4