PaperlayerView exact revision on arXiv

Source-linked AI summary

rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method

Zheng-Hua Tan, Achintya kr. Sarkar, Najim Dehak

arXiv:1906.03588v2cs.SDcs.CLcs.LGeess.AS

TL;DR

Robust VAD must generalize across clean speech and diverse, changing noise without relying on labelled data. The paper proposes rVAD, a two-pass, pitch-anchored segment method, and reports strong performance across VAD and speaker-verification tasks, while rVAD-fast trades moderate performance loss for much lower computation.

  • Problem

    Accurate unsupervised VAD that generalizes across clean, noisy, stationary, rapidly changing, and unseen conditions remains an open problem.

  • Method

    rVAD combines two-pass denoising, pitch-anchored extended segments, and posteriori SNR weighted energy decisions; rVAD-fast substitutes spectral flatness for pitch extraction.

  • Results

    rVAD performs well across clean and noisy conditions for both VAD and speaker verification, with generalization across databases, noise conditions, and tasks.

  • Takeaways & Limitations

    rVAD-fast is approximately 10 times faster than rVAD with moderate VAD degradation, supporting large-scale processing and resource-constrained devices.

  • Takeaways & Limitations

    rVAD-fast breaks down under white noise, while future work includes optimizing rVAD configurations and studying automatic speech recognition.

Abstract

from arXiv · show

This paper presents an unsupervised segment-based method for robust voice activity detection (rVAD). The method consists of two passes of denoising followed by a voice activity detection (VAD) stage. In the first pass, high-energy segments in a speech signal are detected by using a posteriori signal-to-noise ratio (SNR) weighted energy difference and if no pitch is detected within a segment, the segment is considered as a high-energy noise segment and set to zero. In the second pass, the speech signal is denoised by a speech enhancement method, for which several methods are explored. Next, neighbouring frames with pitch are grouped together to form pitch segments, and based on speech statistics, the pitch segments are further extended from both ends in order to include both voiced and unvoiced sounds and likely non-speech parts as well. In the end, a posteriori SNR weighted energy difference is applied to the extended pitch segments of the denoised speech signal for detecting voice activity. We evaluate the VAD performance of the proposed method using two databases, RATS and Aurora-2, which contain a large variety of noise conditions. The rVAD method is further evaluated, in terms of speaker verification performance, on the RedDots 2016 challenge database and its noise-corrupted versions. Experiment results show that rVAD is compared favourably with a number of existing methods. In addition, we present a modified version of rVAD where computationally intensive pitch extraction is replaced by computationally efficient spectral flatness calculation. The modified version significantly reduces the computational complexity at the cost of moderately inferior VAD performance, which is an advantage when processing a large amount of data and running on low resource devices. The source code of rVAD is made publicly available.

1. Introduction

The paper targets unsupervised VAD that remains accurate across clean, noisy, and unseen conditions. It introduces rVAD and rVAD-fast, then evaluates them across VAD and speaker-verification tasks.

  • Motivation: VAD supports speech systems by detecting speech presence and removing non-speech segments before downstream processing.Removing noise-only segments can reduce recognition error rates and computational cost.
  • Limitations of existing methods: Supervised VAD depends strongly on labelled-data quality and train–test matching, limiting robustness under mismatched conditions.The cited RATS example reports nearly an order-of-magnitude DCF gap between unseen and seen environments.
  • Research gap: Accurate VAD across clean speech, noisy speech, stationary noise, rapidly changing noise, and unseen environments remains unresolved.Existing methods described in the paper often perform well in only particular noise or cleanliness conditions.
  • Proposed approach: rVAD combines two-stage denoising, pitch or spectral-flatness detection, and segment-based decisions to improve robustness without supervision.Pitch serves as an anchor for locating speech regions, while a posteriori SNR weighted energy distance determines activity boundaries.
  • Computational variant: rVAD-fast replaces pitch extraction with spectral-flatness detection and is approximately 10 times faster, with moderate VAD degradation.The paper positions this trade-off for large-scale processing and computationally constrained devices.
  • Evaluation: Experiments evaluate rVAD on noisy VAD databases and speaker-verification data, with results comparing favourably against existing methods.The evaluations include RATS, Aurora-2, and noise-corrupted RedDots data.

2. Robust VAD in noise

The method addresses speech signals containing both stationary and burst-like noise, which require different processing strategies. It therefore uses two-pass denoising and pitch-anchored segment processing before VAD decisions.

  • Noise challenge: Real-world speech may contain both stationary noise and high-energy burst-like noise that conventional denoising struggles to estimate and remove.Examples from RATS illustrate the persistence of high-energy noise after denoising.
  • Two-pass denoising: The first denoising pass detects high-energy segments with a posteriori SNR weighted energy difference and removes segments lacking pitch.This targets high-energy non-speech that can distort later noise estimation.
  • Two-pass denoising: The second pass applies a general speech-enhancement method to remove remaining relatively stationary noise.The paper considers enhancement methods based on MMSE, MSNE, and a modified MSNE approach.
  • Pitch-anchored VAD: rVAD first identifies extended pitch segments, then performs segment-based VAD within them.Pitch provides a speech-presence anchor and excludes substantial potentially noisy non-speech regions before thresholding.

3. rVAD: an unsupervised segment-based VAD method

rVAD combines first-pass removal of high-energy noise, second-pass speech enhancement, pitch-segment extension, and SNR-weighted energy decisions. Its design uses pitch to localize candidate speech regions while making final VAD decisions from denoised signal energy.

  • Pipeline: The rVAD pipeline contains first-pass denoising, second-pass denoising, extended pitch-segment detection, and final VAD.The first pass detects and removes high-energy noise segments; later stages operate on the resulting signal.
  • First-pass denoising: Consecutive high-energy frames form segments, and segments containing no more than two pitch frames are classified as noise and zeroed.This pass prevents burst-like noise from inflating later noise estimates and removes difficult high-energy non-speech.
  • Second-pass denoising: The second pass can use MMSE, MSNE, or MSNE-mod spectral-subtraction enhancement methods.MSNE-mod also freezes its noise estimate during detected high-energy noise segments and suppresses selected low-frequency bins.
  • Final VAD: Pitch frames are grouped into segments and extended by 60 frames at both ends to include voiced, unvoiced, and possible non-speech frames.The extended segments provide the domain for subsequent SNR-weighted energy VAD decisions and post-processing.
  • Final VAD: Final VAD applies the same SNR-weighted energy difference within each extended pitch segment, followed by smoothing, thresholding, and post-processing.The threshold uses β = 0.4 by default, while post-processing constrains speech frames relative to nearby pitch frames.

4. rVAD-fast based on spectral flatness

rVAD-fast replaces computationally expensive pitch extraction with spectral flatness (SFT) to identify voiced frames, yielding a more computationally efficient algorithm. An SFT threshold of 0.5 produces labels close to those of the pitch detector on clean and noisy speech examples.

  • SFT-based pitch replacement: rVAD-fast replaces computationally expensive pitch detection with spectral flatness to create a more computationally efficient VAD algorithm.The signal is windowed, transformed with STFT, and analyzed in the spectral domain.
  • SFT-based pitch replacement: SFT is calculated from the magnitude spectrum of each frame and compared with threshold θsft to classify frames as voiced or unvoiced.Frames with SFT ≤ θsft are treated as containing pitch.
  • Empirical comparison: At θsft = 0.5, SFT-generated pitch labels are close to those produced by the pitch detector for TIMIT and NIST 2016 SRE speech signals.The examples include clean TIMIT speech and noisy NIST 2016 SRE speech.
  • Empirical comparison: The SFT detector was compared with pitch detection across utterances from multiple clean and noisy speech databases.The studied databases include NIST 2016 SRE, TIMIT, RSR2015, RedDots, ASVspoof2015, and noisy speech conditions.

5. Experiments on the Aurora-2 database

Experiments on Aurora-2 evaluate rVAD across varied clean and noisy conditions, compare it with existing VAD methods, and test configuration choices. rVAD achieves lower FER than the referenced methods, while its denoising and postprocessing components affect performance and rVAD-fast trades moderate degradation for substantially faster processing.

  • Dataset and evaluation: Aurora-2 contains three test sets with clean and noisy speech spanning multiple noise types, SNR levels, and channel filters.Sets A and B use several environmental noises at 10, 5, 0, and -5 dB; Set C uses subway and street noise with an MIRS filter.
  • Comparison with referenced methods: rVAD with its default configuration achieves significantly lower average FER than all referenced VAD methods across Aurora-2 SNR levels, including Clean.The closest referenced method, VFR VAD, is followed by GMM-NLSM with a 3% absolute higher FER and DSR-AFE with more than 5% absolute higher FER.
  • Comparison with referenced methods: 11% FER for rVAD versus 36% for VQVAD demonstrates the margin over one compared method.The comparison uses frame error rate, with identical experimental settings and labels across the cited studies and this work.
  • Configuration analysis: Second-pass denoising with MSNE or MMSE improves rVAD on Aurora-2, while first-pass denoising has little effect and postprocessing improves performance.MSNE and MMSE perform almost identically; MSNE-mod performs worse because it is tailored to RATS noise characteristics.
  • Threshold sensitivity: A β value of 0.4 gives the best FER, while changing β by ±0.1 only marginally changes performance.Smaller β values yield smaller Pmiss and larger Pfa, whereas larger β values produce the opposite pattern.
  • rVAD-fast evaluation: rVAD-fast is approximately an order of magnitude faster than rVAD at the cost of moderate VAD performance degradation.Processing time was measured as CPU time on an Intel Core i7-4790 desktop with 16 GB RAM.

6. Experiments on the RATS database

On the RATS database, rVAD was evaluated across multiple noisy communication channels and compared with supervised and unsupervised VAD systems. It removed stationary and burst-like noise, achieved lower FER than several baselines, and performed close to a supervised system in the NIST OpenSAD comparison.

  • Illustrative denoising results: rVAD removed both stationary and burst-like noise while producing effective VAD results on illustrative RATS recordings.Figure 4 presents waveform, noisy and denoised spectrograms, and VAD outputs for Channels A and H.
  • Evaluation setup: The RATS evaluation used 200 speech files totaling approximately 44 hours across eight noisy communication channels.The recordings were randomly selected from the RATS database and used to compare rVAD with other VAD methods.
  • VAD comparison: rVAD achieved substantially lower FER than Sohn et al., Kaldi, VQVAD, and SSGMM on the RATS comparison.Kaldi had the lowest Pmiss but substantially higher Pfa and FER, while simple energy-threshold methods performed poorly under highly noisy conditions.
  • Configuration analysis: First-pass denoising improved performance slightly, while the best configuration combined first-pass denoising with modified MSNE for second-pass denoising.Overall denoising gains were modest, possibly because VAD operates on extended pitch segments that exclude non-pitch non-speech regions during pitch-segment detection.
  • Threshold sensitivity: Increasing β made rVAD more aggressive, increasing Pmiss and decreasing Pfa, so the threshold should be selected for the application.The threshold study used the default rVAD configuration on the RATS database.
  • NIST OpenSAD challenge: On the NIST 2015 OpenSAD challenge, rVAD substantially outperformed Sohn et al., GMM-MFCC, and GMM-PNCC and approached the supervised i-vector method.The challenge was also based on the RATS database.
  • Speaker verification: On RATS speaker verification, rVAD (MSNE-mod) obtained 5.6% EER, compared with 5.4% for BUT-VAD and 6.7% for GMM-PLP-RASTA.The unsupervised rVAD system was marginally worse than supervised BUT-VAD but better than supervised GMM-PLP-RASTA.

7. Experiments on the RedDots database for TD-SV

On the RedDots database, rVAD generally improved text-dependent speaker verification over using no VAD and performed comparably with VQVAD. Its fast spectral-flatness variant retained similar speaker-verification performance but failed under white noise, while rVAD remained stable across threshold settings.

  • Evaluation setup: The RedDots evaluation used short utterances from the male part-01 task with GMM-UBM-based text-dependent speaker verification.The database includes diverse speakers and channels, and GMM methods were selected because they perform well with short utterances.
  • Speaker-verification comparison: All VAD methods except Kaldi improved either EER or minDCF relative to the system without VAD.This result was reported for text-dependent speaker verification on RedDots part-01.
  • Speaker-verification comparison: rVAD outperformed Sohn et al. and Kaldi Energy VAD and was comparable to or marginally better than VQVAD.Its EER was almost identical to VQVAD, while its minDCF was slightly better.
  • Fast variant: rVAD-fast had similar speaker-verification performance to rVAD despite worse standalone VAD performance.The results suggest that stronger VAD performance did not necessarily produce better speaker-verification performance in this evaluation.
  • Threshold sensitivity: Changing β did not rapidly change rVAD's speaker-verification performance, demonstrating stability across threshold values.This stability was observed under both standard and noisy test conditions.
  • Noisy conditions: rVAD-fast matched rVAD under babble, market, and car noise but did not work under white noise because spectral flatness was severely affected.At 0–10 dB white-noise SNR, spectral flatness values were mostly near 1.0, so the θ_sft = 0.5 threshold detected no speech frames in many trials.

8. Conclusion

The paper concludes that rVAD combines two-pass denoising, extended pitch-segment detection, and SNR-weighted energy difference to support robust VAD across diverse conditions and tasks. It also presents rVAD-fast as a substantially cheaper alternative, while identifying configuration and application scope as areas for further study.

  • rVAD method: rVAD combines two-pass denoising, extended pitch-segment detection, and a posteriori SNR-weighted energy difference for unsupervised VAD.The first denoising pass removes high-energy noise using pitch as a speech indicator; the second uses speech enhancement before final VAD decisions.
  • rVAD-fast: rVAD-fast replaces computationally intensive pitch extraction with spectral flatness calculation, reducing complexity at the cost of moderately inferior VAD performance.The modified version is intended for large-scale processing and resource-limited devices.
  • rVAD-fast: Spectral flatness is a good indicator of whether a segment contains pitch unless the signal is severely corrupted by white noise.This finding explains the conditions under which the computationally cheaper rVAD-fast approach is applicable.
  • Evaluation: rVAD performed well across clean and noisy conditions for both VAD and speaker verification, with generalization across databases, noise conditions, and tasks.The evaluation covered diverse databases and noise conditions and compared rVAD with 16 supervised and unsupervised VAD methods.
  • Future work: Future work includes investigating optimal rVAD configurations for different applications and evaluating its performance for automatic speech recognition.These directions identify application-specific tuning and an additional downstream task not covered by the stated conclusion.
Loading 1906.03588v2…