Source-linked AI summary
Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics
Thomas Drugman, Abeer Alwan
TL;DR
Pitch tracking remains difficult in noise because existing trackers degrade rapidly outside clean conditions. The paper analyzes residual-signal harmonics with SRH for both pitch estimation and voicing-boundary detection, and reports comparable clean-speech performance with significant improvement in noisy conditions.
Problem
Pitch trackers that perform well in clean speech often degrade rapidly in noisy environments, leaving accurate and robust F0 tracking an open problem.
Method
The paper analyzes harmonics in the residual-signal spectrum and uses SRH for pitch estimation and voicing-boundary determination.
Results
In 9 out of the 10 noisy experiments, SRH led to significant improvement while remaining comparable to other techniques in clean conditions.
Takeaways & Limitations
Residual-harmonic analysis provides a pitch-tracking approach with particular robustness to additive noise across the evaluated conditions.
Takeaways & Limitations
The SRH formulation does not address odd-harmonic ambiguity, which may be problematic for low-pitched voices.
Abstract
from arXiv · showhide
This paper focuses on the problem of pitch tracking in noisy conditions. A method using harmonic information in the residual signal is presented. The proposed criterion is used both for pitch estimation, as well as for determining the voicing segments of speech. In the experiments, the method is compared to six state-of-the-art pitch trackers on the Keele and CSTR databases. The proposed technique is shown to be particularly robust to additive noise, leading to a significant improvement in adverse conditions.
1. Introduction
Pitch tracking estimates F0 contours for voiced speech and supports applications including coding, analysis, synthesis, and recognition. Because current trackers degrade rapidly in noise, the paper proposes residual-spectrum harmonic analysis and evaluates it quantitatively against state-of-the-art methods, especially under noise.
- Pitch tracking estimates the fundamental-frequency F0 contour of voiced speech for use in coding, analysis, synthesis, and recognition.
- Current pitch trackers generally perform well in clean conditions but degrade rapidly in noisy environments.
- The paper proposes estimating pitch from harmonics in the residual signal spectrum rather than directly from the speech spectrum.The method is presented as distinct from prior harmonic-summation approaches and from SIFT's residual-signal autocorrelation.
- The evaluation compares the proposed technique with other state-of-the-art pitch trackers, focusing particularly on robustness to noise.The paper describes an extensive quantitative assessment and reports comparisons in Section 3.
2. Pitch tracking based on residual harmonics
The method whitens speech through inverse filtering, analyzes residual harmonics with SRH, and uses the same criterion for pitch estimation and voicing decisions. It is informative for voicing detection and remains effective in noisy speech, although noise weakens harmonic evidence and the formulation leaves an odd-harmonic ambiguity.
- Method: Inverse filtering removes major vocal-tract and noise contributions, producing a residual spectrum with harmonic peaks for voiced speech.The residual is obtained from autoregressive spectral-envelope modeling, and the spectrum is computed on Hanning-windowed frames.
- Method: SRH sums the first Nharm harmonic contributions, subtracts a neighboring term to reduce even-harmonic maxima, and estimates pitch by maximizing SRH(f).
- Voicing detection: SRH can determine voicing with a local threshold: a frame is voiced when SRH(F0*) exceeds θ after per-frame residual-spectrum energy normalization.
- Limitation: The equation does not address ambiguity with odd harmonics, which may affect low-pitched voices when the third harmonic enters the search range.
- Illustrative results: In a 0 dB SNR Jet-noise example, pitch estimates closely agree with the reference in clean and noisy speech except for a short five-frame octave error.The example used no post-correction through dynamic programming.
- Noise behavior: SRH carries substantial information about voicing decisions, but noise lowers voiced-region values and makes those decisions more difficult.
3. Experiments
The experiments optimize and evaluate SRH-based pitch tracking and voicing detection against established trackers in clean speech and additive noise. Results show comparable clean-speech performance and a clear advantage for SRH in most noisy conditions.
- Evaluation measures: Performance is assessed with VDE, GPE, FPE, and FFE, with FFE combining voicing-decision and gross-pitch errors as an overall measure.GPE uses a 20% relative-F0 threshold, while FPE summarizes relative errors below that threshold.
- Experimental protocol: Noisy speech combines original recordings with Noisex-92 noises at 0 dB SNR, including babble, car, factory, jet-cockpit, and white noise.
- Experimental protocol: Training minimizes overall FFE across speakers and clean/noisy APLAWD conditions, while testing uses Keele and CSTR recordings with provided pitch references.The protocol uses 16 kHz recordings and a 10 ms frame shift.
- Compared methods: The comparison includes SRH, SSH, RAPT-based F0, SHRP, TEMPO, autocorrelation, crosscorrelation, and YIN-based methods.SSH applies the harmonic approach to speech rather than the residual, while YIN uses SRH voicing decisions.
- Results: In 9 out of 10 noisy gender-by-noise cases, SRH outperforms existing methods; the exception is male speech with babble noise.For noisy speech, FFE decreases by at least 8.5% for female speakers and by 5.3% for male speakers relative to existing approaches, while male VDE decreases by 5.7%.
4. Conclusion
The paper presents SRH, a residual-spectrum pitch-tracking criterion used for both pitch estimation and voicing-boundary determination. SRH improves robustness to additive noise while remaining comparable to other techniques in clean conditions.
- SRH analyzes the residual spectrum for pitch tracking and determines voicing boundaries.
- 9 out of the 10 noisy experiments showed significant improvement with SRH.
- SRH performance was comparable to other techniques in clean conditions.