Source-linked AI summary
Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM
Szu-Wei Fu, Yu Tsao, Hsin-Te Hwang, Hsin-Min Wang
TL;DR
Conventional speech-quality tools require a clean reference, limiting their use when that reference is unavailable. Quality-Net addresses this gap with an end-to-end, non-intrusive BLSTM model that estimates utterance quality from frame-level assessments and shows high correlation to PESQ.
Problem
Clean-reference requirements restrict objective speech-quality assessment in real-world scenarios where the reference cannot be accessed.
Method
Quality-Net is an end-to-end, non-intrusive BLSTM model that estimates utterance-level quality from frame-level assessments using speech utterances and quality scores.
Results
Quality-Net yields high correlation to PESQ, and its frame constraint improves overall performance while differentiating clean from degraded frames.
Takeaways & Limitations
Quality-Net's non-intrusive, frame-level, end-to-end design increases practicality and allows direct combination with speech enhancement models.
Takeaways & Limitations
Frame-level quality scores are context-dependent and should be compared only within an utterance, while the experiments use PESQ-based labels as a pilot for future MOS assessment.
Abstract
from arXiv · showhide
Nowadays, most of the objective speech quality assessment tools (e.g., perceptual evaluation of speech quality (PESQ)) are based on the comparison of the degraded/processed speech with its clean counterpart. The need of a "golden" reference considerably restricts the practicality of such assessment tools in real-world scenarios since the clean reference usually cannot be accessed. On the other hand, human beings can readily evaluate the speech quality without any reference (e.g., mean opinion score (MOS) tests), implying the existence of an objective and non-intrusive (no clean reference needed) quality assessment mechanism. In this study, we propose a novel end-to-end, non-intrusive speech quality evaluation model, termed Quality-Net, based on bidirectional long short-term memory. The evaluation of utterance-level quality in Quality-Net is based on the frame-level assessment. Frame constraints and sensible initializations of forget gate biases are applied to learn meaningful frame-level quality assessment from the utterance-level quality label. Experimental results show that Quality-Net can yield high correlation to PESQ (0.9 for the noisy speech and 0.84 for the speech processed by speech enhancement). We believe that Quality-Net has potential to be used in a wide variety of applications of speech signal processing.
3 Institute of Information Science, Academia Sinica, Taipei, Taiwan
The listed affiliation identifies the Institute of Information Science, Academia Sinica, in Taipei, Taiwan.
- The passage lists institutional contact email addresses.
1. Introduction
Speech quality assessment is difficult without a clean reference, motivating Quality-Net, an end-to-end non-intrusive model that learns quality from speech alone. Its design supports frame-level assessment and future integration with speech enhancement.
- Clean-reference requirements restrict objective speech quality assessment in real-world scenarios where reference speech is unavailable.
- Existing non-intrusive methods include handcrafted features or separate feature extraction and prediction stages rather than jointly optimized end-to-end learning.
- Quality-Net uses a BLSTM to provide end-to-end, non-intrusive speech quality evaluation and infer frame-level quality.
- Frame-level quality assessment can locate degraded regions within an utterance.
- Quality-Net can be combined with speech enhancement because its end-to-end structure supports gradient-based optimization.
- The study identifies boosting enhanced-speech PESQ as future work and describes Quality-Net as the first end-to-end non-intrusive model to yield frame-level quality.
2. Quality-Net
Quality-Net maps variable-length speech spectrograms to utterance-level quality scores while learning meaningful frame-level assessments. Its frame constraint and BLSTM context design support localization of degraded regions but limit cross-utterance comparison of frame scores.
- Quality-Net: Quality-Net uses magnitude spectrograms and BLSTM to map variable-length speech utterances to fixed-length quality scores.The model reads the whole spectrogram before predicting the utterance-level score.
- Conditional constraint on frame quality assessment: The model learns frame-level quality despite training with only utterance-level labels.A conditional constraint addresses cases where noise or distortion varies across frames.
- Conditional constraint on frame quality assessment: The loss combines utterance-level accuracy with a conditional constraint that encourages uniformly distributed frame quality for high-quality speech.The constraint’s influence decreases exponentially as estimated utterance quality falls from the metric maximum.
- Conditional constraint on frame quality assessment: The constraint guides Quality-Net to distinguish clean frames from degraded frames within an utterance.This gives the intermediate frame-level assessment a localization function.
- Limited context influence: BLSTM context makes frame scores depend on distant past and future information, so comparisons are valid within an utterance but not across utterances.A clean frame in a noisy utterance may receive a different score from an identical-feature frame in a clean utterance.
3. Experiments
Experiments evaluate Quality-Net on clean, noisy, and enhanced speech, examine frame-level constraints and forget-gate initialization, and compare its assessment with a two-stage model while varying training-set size.
- Experimental setup: Quality-Net was trained and tested on clean, noisy, and enhanced TIMIT speech, using PESQ-based quality labels.The implementation uses one bidirectional LSTM layer, fully connected layers, and a global average layer for utterance-level scoring.
- Effects of frame-level quality constraint: Frame constraints reduce frame-quality variance in clean speech and significantly improve overall assessment performance.The constrained model also differentiates clean from degraded frames more clearly in the reported example.
- Effects of initialization on forget gate bias: A forget-gate bias initialized to -3 preserves low scores in the noisy 40th–100th frames without lowering scores in distant clean regions.With initialization at 1, distant context causes the noisy region to affect clean frames; the -3 setting is used thereafter.
- Detailed assessment results: Enhanced utterances are harder to evaluate than noisy and clean utterances, especially at low PESQ values.The enhanced-speech scatter plot has a lower correlation coefficient than the noisy-and-clean plot.
- Detailed assessment results: Quality-Net outperforms the compared two-stage model, possibly because it jointly optimizes feature extraction and assessment.The comparison is reported for the test-set assessment results.
- Relation between number of training utterances and assessment performance: 1000 training utterances are sufficient for accurate quality prediction, while 100 utterances achieve a correlation coefficient of 0.7.Performance begins to saturate at 1000 utterances.
4. Conclusions
Quality-Net is a non-intrusive, end-to-end speech quality evaluation model that achieves high correlation with PESQ and supports practical speech-processing applications.
- Quality-Net is a novel, non-intrusive, and end-to-end speech quality evaluation model.
- Quality-Net achieves high correlation with PESQ.
- Its non-intrusive property and frame-level quality assessment increase practicality across different applications.
- The end-to-end framework allows direct combination with speech enhancement models, such as use as a perceptual loss.