Source-linked AI summary
A Unified Deep Neural Network for Speaker and Language Recognition
Fred Richardson, Douglas Reynolds, Najim Dehak
TL;DR
Speaker and language recognition had separately benefited from DNN-learned features and posteriors, motivating a unified treatment. The paper uses one DNN to provide both inputs within i-vector systems and reports substantial gains on DAC13 and LRE11. The approach also examines shared i-vector extraction across tasks, while leaving classifier choice and DNN configuration as areas for future study.
Problem
DNN-learned feature representations and sub-phoneme posteriors had been used separately for speaker and language recognition, motivating a unified approach.
Method
A single DNN supplies bottleneck features and DNN posteriors for four i-vector systems covering speaker and language recognition.
Results
The unified bottleneck approach decreased DAC13 out-of-domain speaker-recognition EER by 55% and LRE11 30s-test EER by 48%.
Takeaways & Limitations
One DNN-based bottleneck feature extractor is effective for both speaker and language recognition and outperforms MFCC/SDC i-vector and DNN-posterior approaches.
Takeaways & Limitations
The authors identify future work on alternative classifiers and sensitivity to DNN configuration and training-data quality and quantity.
Abstract
from arXiv · showhide
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The unified DNN approach is shown to yield substantial performance improvements on the the 2013 Domain Adaptation Challenge speaker recognition task (55% reduction in EER for the out-of-domain condition) and on the NIST 2011 Language Recognition Evaluation (48% reduction in EER for the 30s test condition).
1. INTRODUCTION
DNNs have improved speech recognition, motivating their use for speaker and language recognition through direct classifiers and indirect feature or posterior extraction. This work unifies two indirect DNN methods in a single DNN supporting four i-vector systems across both tasks.
- Approaches: DNNs are applied to speaker and language recognition through direct task-specific classifiers or indirect feature and posterior extraction.The indirect approach uses a DNN trained for another purpose, followed by a secondary classifier for recognition.
- Unified approach: The unified approach combines bottleneck features and DNN-derived posteriors as inputs to separate i-vector systems.Bottleneck features are frame-level representations, while DNN posteriors support accumulation of multimodal statistics.
- Unified approach: A single DNN supplies inputs for four i-vector systems spanning speaker recognition, language recognition, bottleneck features, and DNN posteriors.The work also examines whether one i-vector extractor can serve both recognition tasks.
2. I-VECTOR CLASSIFIER FOR SR AND LR
The i-vector framework converts speech into normalized low-dimensional representations for speaker and language recognition, then scores models against test segments. Shared extraction can reduce computation, but may trade away task-specific specialization.
- I-vector pipeline: An i-vector system extracts speech features, accumulates GMM-based sufficient statistics, transforms them with T, and normalizes the resulting representation before scoring.The pipeline includes speech activity detection, feature extraction, UBM posteriors, i-vector transformation, whitening, and length normalization.
- Scoring: PLDA scores model and test i-vectors using within-class and across-class variability matrices.The matrices characterize variation within a speaker or language and between different speakers or languages.
- Hyper-parameters: I-vector hyper-parameters include UBM, T, W, m, Σwc, and Σac, with labeled data required for the within- and across-class matrices.Unlabeled target-domain data can estimate general distributions, whereas SR and LR variability matrices require substantial labeled data.
- Cross-task extraction: Using the same i-vectors for speaker and language recognition can improve efficiency when feature extraction, UBM, and T are shared.The trade-off is that shared components are not specialized for either task.
3. DEEP NEURAL NETWORK CLASSIFIER FOR SPEECH APPLICATIONS
DNNs transform acoustic inputs through layered nonlinear networks and can provide either classifier posteriors or learned hidden-layer features. In i-vector systems, DNN posteriors replace GMM posteriors, while bottleneck activations provide reduced-dimensional features.
- DNN architecture: A DNN computes successive weighted transforms and activation functions across input, hidden, and output layers.Each layer applies a weight matrix followed by an activation function.
- DNN classifier: Classifier DNNs use softmax outputs to estimate posterior probabilities for classes such as sub-phonetic senones.The classifier objective is cross entropy between predicted outputs and true class labels.
- DNN training: DNN training commonly uses mini-batch stochastic gradient descent with backpropagation and validation-based learning-rate scheduling.Training stops when validation error can no longer be reduced.
- DNN bottleneck features: A linear bottleneck layer reduces hidden-layer dimensionality and supplies learned features to a secondary classifier.The bottleneck transformation is optimized during DNN training and is used here near the network output.
- DNN statistics: DNN posteriors can replace GMM component posteriors when accumulating i-vector statistics, after which extraction proceeds as usual.This posterior-based approach has produced gains for both speaker and language recognition.
4. EXPERIMENT SETUP
The experiments train the DNN on Switchboard and evaluate speaker recognition on DAC13 and language recognition on LRE11. Systems share common i-vector infrastructure while varying input features, statistics, and adaptation data across conditions.
- Corpora: The DNN uses 100 hours of Switchboard 1, while speaker recognition is evaluated on DAC13 and language recognition on LRE11.The corpora separate DNN training from the two recognition evaluations.
- Shared system configuration: All systems use GMM-based speech activity segmentation, MAP and PPCA T estimation, PLDA scoring, a 2048-component UBM, and a 600-dimensional i-vector subspace.LR systems additionally use a discriminative backend.
- Baseline front ends: Baseline LR uses static cepstra with SDC, while baseline SR uses 40 MFCC features comprising 20 coefficients and their first derivatives.These baselines omit VTLN and fNAP for LR.
- DNN configuration: The DNN input consists of 39 PLP-derived features stacked over a 21-frame window, with 4,199 senone targets.GMM SAD segmentation is applied to the stacked features.
- DNN configuration: The DNN contains seven hidden layers, including a 64-node linear bottleneck layer, with the remaining hidden layers using sigmoid activations.The bottleneck is the sixth hidden layer.
5. EXPERIMENT RESULTS
Experiments on DAC13 and LRE11 evaluate unified DNN-based i-vector systems across speaker and language recognition, including cross-task extraction. The results show strong language-recognition performance and motivate shared extraction parameters despite mismatched-task degradation.
- DAC13 speaker recognition: DAC13 experiments compare in-domain and out-of-domain conditions with hyper-parameters trained from Switchboard and speaker-recognition evaluation data.The UBM and T parameters use Switchboard data in both conditions, while other parameters differ by domain.
- LRE11 language recognition: Bottleneck features with GMM posteriors outperform the other LRE11 configurations, including a five-system fusion.Bottleneck features with DNN posteriors improve more over baseline on language recognition than in the speaker-recognition experiments.
- Reported results: 2.00% EER / 0.269 DCF is one reported experiment result.
- Reported results: 2.68% EER / 0.368 DCF is another reported experiment result.
- Cross-task extraction: Cross-task i-vector extraction degrades performance on the mismatched task, but the degradation is smaller for DAC13 speaker recognition using LRE11 language-recognition hyper-parameters.The result motivates further research on a unified extractor through careful UBM and T training-data selection.
6. CONCLUSIONS
The paper presents DNN bottleneck features for both speaker and language recognition and reports substantial gains over MFCC/SDC i-vector and DNN-posterior approaches. It also finds that bottleneck features outperform DNN posteriors alone, while combined features degrade performance and motivate further study.
- Contributions: DNN bottleneck features improve speaker and language recognition over MFCC/SDC i-vector and DNN-posterior approaches.
- Speaker recognition: 55% lower out-of-domain EER and 47% lower DCF are reported for DAC13 speaker recognition.The out-of-domain result uses no in-domain data for DNN training or hyper-parameter adaptation.
- Language recognition: 48%, 39%, and 24% EER reductions are reported on LRE11 at 30s, 10s, and 3s test durations, respectively.The bottleneck features also outperform a five-system fusion of acoustic and phonetic recognizers.
- Feature analysis: Using learned bottleneck representations appears better than using DNN output posteriors alone, while combining the two degrades performance.The paper suggests this may reflect a better-matched posterior estimator trained with task-matched data.
- Future work: Future work will examine alternative classifiers and the sensitivity of bottleneck features to DNN configuration and training-data quality and quantity.