Source-linked AI summary
A Unified Deep Neural Network for Speaker and Language Recognition
Fred Richardson, Douglas Reynolds, Najim Dehak
TL;DR
DNN-derived features and posteriors had been applied separately to speaker and language recognition, and this paper examines whether a single DNN can serve both tasks. Its unified approach improves results on both recognition tasks, including a 55% reduction in out-of-domain DAC13 EER.
Problem
Prior work applied DNNs to speaker and language recognition through separate task-specific methods, leaving a unified DNN approach for both tasks to be examined.
Source: §1 · p. 1Method
A single DNN provides bottleneck features and posterior statistics for separate speaker- and language-recognition i-vector systems, while the paper also examines a shared i-vector extractor.
Source: §1 · p. 1Results
The unified bottleneck features improved performance on both DAC13 speaker recognition and LRE11 language recognition, reducing out-of-domain DAC13 EER by 55%.
Source: §6 · p. 4Takeaways & Limitations
The authors report that learned bottleneck features outperform DNN posteriors alone for speaker or language recognition, while combining the two degrades performance.
Source: §6 · p. 4The sensitivity of bottleneck features to DNN configuration and training-data quality and quantity remains for future study.
Source: §6 · p. 4
Abstract
from arXiv · showhide
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The unified DNN approach is shown to yield substantial performance improvements on the the 2013 Domain Adaptation Challenge speaker recognition task (55% reduction in EER for the out-of-domain condition) and on the NIST 2011 Language Recognition Evaluation (48% reduction in EER for the 30s test condition).
1. INTRODUCTION
DNNs have been applied to speaker and language recognition through direct task-specific classification and indirect feature or posterior extraction. This work unifies two indirect methods with a single DNN across four i-vector systems and examines whether one i-vector extractor can serve both tasks.
1. INTRODUCTION
- The introduction distinguishes direct DNN classifiers trained for speaker or language recognition from indirect DNN extraction used to train a secondary classifier.Indirect approaches have extracted frame-level features, multinomial vectors, or multi-modal statistics for i-vector systems.Source: §1 · p. 1
- The unified approach combines bottleneck frame-level features with DNN posteriors used to accumulate multi-modal statistics for i-vector systems.The bottleneck features come from a DNN with a special bottleneck layer, while the posterior statistics follow.Source: §1 · p. 1
- A single DNN serves four i-vector systems—one per recognition task and method—and the work examines whether a single i-vector extractor can serve both speaker and language recognition.Source: §1 · p. 1
2. I-VECTOR CLASSIFIER FOR SR AND LR
The section describes i-vector systems as the SR/LR baseline and outlines their extraction, scoring, training requirements, and potential for shared use across both tasks.
2. I-VECTOR CLASSIFIER FOR SR AND LR
- i-vector systems are the SR/LR baseline, and the paper proposes integrating phonetic-knowledge-rich DNN features and posteriors into that framework.The authors note that this approach can provide significant performance improvements.Source: §2 · p. 1
- i-vector extraction uses speech detection, acoustic features and UBM-derived sufficient statistics, then maps them through T to normalized 400–600-dimensional vectors.Typical features are 20-dimensional MFCCs and derivatives for SR, or 56-dimensional static cepstra plus SDC at 100 vectors per second for LR; the accumulated statistics are zeroth, first, and second order.Source: §2 · p. 1, Figure 1 · p. 2
- Models and test segments are scored by i-vector similarity: cosine is simplest, while PLDA models within- and across-class variation using Σwc and Σac.Source: §2 · p. 2
- UBM, T, W, and m can be estimated from unlabeled in-domain audio, but Σwc and Σac require large labeled collections for speaker or language variation.For SR, these collections typically include thousands of speakers with tens of samples each; LR can use hundreds of enrollment samples per language from many speakers.Source: §2 · p. 2
- Because i-vector extraction is the most computationally expensive step, SR and LR can share vectors when they use the same features, UBM, and T, potentially trading off specialization.Source: §2 · p. 2
3. DEEP NEURAL NETWORK CLASSIFIER FOR SPEECH APPLICATIONS
This section describes DNN architecture and training for speech applications, then explains how DNN outputs support downstream feature extraction and i-vector statistics. DNN class posteriors can replace GMM posteriors for i-vector extraction, while hidden-layer activations can serve as features.
3.1. DNN architecture
- A DNN has fully connected layers with transformed activations; classifiers use softmax outputs to estimate class posterior probabilities.Classifier training uses cross-entropy against true class labels, whereas regression uses linear outputs and mean squared error.Source: §3.1 · p. 2
3.2. DNN Training for ASR
- ASR DNN training requires substantial transcribed speech, with senone labels derived using a phonetic dictionary and GMM/HMM ASR system.Training commonly uses mini-batch stochastic gradient descent and backpropagation; validation error guides learning-rate scheduling and stopping.Source: §3.2 · pp. 2–3
3.3. DNN bottleneck features
- A hidden-layer activation can serve as a feature vector, and a linear bottleneck layer reduces its dimensionality as part of DNN training.This work uses the second-to-last layer as the bottleneck, aiming to limit information loss from adversely affecting posterior prediction.Source: §3.3 · p. 3
3.4. DNN stats extraction for an i-vector system
- DNN class posteriors used instead of GMM component posteriors for i-vector statistics have yielded significant gains for speaker and language recognition.After statistics are accumulated, i-vector extraction proceeds as with GMM-based statistics.Source: §3.4 · p. 3
4. EXPERIMENT SETUP
Experiments used three corpora for DNN training and speaker and language recognition, with shared system components and distinct baseline and DNN front ends. The setup specifies the data, features, and architecture used for evaluation.
4.1. Corpora
- The DNN was trained on 100 hours of Switchboard 1, speaker recognition used DAC13, and language recognition used LRE11.The Switchboard subset follows the Kaldi example system; LR training and development data are detailed in.Source: §4.1 · p. 3
4.2.1. Commonalities
- Systems shared GMM-based speech activity segmentation; i-vector systems used MAP and PPCA for T-matrix estimation, PLDA scoring, a 2048-component UBM, and a 600-dimensional subspace.All LR systems used the discriminative backend described in.Source: §4.2.1 · p. 3
4.2.2. Baseline systems
- The baseline LR front end used 7 static cepstra plus 49 SDC, while baseline SR used 20 MFCCs including C0 and their first derivatives.Unlike the front end described in, baseline LR did not use VTLN or fNAP.Source: §4.2.2 · p. 3
4.2.3. DNN system
- The DNN used 4,199 senone targets and 819 input features formed from 39 PLP features stacked across a 21-frame window.It had seven 1024-node sigmoid hidden layers except for a 64-node linear sixth-layer bottleneck, and was trained on an nVidia Tesla K40 GPU with custom MIT/CSAIL software [13].Source: §4.2.3 · p. 3
5. EXPERIMENT RESULTS
Across speaker and language recognition experiments, bottleneck-based systems show stronger results than the baseline, while combining bottleneck features with DNN posteriors degrades speaker-recognition performance. Cross-task i-vector extraction incurs a mismatch penalty, with less degradation on DAC13 speaker recognition when using LRE11 language-recognition hyper-parameters.
5.1. Speaker recognition experiments
- DNN-posterior MFCCs improve on the baseline in both DAC13 conditions, bottleneck features with a GMM improve more, and combining bottleneck features with DNN posteriors degrades performance.For both in-domain and out-of-domain experiments, UBM and T hyper-parameters were trained on Switchboard data; other hyper-parameters used SRE data in-domain and Switchboard data out-of-domain.Source: §5.1 · p. 4
5.2. Language recognition experiments
- Bottleneck features with GMM posteriors outperform other LRE11 configurations, including five-system postevaluation fusion; bottleneck features with DNN posteriors improve over baseline more than in speaker recognition.Source: §5.2 · p. 4, Table 3 · p. 4
5.3. Cross-task i-vector Extraction
- Cross-task i-vector extraction degrades performance on both tasks, but degradation is smaller on DAC13 speaker recognition with LRE11 language-recognition hyper-parameters, motivating unified-system research.Source: §5.3 · p. 4, Table 4 · p. 4
6. CONCLUSIONS
A single DNN bottleneck feature extractor improved speaker and language recognition, with gains across DAC13 and LRE11 test conditions. The conclusions also identify the learned representation’s advantage over posteriors alone and directions for further work.
6. CONCLUSIONS
- DAC13 EER fell 26% in-domain and 55% out-of-domain, while LRE11 EER fell 48%, 39%, and 24% at 30s, 10s, and 3s.DAC13 DCF also fell 33% in-domain and 47% out-of-domain; on LRE11, bottleneck features outperformed a five-system fusion. The out-of-domain result used no in-domain data for DNN training or hyper-parameter adaptation.Source: §6 · p. 4
- Experiments suggest a common i-vector extractor may be possible for unified speaker and language recognition, and bottleneck-feature recognizers showed better calibrated scores by CLLR.The calibration observation was not presented in the paper.Source: §6 · p. 4
- Learned DNN bottleneck representations performed better than output posteriors alone, but combining the two degraded performance.The authors suggest a better task-matched UBM may explain the result, and propose exploring alternative classifiers and sensitivity to DNN configuration and training-data quality and quantity.Source: §6 · p. 4