Source-linked AI summary
Generalized End-to-End Loss for Speaker Verification
Li Wan, Quan Wang, Alan Papir, Ignacio Lopez Moreno
TL;DR
The paper addresses inefficient end-to-end training for speaker verification and the challenge of supporting multiple keywords and languages. It introduces GE2E as a more efficient generalization of TE2E and MultiReader for combining data sources. GE2E improves EER by more than 10% and trains about 3× faster, while MultiReader improves multi-keyword verification by around 30% relative across four evaluation cases.
Problem
Speaker verification needs efficient training, while supporting multiple keywords creates a setting between text-dependent and text-independent verification.
Method
The paper generalizes TE2E with GE2E and introduces MultiReader to combine data sources for models supporting multiple keywords and languages.
Results
GE2E improves EER by more than 10% and trains about 3× faster than other loss functions, while MultiReader brings around 30% relative improvement across four keyword evaluation cases.
Takeaways & Limitations
Combining GE2E and MultiReader produces more accurate speaker-verification models supporting multiple keywords and languages.
Abstract
from arXiv · showhide
In this paper, we propose a new loss function called generalized end-to-end (GE2E) loss, which makes the training of speaker verification models more efficient than our previous tuple-based end-to-end (TE2E) loss function. Unlike TE2E, the GE2E loss function updates the network in a way that emphasizes examples that are difficult to verify at each step of the training process. Additionally, the GE2E loss does not require an initial stage of example selection. With these properties, our model with the new loss function decreases speaker verification EER by more than 10%, while reducing the training time by 60% at the same time. We also introduce the MultiReader technique, which allows us to do domain adaptation - training a more accurate model that supports multiple keywords (i.e. "OK Google" and "Hey Google") as well as multiple dialects.
1. INTRODUCTION
Speaker verification models use end-to-end neural training to compare utterance embeddings, but supporting multiple keywords requires a setting between text-dependent and text-independent verification. The paper generalizes TE2E with GE2E and introduces MultiReader for training across keywords and languages.
- Background: Speaker verification checks whether an utterance belongs to a specific enrolled speaker.Applications include Voice Match.
- Background: Text-dependent and text-independent speaker verification differ in how strongly enrollment and verification transcripts are constrained.The paper focuses on TI-SV and a particular TD-SV subtask.
- Background: Earlier neural speaker-verification systems commonly represented utterances with fixed-dimensional embedding vectors produced by end-to-end training.These embeddings can then be compared using simpler methods.
- Prior approach: TE2E simulates runtime enrollment and verification by training on tuples containing one evaluation utterance and M enrollment utterances.Positive tuples use the same speaker for evaluation and enrollment; negative tuples use different speakers.
- Prior approach: The TE2E loss encourages high similarity for same-speaker pairs and low similarity for different-speaker pairs.Its update uses positive and negative tuples and resembles triplet loss.
- Proposed approach: GE2E generalizes the TE2E architecture by constructing tuples from input sequences of various lengths more efficiently.The paper presents a definition, theoretical justification, and experimental evaluation of GE2E.
- Proposed approach: MultiReader enables one model to support multiple keywords and languages by combining different data sources.The technique is introduced as part of the paper’s approach to training a single model across domains.
2. GENERALIZED END-TO-END MODEL
GE2E trains speaker-verification embeddings in batches by comparing every utterance with all speaker centroids, emphasizing hard negative matches. Its contrast and softmax variants support different tasks, while MultiReader combines multiple data sources during training.
- GE2E training: GE2E processes batches containing N speakers and M utterances per speaker, rather than selecting examples before training.Each feature vector represents one utterance from one speaker.
- Embedding and similarity: GE2E L2-normalizes LSTM outputs into embeddings and compares each embedding with all speaker centroids using a scaled cosine similarity matrix.The similarity scale includes learnable parameters w and b, with w constrained positive.
- Loss design: The loss pulls each embedding toward its true-speaker centroid and away from other centroids, especially the most similar false-speaker centroid.The contrast formulation adds one positive component and one hard-negative component for every utterance.
- Experimental behavior: Contrast loss performs better for TD-SV, whereas softmax loss performs slightly better for TI-SV.Both GE2E implementations were useful in the experiments.
- GE2E versus TE2E: Each GE2E update for an utterance is identical to at least 2(N −1) TE2E steps when P = M, explaining its shorter convergence time.The comparison is based on GE2E updating all N×M embeddings toward their own centroids and away from other centroids.
- MultiReader: MultiReader combines K possibly unbalanced data sources by weighting each source’s expected loss with α_k during each training step.The method fetches one batch or tuple from each data source and computes a combined loss.
3. EXPERIMENTS
The experiments evaluate MultiReader for multi-keyword text-dependent speaker verification and compare GE2E with other losses for text-independent verification. Across these settings, MultiReader and GE2E improve verification performance while GE2E also reduces training time.
- Text-Dependent Speaker Verification: MultiReader addresses multi-keyword verification, whose transcripts are neither single-phrase constrained nor completely unconstrained.The technique is applied to multiple keyword data sources with substantially different sizes.
- Text-Dependent Speaker Verification: 30% relative EER improvement is reported for MultiReader across all four keyword enrollment-verification cases.Evaluation covers enrollment and verification with either keyword.
- Text-Dependent Speaker Verification: GE2E is about 10% better than TE2E on average EER in the larger text-dependent evaluation.The evaluation uses approximately 83K speakers and varied environmental conditions.
- Text-Independent Speaker Verification: TI-SV training uses batches containing partial utterances with a randomly selected common length between 140 and 180 frames.At inference, 160-frame windows with 50% overlap are averaged after L2 normalization to form the utterance d-vector.
- Text-Independent Speaker Verification: GE2E improves EER by more than 10% over softmax and TE2E and trains about 3× faster in TI-SV experiments.The comparison uses models trained on approximately 36M utterances from 18K speakers.
4. CONCLUSIONS
The paper proposes GE2E for more efficient speaker-verification training and MultiReader for combining data sources. Together, the techniques support more accurate models with multiple keywords and languages.
- 4. CONCLUSIONS: GE2E is proposed to train speaker verification models more efficiently.The conclusion reports theoretical and experimental verification of its advantage.
- 4. CONCLUSIONS: MultiReader combines different data sources to enable one model to support multiple keywords and languages.The technique is presented as part of the paper’s domain-adaptation approach.
- 4. CONCLUSIONS: Combining GE2E and MultiReader produces more accurate speaker verification models.This is the paper’s stated combined outcome.