Source-linked AI summary
Attention-Based Models for Text-Dependent Speaker Verification
F A Rezaur Rahman Chowdhury, Quan Wang, Ignacio Lopez Moreno, Li Wan
TL;DR
Text-dependent speaker verification must summarize utterances while handling phonemes mixed with silence and background noise. The paper evaluates attention mechanisms and pooling methods in an end-to-end LSTM system, finding that the best configuration reduces EER from 1.72% to 1.48%, a 14% relative improvement over the baseline.
Problem
The prior architecture does not capture silence and background noise well, despite phonemes being surrounded by these frames in short keyword segments.
Method
The paper evaluates attention-layer topologies, scoring functions, and attention-weight pooling methods within an end-to-end text-dependent speaker-verification system.
Results
1.72% to 1.48% EER, a 14% relative improvement over the baseline LSTM model, using shared-parameter nonlinear scoring, divided-layer attention, and sliding-window maxpooling.
Takeaways & Limitations
Attention-based models can improve the EER of this keyword-based text-dependent speaker-verification system over its non-attention LSTM baseline.
Abstract
from arXiv · showhide
Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence. In this paper, we analyze the usage of attention mechanisms to the problem of sequence summarization in our end-to-end text-dependent speaker recognition system. We explore different topologies and their variants of the attention layer, and compare different pooling methods on the attention weights. Ultimately, we show that attention-based models can improves the Equal Error Rate (EER) of our speaker verification system by relatively 14% compared to our non-attention LSTM baseline model.
1. INTRODUCTION
Speaker verification checks whether a verification utterance belongs to a known speaker, including fixed-phrase global-password tasks such as “OK Google” and “Hey Google.” The paper addresses silence and background noise in an LSTM-based end-to-end system by adding attention to emphasize relevant input frames.
- Speaker verification determines whether a verification utterance belongs to a known speaker using reference enrollment utterances.
- Text-dependent speaker verification constrains enrollment and verification transcripts to a specific phrase, including “OK Google” and “Hey Google” in this study.
- Earlier speaker-verification systems primarily used i-vectors with PLDA, while newer approaches increasingly use integrated end-to-end neural architectures.
- LSTM networks had previously achieved better performance than DNNs for integrated end-to-end text-dependent speaker verification.
- The prior architecture does not capture silence and background noise well, although keyword segments contain phonemes surrounded by both.
- The paper adds attention as a soft mechanism for emphasizing the most relevant elements of the input sequence and compares attention variants and pooling methods.
2. BASELINE ARCHITECTURE
The baseline trains an end-to-end LSTM system on evaluation and enrollment utterance tuples, represents each utterance by its final LSTM output, and compares speaker voiceprints using cosine similarity and a learned loss.
- Each training tuple contains one evaluation utterance and N enrollment utterances, with positive or negative labels determined by whether their speakers match.
- The LSTM processes fixed-length log-mel-filterbank feature segments, and tuples are generated alternately as positive and negative examples.
- The baseline represents each utterance with the last LSTM-layer output, ω = hT, as its d-vector.
- The centroid of the enrollment d-vectors represents the voiceprint constructed from N enrollment utterances.
- The system compares the evaluation d-vector with the enrollment centroid using cosine similarity and learns from a tuple-based end-to-end loss.
- The loss encourages higher similarity for same-speaker tuples and lower similarity for different-speaker tuples, resembling triplet loss.
3. ATTENTION-BASED MODEL
The model replaces last-frame d-vector extraction with attention-based sequence summarization, exploring scoring functions, layer topologies, and pooling methods. Variants alter where attention is computed, how representations are divided, or which attention weights are retained.
- 3.1. Basic attention layer: Attention scores can be learned from each LSTM frame output, normalized into weights, and used to form the d-vector as a weighted average across frames.This contrasts with the baseline, which directly uses the last frame output.
- 3.2. Scoring functions: Bias-only attention does not depend on the LSTM output, while linear attention uses frame-specific parameters and shared-parameter linear attention shares them across frames.All scoring-function parameters are trainable within the end-to-end architecture.
- 3.2. Scoring functions: Non-linear attention applies a tanh-based scoring function with tunable hidden dimension, while shared-parameter non-linear attention uses the same parameters for every frame.The non-linear formulation uses W_t, b_t, and v_t; its hidden dimension can be tuned on development data.
- 3.3. Attention layer variants: Cross-layer attention computes scores and weights from an intermediate LSTM layer but still forms the d-vector from the last-layer output.The variant separates the layer used for attention scoring from the layer supplying the representation.
- 3.3. Attention layer variants: Divided-layer attention doubles the last-layer output dimension, using one part to build the d-vector and the other to learn attention scores.The two parts are denoted part-a and part-b in the model description.
- 3.4. Weights pooling: Maxpooling can be applied to attention weights using sliding windows or global top-K selection, retaining only the largest values.The paper motivates this optional pooling as potentially increasing robustness to temporal input variation.
4. EXPERIMENTS
Experiments compare attention scoring functions, layer variants, and pooling methods under a common speaker-verification setup. Sliding-window maxpooling with the selected attention configuration improves EER, while pooled attention emphasizes later utterance frames.
- 4.1. Datasets and basic setup: The experiments use the same training and testing datasets and report speaker-verification EER across four enrollment–verification combinations.Training uses anonymized “OK Google” and “Hey Google” queries; testing includes 665 speakers.
- 4.1. Datasets and basic setup: The baseline is a 3-layer LSTM with 128-dimensional layers, a 64-dimensional projection, and a 64-dimensional linear output layer.The system uses 40-dimensional log-mel-filterbank features and 800ms keyword segments.
- 4.2. Basic attention layer: Non-linear attention improves EER more than bias-only or linear attention, especially when its scoring parameters are shared.The non-linear scoring-function intermediate dimension is m′ = 64.
- 4.3. Attention-layer variants: Divided-layer attention performs slightly better than basic and cross-layer attention, but doubles the final LSTM layer’s dimension.All variants use the shared-parameter non-linear scoring function fSNL.
- 4.4. Weights pooling: Sliding-window maxpooling further improves EER using a window size of 10 frames and a step of 5 frames.Global top-K maxpooling performs best with K = 5 in the reported evaluations.
5. CONCLUSIONS
The paper evaluates attention mechanisms for keyword-based text-dependent speaker verification and identifies a best-practice configuration. Combining shared-parameter non-linear scoring, divided-layer attention, and sliding-window maxpooling reduces baseline EER from 1.72% to 1.48%.
- 5. CONCLUSIONS: The recommended configuration uses shared-parameter non-linear scoring, divided-layer attention, and sliding-window maxpooling.These practices are selected from the paper’s comparisons of attention mechanisms and pooling methods.
- 5. CONCLUSIONS: 1.48% EER versus 1.72% for the baseline LSTM represents a 14% relative improvement.The result follows after combining the identified best practices.
- 5. CONCLUSIONS: The same attention mechanisms could potentially be applied to text-independent speaker verification and speaker diarization.The paper states this as a potential extension rather than an evaluated result.