Source-linked AI summary
Attentive Statistics Pooling for Deep Speaker Embedding
Koji Okabe, Takafumi Koshinaka, Koichi Shinoda
TL;DR
Text-independent speaker verification needs fixed-dimensional speaker embeddings from variable-length utterances, while conventional pooling averages frame features. The paper introduces attention-weighted means and standard deviations, and reports lower EERs on NIST SRE 2012 and VoxCeleb than statistics pooling.
Problem
Conventional speaker embedding averages frame-level features, limiting its representation of important frames and long-term speaker variation in variable-length utterances.
Method
Attentive statistics pooling uses an attention mechanism to compute importance-weighted means and standard deviations over frame-level features.
Results
7.5% and 8.1% lower EERs than statistics pooling were achieved on NIST SRE 2012 and VoxCeleb, respectively.
Takeaways & Limitations
Combining attention with standard deviations captures long-term speaker variations and yields higher-discriminative-power speaker embeddings in the evaluated text-independent tasks.
Takeaways & Limitations
Different training data were used for the SRE12 CC2 and VoxCeleb evaluation tasks to avoid condition mismatch.
Abstract
from arXiv · showhide
This paper proposes attentive statistics pooling for deep speaker embedding in text-independent speaker verification. In conventional speaker embedding, frame-level features are averaged over all the frames of a single utterance to form an utterance-level feature. Our method utilizes an attention mechanism to give different weights to different frames and generates not only weighted means but also weighted standard deviations. In this way, it can capture long-term variations in speaker characteristics more effectively. An evaluation on the NIST SRE 2012 and the VoxCeleb data sets shows that it reduces equal error rates (EERs) from the conventional method by 7.5% and 8.1%, respectively.
1. Introduction
Speaker-recognition systems evolved from i-vector representations toward deep speaker embeddings, but conventional pooling in variable-length utterances mainly averaged frame-level features. The paper proposes attentive statistics pooling to combine attention-weighted means and standard deviations for capturing long-term speaker variation.
- 1. Introduction: Deep neural networks increasingly replaced or supplemented i-vector-based feature extraction in speaker recognition.DNNs have been used for phoneme posteriors, bottleneck features, and speaker-discriminative vectors, with particularly strong results for short utterances.
- 1. Introduction: Text-independent speaker verification commonly uses average pooling to convert variable-length frame sequences into fixed-dimensional speaker embeddings.Average pooling aggregates frame-level speaker features across an utterance.
- 1. Introduction: Statistics pooling adds standard deviations to means, but its accuracy benefit had not yet been reported in the cited prior work.The added standard deviations represent higher-order information beyond the mean.
- 1. Introduction: Attention mechanisms weight frames by importance so embeddings focus on informative frames and represent long-term speaker characteristics more discriminatively.A small attention network computes importance values used to calculate weighted means.
- 1. Introduction: The proposed attentive statistics pooling combines attention-weighted means with attention-weighted standard deviations for text-independent, variable-duration speaker verification.The paper presents this as the first reported use of attentive statistics pooling in this setting.
2. Deep speaker embedding
A conventional DNN speaker-embedding system extracts frame-level features, pools variable-length sequences, and transforms the pooled representation into an utterance-level feature. Its pooling layer typically averages frame-level features before later neural processing produces the embedding.
- 2. Deep speaker embedding: A conventional DNN speaker-feature extractor contains three blocks for producing utterance-level features.The blocks are a frame-level feature extractor, a pooling layer, and an utterance-level feature extractor.
- 2. Deep speaker embedding: The frame-level feature extractor converts acoustic-feature sequences such as MFCCs or filter-bank coefficients into frame-level representations.TDNNs, CNNs, LSTMs, and GRUs can serve as the extractor.
- 2. Deep speaker embedding: The pooling layer converts variable-length frame-level features into a fixed-dimensional vector.This fixed-dimensional representation supports utterance-level speaker-feature extraction.
- 2. Deep speaker embedding: Average pooling obtains the average of all frame-level features.It is described as the most standard pooling type.
- 2. Deep speaker embedding: The utterance-level feature extractor uses stacked fully connected layers and can produce bottleneck features as utterance-level representations.Training may use cross-entropy, contrastive, or triplet loss, and PLDA can measure speaker similarity.
3. Higher-order pooling with attention
This section extends pooling with higher-order statistics and attention: attentive statistics pooling uses attention-weighted means and standard deviations to represent important frames and long-term speaker variation.
- 3.1. Statistics pooling: Standard deviation pooling captures temporal variability over long contexts that mean features do not represent.The paper contrasts this with LSTM context, whose effective scope is described as approximately 100 frames because of vanishing gradients.
- 3.2. Attention mechanism: Attention computes a scalar importance score for each frame, normalizes scores across the utterance, and uses them to calculate a weighted mean.This focuses the utterance-level feature on frames considered important for speaker discrimination.
- 3.3. Attentive statistics pooling: Attentive statistics pooling combines attention-based frame weighting with both weighted means and weighted standard deviations.The same attention weights are used for both statistics.
- 3.3. Attentive statistics pooling: The proposed method is presented as a way to capture speaker factors involving long-term variations more accurately and efficiently.The paper describes it as the first reported use of attentive statistics pooling for text-independent, variable-duration scenarios.
4. Experiments
Experiments compare four pooling strategies for deep speaker embeddings on NIST SRE12 CC2 and VoxCeleb, using controlled training, preprocessing, and PLDA evaluation. Attentive statistics pooling achieved the strongest reported performance, improving EER over statistics pooling on both datasets and across most tested durations.
- Experimental design: The experiments evaluated speaker verification on NIST SRE12 CC2 and VoxCeleb against conventional statistics pooling, attentive average pooling, and i-vector extraction.Performance was reported using EER and normalized detection cost functions.
- Experimental design: Four pooling methods were compared: average, statistics, attentive average, and attentive statistics pooling.The methods respectively use means; means plus standard deviations; weighted means; and weighted means plus weighted standard deviations.
- Data and setup: SRE12 training used telephone recordings with augmentation, while VoxCeleb used its development and test sets with noise and reverberation augmentation.Different training data were used for the two tasks to avoid condition mismatch.
- NIST SRE12: 7.5% lower EER was obtained by attentive statistics pooling than statistics pooling on NIST SRE12 CC2.It also achieved the best EER and minimum detection costs in the reported comparison.
- NIST SRE12: Attentive statistics pooling achieved the best EER under all tested SRE12 CC2 duration conditions except 300 seconds.It outperformed statistics pooling at every duration and was better than i-vectors on both 30-second and 100-second trials.
- VoxCeleb: 8.1% lower EER was obtained by attentive statistics pooling than statistics pooling on VoxCeleb, with the proposed method best on every evaluation measure.VoxCeleb evaluation segments averaged about 8 seconds, and speaker embeddings outperformed i-vectors in this comparison.
5. Summary and Future Work
The paper proposes attentive statistics pooling for deep speaker embeddings in text-independent speaker verification. It combines attention-weighted means and standard deviations, improving EER on both short- and long-duration evaluations while leaving the longest-duration challenge for future work.
- Summary: Attentive statistics pooling calculates attention-weighted means and standard deviations over frame-level features.The attention model emphasizes important frames, while standard deviations represent long-term speaker variations.
- Summary: 7.5% and 8.1% EER reductions were achieved on NIST SRE12 and VoxCeleb, respectively, relative to a conventional method.The experiments covered both short- and long-duration conditions.
- Future work: i-vectors remained competitive for long durations, including 300-second trials in SRE12 CC2.Improving accuracy under such conditions is identified as future work.