Source-linked AI summary
Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition
Xu Xiang, Shuai Wang, Houjun Huang, Yanmin Qian, Kai Yu
TL;DR
The paper addresses the limitation that Softmax training does not explicitly enforce compact same-speaker embeddings and separated different-speaker embeddings. It introduces three fixed-margin losses within x-vector speaker embedding learning and evaluates them on VoxCeleb1 and SITW. The proposed systems reduce EER by 25% ∼30% against strong Softmax baselines, reaching 2.238% and 2.761% on the two tasks.
Problem
Softmax loss does not explicitly enforce intra-class similarity and inter-class diversity, leaving speaker embeddings suboptimal for speaker recognition.
Method
The paper introduces A-Softmax, AM-Softmax, and AAM-Softmax, which impose fixed margins between classes in x-vector speaker embedding learning.
Results
25% ∼30% EER reduction is achieved on both tasks versus strong Softmax baselines, with 2.238% EER on VoxCeleb1 and 2.761% on SITW core-core.
Takeaways & Limitations
The results show that incorporating margin-based classification losses substantially improves discriminative speaker embeddings over the Softmax baseline.
Abstract
from arXiv · showhide
Recently, speaker embeddings extracted from a speaker discriminative deep neural network (DNN) yield better performance than the conventional methods such as i-vector. In most cases, the DNN speaker classifier is trained using cross entropy loss with softmax. However, this kind of loss function does not explicitly encourage inter-class separability and intra-class compactness. As a result, the embeddings are not optimal for speaker recognition tasks. In this paper, to address this issue, three different margin based losses which not only separate classes but also demand a fixed margin between classes are introduced to deep speaker embedding learning. It could be demonstrated that the margin is the key to obtain more discriminative speaker embeddings. Experiments are conducted on two public text independent tasks: VoxCeleb1 and Speaker in The Wild (SITW). The proposed approach can achieve the state-of-the-art performance, with 25% ~ 30% equal error rate (EER) reduction on both tasks when compared to strong baselines using cross entropy loss with softmax, obtaining 2.238% EER on VoxCeleb1 test set and 2.761% EER on SITW core-core test set, respectively.
1. Introduction
Speaker recognition has shifted toward DNN-derived embeddings, but standard Softmax training does not explicitly enforce compact same-speaker clusters and separated different-speaker clusters. This paper studies fixed-margin losses and reports strong gains on two text-independent tasks.
- Task definition: Speaker recognition identifies or confirms a person’s identity from speech segments through enrollment and testing phases.The task is text-independent when enrollment and testing speech need not share the same content.
- DNN embeddings: DNN speaker classifiers can produce utterance-level x-vector embeddings for speaker modeling.This approach emerged alongside deep neural networks and was investigated as an alternative to conventional speaker modeling.
- Research gap: Softmax loss does not explicitly encourage inter-class separability or intra-class compactness, limiting embedding generalizability to unseen speakers.The paper contrasts this with the need for same-identity embeddings to aggregate and different-identity clusters to separate.
- Proposed approach: Three fixed-margin losses—A-Softmax, AM-Softmax, and AAM-Softmax—are studied for discriminative embedding learning.The experiments use the VoxCeleb1 and SITW text-independent tasks.
- Results: 25% ∼30% EER reduction is reported against strong Softmax baselines, with 2.238% EER on VoxCeleb1 and 2.761% EER on SITW core-core.The paper reports these results as state-of-the-art performance on the evaluated tasks.
2. DNN speaker embedding systems
The system uses an x-vector-style DNN that aggregates frame-level speech into segment-level embeddings. It is trained with stochastic gradient descent, while the paper questions whether Softmax is sufficient for discriminative speaker embedding learning.
- System design: The DNN systems are based on the x-vector architecture and use configurations similar to Kaldi recipes.Implementations are built using Kaldi or PyTorch.
- Architecture: Five time-delay layers process frame-level inputs before statistical pooling aggregates each sequence into a segment-level representation.The pooling layer computes the mean and standard deviation of the input sequence.
- Training and extraction: The network is optimized with stochastic gradient descent, and speaker embeddings are extracted from the trained segment-level layer.The paper uses this embedding-learning setup to study alternative classification losses.
- Training objective: The effectiveness of Softmax for deep speaker embedding learning is questioned, motivating loss functions that explicitly model classification margins.This motivation leads to the paper’s margin-based loss investigation.
3. Losses for training the speaker discriminative DNN
The section introduces Softmax and three margin-based losses for speaker-discriminative DNN training. These losses modify angular or additive margins to encourage more discriminative embeddings.
- Softmax loss: Softmax loss does not explicitly enforce intra-class similarity or inter-class diversity, creating a performance gap for speaker recognition.The loss primarily penalizes classification error rather than directly shaping embedding structure.
- A-Softmax loss: A-Softmax produces intrinsically angularly distributed embeddings because class probabilities depend on angles between weights and inputs.The modified Softmax formulation discards bias and focuses on weight directions.
- A-Softmax loss: A-Softmax introduces a multiplicative angular margin controlled by the integer m.The margin is incorporated through a piecewise function applied to the target-class angle.
- AM-Softmax loss: AM-Softmax uses an additive margin and normalizes inputs to unit vectors to provide a hyperspherical embedding geometry.The scaling factor s prevents gradients from becoming too small during training.
- AAM-Softmax loss: AAM-Softmax defines the target function with an additive angular margin, using φ(θyi,i) = cos(θyi,i + m).This angular formulation follows the interpretation of distance on a unit hypersphere through the angle between vectors.
4. Experiments
The experiments evaluate the systems on the VoxCeleb1 and SITW test sets, using mostly shared settings while allowing detailed training data to differ by task.
- Experimental tasks: Experiments are conducted on the VoxCeleb1 and SITW test sets.The two evaluations use mostly the same settings, with task-specific training data described separately.
4.1. Basic experimental set-up
The basic setup combines augmented acoustic features, a speaker-discriminative DNN that extracts 512-dimensional embeddings, distributed SGD training, and PLDA scoring after LDA projection.
- Data and features: Training data are augmented with noise, music, babble, and reverberation to increase data amount and diversity.
- Data and features: The system uses 30-dimensional MFCCs with 10 ms frame shift, 25 ms windows, sliding-window mean normalization, and energy-based VAD.
- DNN architecture: The DNN extracts 512-dimensional speaker embeddings from segment6’s affine layer after training.The architecture is similar to Kaldi’s VoxCeleb1 or SITW recipe architecture.
- Optimization: Models are trained on randomly cut 2–4 second segments using synchronous SGD coordinated across eight GPUs.PyTorch and Horovod are used, with batch size 64 on each GPU.
- Optimization: Training begins with a learning rate of 0 and increases to 1e-4 over the first 65,536 batches.
- Optimization: All systems use three epochs, learning rate 1e-4, momentum 0.7, weight decay 1e-5, and maximum gradient norm 1e3.
- Scoring: PLDA scores centered embeddings after LDA projection to 128 dimensions and length normalization.
4.2. System evaluation on VoxCeleb1
On VoxCeleb1, margin-based losses substantially outperform Softmax baselines, with AAM-Softmax achieving the strongest reported result. Evaluation also extends to disjoint and harder VoxCeleb1 test sets.
- Original VoxCeleb1 test set: 2.238% EER: AAM-Softmax achieves a 30% reduction compared with the baseline on the VoxCeleb1 test set.The result is reported as the best published number on that test set.
- Original VoxCeleb1 test set: A-Softmax, AM-Softmax, and AAM-Softmax all outperform the standard Softmax baseline by a large margin.These systems replace the standard Softmax loss in the speaker embedding model.
- VoxCeleb2 development set only: The VoxCeleb2-only evaluation uses 5,994 disjoint speakers without data augmentation and tests extended and same-gender, same-nationality trial conditions.The hard set draws pairs from identities sharing gender and nationality.
- VoxCeleb2 development set only: 16%, 12%, and 6% EER reductions are achieved on VoxCeleb1, VoxCeleb1-E, and VoxCeleb1-H, respectively, versus previous state-of-the-art systems.The three test sets include the original, extended, and hard VoxCeleb1 evaluations.
4.3. System evaluation on SITW
On SITW, margin-based losses outperform the baselines, with AAM-Softmax producing the largest reported EER reduction. The evaluation uses VoxCeleb-derived training data with overlapping speakers removed, but variable utterance lengths complicate interpretation.
- SITW evaluation: The SITW model is trained on VoxCeleb2 development data and VoxCeleb1, excluding 60 VoxCeleb1 speakers overlapping with the SITW core-core test set.The training set contains 1,236,567 utterances from 7,185 speakers and uses the same augmentation strategy as the VoxCeleb1 experiment.
- SITW evaluation: 20%, 17%, and 25% EER reductions result from A-Softmax, AM-Softmax, and AAM-Softmax, respectively, compared with the baselines.AAM-Softmax gives the largest reduction among the three losses.
5. Conclusions
The paper concludes that margin-based losses improve x-vector speaker embeddings by promoting the desired separation and compactness, with strong gains on VoxCeleb1 and SITW.
- Conclusions: A-Softmax, AM-Softmax, and AAM-Softmax are introduced into x-vector learning to improve interspeaker separation and intraspeaker compactness.The paper contrasts these margin-based objectives with Softmax loss.
- Conclusions: 25%–30% EER reduction is achieved on both VoxCeleb1 and SITW versus strong Softmax baselines, with 2.238% and 2.761% EER, respectively.The paper reports these results as state-of-the-art performance.