Source-linked AI summary
Large Margin Softmax Loss for Speaker Verification
Yi Liu, Liang He, Jia Liu
TL;DR
Speaker verification needs embeddings that distinguish speakers while remaining compact within each speaker, but standard softmax does not explicitly enforce discriminative features. The paper evaluates large margin softmax configurations with Ring loss and MHE, finding that the AMSoftmax–MHE combination substantially outperforms the baseline on VoxCeleb.
Problem
Speaker verification requires embeddings that are discriminative between speakers and compact within speakers, whereas standard softmax does not explicitly encourage discriminative feature learning.
Method
The paper compares large margin softmax configurations and adds Ring loss or MHE as auxiliary objectives to improve feature normalization and inter-speaker separability.
Results
15% lower EER, 13% lower minDCF08, and 33% lower minDCF10 are reported for the AMSoftmax system with MHE relative to the baseline.
Takeaways & Limitations
AMSoftmax with MHE is the paper’s best-performing system and substantially outperforms the baseline on VoxCeleb.
Takeaways & Limitations
The practical application of the angular-softmax equations is scoped to settings with m1 approximately 1 and m2 less than 1.
Abstract
from arXiv · showhide
In neural network based speaker verification, speaker embedding is expected to be discriminative between speakers while the intra-speaker distance should remain small. A variety of loss functions have been proposed to achieve this goal. In this paper, we investigate the large margin softmax loss with different configurations in speaker verification. Ring loss and minimum hyperspherical energy criterion are introduced to further improve the performance. Results on VoxCeleb show that our best system outperforms the baseline approach by 15\% in EER, and by 13\%, 33\% in minDCF08 and minDCF10, respectively.
1. Introduction
The paper targets speaker embeddings that separate speakers while remaining compact within each speaker, addressing limitations of standard softmax training. It evaluates large margin softmax configurations with Ring loss and MHE on VoxCeleb.
- 1. Introduction: Speaker verification requires embeddings that are discriminative across speakers and compact within the same speaker.The paper frames verification as open-set recognition based on whether similarity exceeds a threshold.
- 1. Introduction: Cross-entropy with softmax is widely used but is designed for classification and does not explicitly encourage discriminative feature learning.
- 1. Introduction: The paper compares large margin softmax losses using multiplicative or additive angular margins, collectively termed large margin softmax loss.The compared variants include angular, additive angular, and additive margin formulations.
- 1. Introduction: The baseline outperforms the Kaldi x-vector recipe, reducing EER from 3.10% to 2.34%, minDCF08 from 0.0169 to 0.0122, and minDCF10 from 0.4977 to 0.3754.
2. Speaker embedding
The speaker embedding is extracted from an x-vector architecture that converts variable-length acoustic input into a fixed-dimensional speaker representation. Frame-level features are pooled across the utterance and processed by segment-level layers before embedding extraction.
- 2. Speaker embedding: The x-vector architecture extracts the speaker embedding from a hidden layer in its segment-level network.
- 2. Speaker embedding: Frame-level networks transform acoustic features into speaker-related vectors, which temporal pooling aggregates across the entire utterance.
- 2. Speaker embedding: The pooled representation is further processed by fully connected layers before the hidden-layer output is used as the speaker embedding.
3. Large margin softmax loss
The paper reformulates softmax around angular margins, then adds Ring loss and MHE to control feature norms and improve inter-speaker separation. An annealing strategy stabilizes optimization, while the formulation also connects to GE2E training.
- 3.1. Definition: Large margin softmax replaces norm-sensitive logits with angular target functions using multiplicative, additive-angular, or additive margins.The paper uses one margin at a time and names the resulting losses ASoftmax, ArcSoftmax, and AMSoftmax.
- 3.2. Feature normalization: Feature normalization removes feature norms from the loss, directing optimization toward angular discrimination and giving larger gradients to low-quality samples.The scaling factor s is fixed when features are normalized.
- 3.2. Feature normalization: Ring loss softly constrains feature norms toward a target R, with R learnable during training and λR weighting the auxiliary objective.It provides a soft alternative to an explicit normalization layer.
- 3.3. Enlarge inter-speaker feature separability: MHE adds an auxiliary objective that encourages normalized output weights to distribute evenly on a hypersphere, enlarging overall inter-class separability.The criterion uses a decreasing inverse-square function of pairwise normalized-weight distances.
- 3.4. Annealing strategy during training: Annealing replaces the target logit with a weighted average of its original and margin-based forms, reducing training difficulty during optimization.The coefficient λ decays according to training step and is bounded below by λ0.
- 3.5. Other discussions: Large margin softmax can be combined with GE2E-style minibatch computation, suggesting applicability when the training set contains millions of speakers.The connection follows by replacing batch-estimated centers with learnable output weights and omitting the bias.
4. Experimental setup
The experiments evaluate large margin softmax configurations on VoxCeleb using a Kaldi-like x-vector network, augmentation, and PLDA scoring. Training uses speaker-balanced segments and an annealing schedule for margin-based losses.
- 4. Experimental setup: Experiments use VoxCeleb1 development plus VoxCeleb2 for training and VoxCeleb1 test for evaluation, matching the Kaldi recipe.Performance is reported with EER, minDCF08, and minDCF10.
- 4. Experimental setup: The acoustic pipeline uses 30-dimensional MFCCs with cepstral mean normalization, energy-based VAD, and MUSAN/RIR data augmentation.The systems are implemented with Kaldi and TensorFlow.
- 4. Experimental setup: The network uses a 5-layer undilated TDNN, statistics pooling, and a 2-layer segment-level network with batch normalization and ReLU activations.The final segment-level ReLU is removed because it limits feasible feature-weight angles under angular large margin softmax.
- 4. Experimental setup: Training uses 64 sampled speakers per step, 200–400-frame segments, SGD, L2 regularization, and validation-triggered learning-rate halving.The baseline is trained with softmax cross-entropy.
- 4. Experimental setup: Margin-loss training applies λ annealing with slower decay for ASoftmax and ArcSoftmax than for AMSoftmax.The configurations use λ0=0 for ArcSoftmax and AMSoftmax, while ASoftmax uses λ0=10.
- 4. Experimental setup: Speaker embeddings are extracted from the second-last segment-level layer, reduced to 200 dimensions with LDA, and scored with PLDA.The experiments generally find PLDA better than a simple cosine backend.
5. Results
On VoxCeleb, large-margin softmax variants were compared with Ring and MHE enhancements, with AMSoftmax plus MHE producing the strongest reported system.
- 5. Results: Increasing weight decay from 0.0001 to 0.01 improves EER from 3% to 2.34%, and the baseline outperforms the standard Kaldi result.Simply normalizing weights without margins does not improve the modified softmax result.
- 5. Results: Removing the final ReLU generally improves large-margin results, although ASoftmax with m1 = 4 changes from 2.12% to 2.15% EER.For that configuration, minDCF08 changes from 0.0122 to 0.0113 and minDCF10 from 0.3214 to 0.3108 without ReLU.
- 5. Results: AMSoftmax performs best among the large-margin losses with an optimal margin of m3 = 0.20.ASoftmax is best at m1 = 4, while ArcSoftmax performs similarly with margins around m2 = 0.25 to m2 = 0.30.
- 5. Results: Ring loss improves minDCF08 and minDCF10, while its effectiveness with AMSoftmax is less significant because AMSoftmax already has relatively small norm variance.The margin reduces norm variance, and Ring loss shrinks the norm distribution further.
- 5. Results: 15% lower EER, 13% lower minDCF08, and 33% lower minDCF10 are achieved by AMSoftmax with MHE versus the baseline.The MHE-enhanced AMSoftmax system achieves the best result among the evaluated systems.
- 5. Results: MHE reduces the variance of pairwise squared distances between normalized output weights while keeping their means near 2.0.The smaller variance indicates more even speaker distribution on the hypersphere and better overall separability in the feature space.
6. Conclusions
The paper finds that large-margin softmax is promising for speaker verification, with AMSoftmax plus MHE providing the strongest system on VoxCeleb. It identifies future work in combining Ring and MHE losses and simplifying scoring.
- 6. Conclusions: AMSoftmax with MHE substantially outperforms the baseline on VoxCeleb and is the paper’s best system.The paper also reports that AMSoftmax is easier to train and generally performs better than ASoftmax and ArcSoftmax.
- 6. Conclusions: Future work will combine Ring and MHE losses with large-margin softmax and pursue cosine scoring without a PLDA backend.