Source-linked AI summary
In defence of metric learning for speaker recognition
Joon Son Chung, Jaesung Huh, Seongkyu Mun, Minjae Lee, Hee Soo Heo, Soyeon Choe, Chiheon Ham, Sunghwan Jung, Bong-Jin Lee, Icksang Han
TL;DR
The paper asks whether metric learning can effectively produce discriminative embeddings for open-set speaker recognition, where test speakers are unseen during training. It compares popular loss functions under controlled VoxCeleb experiments and finds competitive triplet-loss performance and stronger results for its proposed angular objective. The study concludes that metric-learning approaches can outperform popular classification-based methods within the evaluated setting.
Problem
Open-set speaker recognition requires embeddings with small intra-speaker and large inter-speaker distances, while classification objectives are widely believed to outperform metric learning.
Method
The paper conducts controlled experiments comparing popular classification and metric-learning objectives, including GE2E, prototypical losses, and a proposed angular objective.
Results
The vanilla triplet loss outperforms the common AM-Softmax setting, while the proposed angular objective exceeds all classification-based and metric-learning methods.
Takeaways & Limitations
Metric-learning objectives, particularly the proposed angular objective, are competitive alternatives to classification-based training for open-set speaker recognition.
Takeaways & Limitations
AM-Softmax and AAM-Softmax performance is sensitive to training scale and margin values, and speaker-recognition comparisons can vary with system design choices.
Abstract
from arXiv · showhide
The objective of this paper is 'open-set' speaker recognition of unseen speakers, where ideal embeddings should be able to condense information into a compact utterance-level representation that has small intra-speaker and large inter-speaker distance. A popular belief in speaker recognition is that networks trained with classification objectives outperform metric learning methods. In this paper, we present an extensive evaluation of most popular loss functions for speaker recognition on the VoxCeleb dataset. We demonstrate that the vanilla triplet loss shows competitive performance compared to classification-based losses, and those trained with our proposed metric learning objective outperform state-of-the-art methods.
1. Introduction
The paper focuses on open-set speaker recognition, where unseen speakers must be represented in a discriminative embedding space. It evaluates competing loss functions under controlled conditions to test whether metric learning can match or exceed classification-based approaches.
- Problem setting: Open-set speaker recognition evaluates identities unseen during training and is framed as learning a discriminative metric-embedding space.Closed-set recognition can be treated as classification because testing identities are predefined in training.
- Prevailing approach: Classification-based training has dominated speaker-embedding research, although softmax does not explicitly optimise embedding similarity.Softmax-trained models have often been combined with PLDA back-ends to generate scoring functions.
- Prevailing approach: Angular-softmax variants improve on vanilla softmax but are sensitive to the scale and margin values used during training.AM-Softmax and AAM-Softmax introduce cosine-margin penalties to increase inter-class variance.
- Metric learning: Metric learning directly optimises distances, targeting small intra-class and large inter-class distances, but pair or triplet selection can be costly and performance-sensitive.Prototypical methods and multiple negatives offer alternatives intended to stabilise metric learning.
- Evaluation rationale: Comparisons across prior studies are difficult because systems vary in architectures, pooling methods, augmentation, and other training details.The paper addresses this comparability problem with over 20,000 GPU-hours of experiments while keeping other training details constant.
2. Training functions
The paper evaluates classification and metric-learning objectives for open-set speaker recognition, including a new angular prototypical variant and several established losses. These objectives differ in how they shape embeddings, form speaker prototypes, and select negatives during training.
- Classification and metric-learning objectives: The experiments include softmax, AM-Softmax, AAM-Softmax, triplet, prototypical, GE2E, and angular prototypical objectives.The angular prototypical loss is introduced as a new angular variant of the prototypical loss.
- Classification objectives: Softmax penalises classification error but does not explicitly enforce intra-class compactness or inter-class separation.AM-Softmax incorporates a cosine margin, while AAM-Softmax uses an additive angular margin between embeddings and class weights.
- Triplet loss: Triplet loss reduces anchor-positive L2 distance and increases anchor-negative distance, using hard negatives sampled from different speakers in the mini-batch.This implementation requires M = 2 utterances from each speaker.
- Prototypical loss: Prototypical training classifies each query using a softmax over distances to speaker prototypes formed from a support set.Choosing M to match the expected test-time shot setting makes the training task exactly match the test scenario.
- GE2E: GE2E forms centroids from all other batch utterances and compares embeddings with all centroids using scaled cosine similarity.The centroid for the query’s class contains one fewer utterance than centroids for other classes.
- Angular prototypical loss: Angular prototypical training reserves one utterance from every class as a query, giving every centroid the same support-set size and enabling exact test-scenario matching.Its cosine-based similarity metric uses learnable scale and bias; the angular loss introduces scale invariance and more stable convergence.
3. Experiments
The experiments compare speaker-recognition loss functions under controlled settings across several network architectures, using VoxCeleb2 for training and disjoint VoxCeleb1 identities for evaluation. Metric-learning objectives perform strongly, with triplet loss competitive against common classification settings and the proposed angular objective exceeding the compared methods.
- Experimental setup: Experiments keep training details identical across objectives and evaluate repeated runs on VoxCeleb1 using mean and standard deviation.The setup uses three independent repetitions and reports EER on the VoxCeleb1 test set.
- Trunk architecture: Thin ResNet-34 reduces computational cost to 1.4 million parameters, compared with 22 million for standard ResNet-34, while using self-attentive pooling.Fast ResNet-34 further reduces computation to less than half that of the Thin ResNet models.
- Loss-function comparison: The common AM-Softmax setting with m = 0.3 and s = 30 is outperformed by vanilla triplet loss.AM-Softmax and AAM-Softmax performance is sensitive to the selected margin and scale.
- Loss-function comparison: Generalised end-to-end and prototypical losses improve over triplet loss by using multiple negatives during training.Prototypical networks perform best when M matches the test scenario, avoiding hyperparameter optimisation.
- Loss-function comparison: The proposed angular objective exceeds all classification-based and metric-learning methods compared in the experiments.The authors report no known work outperforming their method with a similar number of network parameters.
- Batch size: Larger batch sizes improve metric-learning performance by enabling harder negative sampling, whereas no such effect is observed for classification loss.The comparison uses Thin ResNet-34 and EER on the VoxCeleb1 test set.
4. Conclusions
The paper argues for metric learning in speaker recognition based on extensive experiments. GE2E and prototypical networks outperform popular classification-based methods, while the proposed angular prototypical objective outperforms existing training functions.
- Conclusions: GE2E and prototypical networks show superior performance to popular classification-based methods.The conclusion reports this outcome as evidence supporting metric learning for speaker recognition.
- Conclusions: The proposed angular variant of prototypical networks outperforms all existing training functions.The paper also releases a flexible PyTorch trainer for large-scale speaker recognition.