Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification

1. INTRODUCTION

• Adversarial UDA uses target-domain distribution information, but overly aggressive embedding matching may weaken speaker discrimination; clustering can also produce inaccurate pseudo-labels.Clustering additionally requires determining the number of speakers.Source: §1 · p. 1
• 10.2% EER on CnCeleb without target speaker labels is reported for the best SSDA system, exceeding the state-of-the-art result.Jointly training with source and target labels further reaches 8.86% EER on CnCeleb.Source: §1 · p. 1
• SSDA combines self-supervised learning on unlabeled target data with a source-trained speaker embedding extractor.The approach forms positive pairs from segments in the same utterance and negative pairs from different utterances.Source: §1 · p. 1

2. METHOD

• The method section introduces contrastive learning as the foundation of self-supervised training and previews four objectives for comparison in SSDA experiments.It also covers the SSDA algorithm, system configuration, and training strategies.Source: §2 · p. 1
• The method assumes each utterance contains one speaker, noting that speaker diarization can prepare conversational speech for this setup.Source: §2 · p. 1

2.1. Contrastive Learning

• Two segments from one utterance form a positive pair, while segments from different utterances form a negative pair.The paper motivates this sampling from the continuity of speech and the usual presence of one speaker in a short duration.Source: §2.1 · p. 2

2.2. Contrastive Learning Objectives for Self-Supervised Training

• Segments sampled from the same utterance are treated as the same speaker, and segments from different utterances as different speakers.The objective setup samples M segments from each of N utterances.Source: §2.2 · p. 2

2.2.1. Contrastive Loss

• Contrastive loss minimizes positive-pair distance and maximizes negative-pair distance, using hard negatives and M = 2.Its margin m is set to 4.0.Source: §2.2.1 · p. 2

2.2.2. Triplet Loss

• Triplet loss pulls an anchor toward a positive and pushes it away from a hard negative, with M = 2 and margin m = 4.0.Source: §2.2.2 · p. 2

2.2.3. Angular Prototypical Loss

• Angular prototypical loss pairs a query with a centroid, measures similarity by cosine score, and adds temperature τ to control distribution concentration.The angular loss uses a softmax to classify each query against N centroids.Source: §2.2.3 · p. 2

2.2.4. Generalised End to End Loss (GE2E)

• GE2E uses each segment to calculate centroids and as a query, excluding the query itself when its centroid comes from the same utterance.The cosine similarity also uses temperature τ.Source: §2.2.4 · p. 2

2.3.1. Self-Supervised adaptation training

• Source-pretrained embeddings are adapted on target data using contrastive-learning objectives, with the extractor supervised by Lcl during this adaptation.Figure 1 depicts the self-supervised adaptation training strategy.Source: §2.3.1 · p. 2, Figure 1 · p. 3

2.3.2. Self-Supervised adaptation with joint training

• SSDA-Joint is proposed because self-supervised-only adaptation may overfit target data and weaken source-learned speaker discrimination.Source: §2.3.2 · p. 3
• SSDA-Joint feeds labeled source and unlabeled target data through the extractor simultaneously, applying speaker-classification loss to source embeddings and contrastive loss to target embeddings.The two losses are jointly optimized; Lcla is the speaker-classification loss and λ weights the summed losses.Source: §2.3.2 · p. 3

3.1. Dataset

• VoxCeleb1 and VoxCeleb2 are the English source domain, while Chinese CnCeleb is the target domain; training uses 5,994 source speakers and 800 target speakers, with 200 target speakers for evaluation.VoxCeleb1 is the source evaluation set; CnCeleb evaluation uses 18,024 target pairs and 3,586,776 nontarget pairs, and VoxCeleb evaluation uses Vox1-O, Vox1-E, and Vox1-H. No CnCeleb speaker labels are used in target self-supervised training.Source: §3.1 · p. 3

3.2. Data Preprocessing

• CnCeleb utterances are combined to exceed 5 seconds, leaving 53,288 utterances; audio uses 40-dimensional Fbank features and utterances under 400 frames are discarded.Features use a 25 ms window, 10 ms frame shift, energy-based silence removal, and sliding-window cepstral mean with a 300-frame window.Source: §3.2 · p. 3

3.3. System Configuration

• The system uses a 256-dimensional ResNet-based r-vector extractor [4] and cosine similarity to score trials.ProtoLoss and GE2E use M = 2, τ = 32; AAM margins are 0.2 for source and 0.15 for target, and SSDA-Joint uses λ = 1.Source: §3.3 · p. 3

4.1. Comparison between Supervised and Self-Supervised Baselines.

• With CnCeleb speaker labels, supervised training outperforms self-supervised training, while the best self-supervised result is close; the VoxCeleb-trained model is the later baseline.The authors attribute the VoxCeleb model’s advantage over CnCeleb-trained models mainly to the larger source dataset.Source: Table 2 · p. 3, §4.1 · pp. 3–4

4.2.1. SSDA training result

• SSDA substantially outperforms target-only self-supervised training but remains below the strong baseline, with overfitting and loss of speaker discrimination offered as possible explanations.The comparison underscores the importance of source-domain pretraining.Source: §4.2.1 · p. 4

4.2.2. SSDA-Joint training result

• 15.7% relative improvement in EER over baseline: SSDA-Joint with ProtoLoss exceeds the reported state-of-the-art on CnCeleb without target speaker labels.Most contrastive losses in SSDA-Joint also outperform the strong baseline.Source: §4.2.2 · p. 4

4.2.3. Evaluation on the source domain dataset

• SSDA-Joint performs better than the original source-domain model on VoxCeleb1; the authors suggest joint target training may regularize the source task.The evaluation uses three official VoxCeleb1 trials.Source: Table 4 · p. 4, §4.2.3 · p. 4

4.2.4. Comparison with fully supervised-joint training results

• The gap between SSDA-Joint and fully supervised joint training is small, with minor target-domain performance degradation and reduced labeling cost.The supervised-joint result is treated as an upper bound in these experiments.Source: Table 5 · p. 4, §4.2.4 · p. 4

• SSDA combines target-domain self-supervised information with source-domain speaker discrimination, reporting 10.2% EER on CnCeleb without target speaker labels and 8.86% EER with jointly used source and target labels.The unlabeled-target SSDA result is reported to exceed the state of the art.Source: §5 · p. 4