Cross-Lingual Speaker Verification with Self-Supervised Pre-Trained Models

• Language mismatch confounds speaker characteristics with language-specific acoustics, often degrading accuracy when enrollment and verification use different languages.TidyVoice2026 benchmarks speaker verification under this mismatch.Source: §1 · p. 1
• The approach builds on self-supervised models trained on multilingual speech, whose representations capture general acoustic, phonetic, and semantic information.The paper proposes these representations as a foundation for language-agnostic speaker verification rather than training from scratch on limited, language-specific data.Source: §1 · p. 1
• The system uses w2v-BERT 2.0 as a front end, aggregates layer-wise outputs, pools utterance-level features, and projects them into a fixed-dimensional speaker embedding.Source: §1 · p. 1
• 2.21% EER on tv26 eval-A and 2.99% EER on tv26 eval-U are reported for the proposed systems.Source: §1 · p. 1
• The paper analyzes pre-trained model selection, feature aggregation, and training strategies, and presents practical guidance for model selection and training.Source: §1 · p. 1

2.1. Training dataset

• Training combines challenge-specific TidyVoiceX Train (3,666 speakers across 40 languages) with VoxCeleb2, VoxBlink2, CN-Celeb, WenetSpeech, and 3D-Speaker.The collection also includes VoxBlink2 (111,284 speakers), a filtered WenetSpeech subset (35,285 speakers), and 3D-Speaker (10,000 Chinese speakers).Source: §2.1 · pp. 1–2

2.2. Development dataset

• TidyVoiceX Dev contains 808 speakers across 40 languages and 12 million verification pairs, and supports model selection, threshold adjustment, and score calibration.Source: §2.2 · p. 2

2.3. Evaluation dataset

• Evaluation uses tv26 eval-A for partial mismatch and tv26 eval-U for full generalization across 38 languages unseen in training and development.Source: §2.3 · p. 2

3.1.1. PTM-based speaker verification systems

• The PTM system adapts hidden representations from every SSL layer, fuses them through multi-scale aggregation, pools utterance features, and projects a 256-dimensional speaker embedding.The network is trained with Additive Angular Margin Softmax Loss.Source: §3.1.1 · p. 2

3.1.2. ReDimNet systems

• ReDimNet-B5 and ReDimNet-B6 serve as comparison and fusion systems, producing 192-dimensional speaker embeddings trained with SphereFace2-C loss.Source: §3.1.2 · p. 2

3.2. Training paradigm and details

• PTM training progresses from a frozen SSL backbone, to full-backbone fine-tuning, then in-domain large-margin fine-tuning on TidyVoiceX Train.The stages last 60, 20, and 6 epochs, respectively; the final stage uses 6-second segments.Source: §3.2 · pp. 2–3
• ReDimNet uses full-set pre-training followed by six-epoch large-margin fine-tuning on TidyVoiceX Train with 6-second segments.Source: §3.2 · p. 3
• Training applies no speed perturbation; noise augmentation has probability 0.6, with AdamW for PTM systems and Nesterov SGD for ReDimNet.Source: §3.2 · p. 3

4.1. Results

• After QMF calibration, w2v-BERT 2.0 achieved EERs of 2.73% on eval-A and 2.84% on eval-U, while ReDimNet-B6 achieved 2.52% and 3.43%, respectively.The w2v-BERT 2.0 system had lower EER on eval-U than the ReDimNet systems, which the authors describe as stronger generalization to unseen languages.Source: §4.1 · p. 3
• 2.21% EER on tv26 eval-A and 2.99% EER on tv26 eval-U were achieved by the fused system, the best-performing submitted system.The fusion averages calibrated scores from all QMF-enhanced systems.Source: §4.1 · p. 3, Table 1 · p. 4

4.2. Ablation studies

• Among tested pre-trained models, w2v-BERT 2.0 achieved the best TidyVoiceX Dev EER, at 2.09%.The model has 580M parameters and was pre-trained on unlabeled speech covering 143 languages.Source: §4.2 · p. 3, Table 2 · p. 4
• Channel Concatenation achieved the lowest EER among aggregation strategies, while Hierarchical Cross-Attention Fusion achieved the best minDCF.Performance differences were relatively small; Channel Concatenation adds no trainable parameters and was selected for the final submission.Source: Table 3 · p. 4, §4.2 · p. 4
• The three-stage training ablation improved development EER from 1.54% with a frozen front end to 1.14% after full fine-tuning and 1.03% after in-domain LM-FT.Across the same stages, minDCF decreased from 0.66 to 0.63 and then 0.61.Source: Table 4 · p. 4, §4.2 · p. 4

5. Conclusions

• The final fused system achieved EERs of 2.21% on tv26 eval-A and 2.99% on tv26 eval-U.The conclusion also reports that, with the same training data, the w2v-BERT 2.0-based system performs better on evaluation sets containing unseen languages than ReDimNet.Source: §5 · p. 4
• The framework uses w2v-BERT 2.0, compares multi-scale feature aggregation strategies, and applies a three-stage training paradigm.Source: §5 · p. 4
• The authors frame the framework’s goal as building more generalizable, language-agnostic speaker-recognition systems from multilingual self-supervised representations.Source: §5 · p. 4

6. Generative AI Use Disclosure

• Generative AI assistants were used solely for manuscript language editing and polishing; the authors developed the scientific content.Source: §6 · p. 5