Source-linked AI summary
EDITnet: A Lightweight Network for Unsupervised Domain Adaptation in Speaker Verification
Jingyu Li, Wei Liu, Tan Lee
TL;DR
Language mismatch can degrade speaker verification across languages, while target-domain speaker labels may be unavailable. EDITnet uses a conditional variational auto-encoder and self-supervised loss to transfer embeddings without fine-tuning the fixed extractor. On CN-Celeb, it reports a 30% relative improvement over the non-adapted system with ECAPA-512, with gains also reported across other embedding extractors.
Problem
Language mismatch between training and test speech challenges speaker verification, and target-domain speaker labels may be unavailable in practical or low-resource settings.
Source: §1 · p. 1Method
EDITnet uses a conditional variational auto-encoder and self-supervised loss to transfer target-domain speaker embeddings toward the source domain while keeping the embedding extractor fixed.
Source: §1 · p. 1, §2.2 · p. 2Results
30% relative improvement over the non-adapted system on CN-Celeb with ECAPA-512; the paper also reports consistent gains across different embedding extractors.
Source: §4.2 · p. 4, §5 · p. 4Takeaways & Limitations
EDITnet mitigates language-related domain mismatch and improves transferred-embedding performance across the evaluated embedding extraction models.
Source: §5 · p. 4The cosine-loss setup assumes most samples in a mini-batch come from different speakers, and optimizes the loss only for angles below 90°.
Source: §2.2 · p. 3
Abstract
from arXiv · showhide
Performance degradation caused by language mismatch is a common problem when applying a speaker verification system on speech data in different languages. This paper proposes a domain transfer network, named EDITnet, to alleviate the language-mismatch problem on speaker embeddings without requiring speaker labels. The network leverages a conditional variational auto-encoder to transfer embeddings from the target domain into the source domain. A self-supervised learning strategy is imposed on the transferred embeddings so as to increase the cosine distance between embeddings from different speakers. In the training process of the EDITnet, the embedding extraction model is fixed without fine-tuning, which renders the training efficient and low-cost. Experiments on Voxceleb and CN-Celeb show that the embeddings transferred by EDITnet outperform the un-transferred ones by around 30% with the ECAPA-TDNN512. Performance improvement can also be achieved with other embedding extraction models, e.g., TDNN, SE-ResNet34.
1. Introduction
Language mismatch challenges speaker verification because languages in practical use are unconstrained, while target-domain speaker labels may be unavailable and fine-tuning can be costly or overfit limited data. EDITnet addresses this setting by transferring embeddings without fine-tuning the fixed extractor or requiring target-domain speaker labels.
1. Introduction
- Language mismatch between training and test speech challenges speaker verification, and is common because speakers’ languages are unconstrained.Domain mismatch can also reflect background noise and channel distortion.Source: §1 · p. 1
- Unsupervised domain adaptation is relevant when target-domain speaker labels are unavailable, while fine-tuning can be time-consuming and may overfit limited data.The paper notes low-resourced languages as one setting where speaker labels may be unavailable.Source: §1 · p. 1
- Prior approaches align domain distributions explicitly with statistics such as covariance or feature means, or implicitly through adversarial training.Other work considered Gaussian speaker representations and self-supervised domain adaptation.Source: §1 · p. 1
- EDITnet uses a conditional variational auto-encoder to transfer target-domain speaker embeddings into the source domain, with a self-supervised loss to increase separation between speakers.The embedding extractor remains fixed, so the approach does not require target-domain speaker labels or extractor fine-tuning.Source: §1 · p. 1
2. Methodology
The methodology describes conditional variational auto-encoding and applies it to transfer speaker embeddings between target and source domains. EDITnet combines reconstruction and latent-prior losses with self-supervised cosine repulsion, without supervised speaker classification.
2.1. Conditional VAE
- A conditional VAE conditions the latent-variable distribution on a label; here, the condition specifies whether the input embedding is from the target or source domain.The VAE objective includes reconstruction and latent-prior regularization terms.Source: §2.1 · p. 2
2.2. Architecture of the EDITnet
- EDITnet transfers target-domain embeddings toward the source domain using a conditional variational auto-encoder, with domain labels conditioning the latent distributions.Target and source embeddings use distinct one-hot condition labels and their latent priors are modeled as N(priortar, I) and N(priorsrc, I), respectively.Source: §2.2 · p. 2
- The architecture combines reconstruction and KL-divergence losses with a self-supervised cosine loss, and requires no supervised speaker classification.The reconstruction term is an L2 distance; the KL terms compare each domain’s latent distribution with its conditional prior.Source: §2.2 · pp. 2–3
2. Methodology
- The training-data summary reports that CN-Celeb contains much less data than VoxCeleb 2.The experiments treat VoxCeleb as the source domain and CN-Celeb as the target domain.Source: Table 2 · p. 3
2.2. Architecture of the EDITnet
- The cosine loss repels transferred embeddings, under the assumption that most randomly sampled mini-batch examples come from different speakers.The repulsion also applies between transferred target embeddings and source embeddings; the loss is optimized only for angles below 90°.Source: §2.2 · p. 3
3. Experimental setup
The setup uses VoxCeleb 2 development data as the source and CN-Celeb as the target, with transfer evaluated on the official CN-Celeb test set. EDITnet is trained on embeddings from a fixed ECAPA-TDNN512 extractor.
3.1. Datasets
- VoxCeleb 2 development data with 5,994 speakers trains the source extractor, CN-Celeb is the target, and evaluation uses 3,604,800 official CN-Celeb test utterance pairs.The source extractor is evaluated on the VoxCeleb 1 test set.Source: §3.1 · p. 3
3.2. Embedding extraction
- The fixed ECAPA-TDNN512 extractor produces 256-dimensional embeddings and achieves 1.12% EER on the VoxCeleb 1 test set.Its parameters remain fixed during EDITnet training.Source: §3.2 · p. 3
3.3. EDITnet training
- EDITnet training samples 256 utterances per domain per step, crops two-second segments, and normalizes embeddings using domain-specific channel means and standard deviations.Training lasts 20 CN-Celeb epochs; evaluation averages cosine-similarity scores across two-second segments with one-second overlap.Source: §3.3 · p. 3
4. Results
EDITnet improves speaker-verification performance on CN-Celeb, while its component ablations and tests with other embedding models show where its gains hold. The results also describe challenges for supervised adaptation and source-to-target transfer.
4.1. Baselines
- Directly applying Vox.2-trained ECAPA-512 to CN-Celeb yields 17.78% EER, while supervised fine-tuning performs significantly better than no adaptation.The authors note over-fitting during supervised training with insufficient data and judge vanilla supervised training inadequate in this limited-data setting.Source: §4.1 · p. 3, Table 3 · p. 4
4.2. Performance of EDITnet
- 30% relative improvement over the non-adapted system, with EDITnet reaching 12.06% EER on the CN-Celeb test set.This is the reported result for ECAPA-512.Source: §4.2 · p. 4, Table 3 · p. 4
- Removing normalization slightly worsens performance to 12.52% EER; the authors also report that normalization stabilizes EDITnet training.Normalization is applied before the CVAE and is intended to remove differences in embedding means and standard deviations.Source: §4.2 · p. 4
- Without latent-variable mean transfer, EER rises to 13.65%; without the self-supervised cosine loss, it rises to 15.96%.Without the conditional prior, all domains approximate a global N(0, I) prior; replacing the CVAE with two fully connected layers also worsens performance when the cosine loss is retained.Source: §4.2 · p. 4
4.3. Adaptation on different models
- EDITnet is evaluated with TDNN and SE-ResNet embeddings in addition to ECAPA-512; the comparison includes statistics-based transfer methods.The supplied passages state that EDITnet performs best among the compared methods, but do not provide the individual Table 4 EER values.Source: §4.3 · p. 4, Table 4 · p. 4
- EDITnet gives the best performance among the compared transfer methods for all three embedding models, and SERes34 reaches 9.60% EER versus 10.20% for SSDA.The SSDA comparison uses a similar corpus setting and requires joint training with source-domain speaker classification and fine-tuning the full backbone.Source: §4.3 · p. 4, Table 4 · p. 4
5. Conclusions
EDITnet is a lightweight embedding-level adaptation network that does not require speaker identity information. It mitigates language-related domain mismatch, improves performance over original embeddings, and gains consistently across embedding extraction models while outperforming statistics-based transfer methods.
5. Conclusions
- EDITnet transfers target-domain speaker embeddings into the source domain using a CVAE and self-supervised learning, without requiring speaker identity information.The conclusion describes the approach as a lightweight network for fast adaptation at the embedding level.Source: §5 · p. 4
- The transferred embeddings show notable improvement over original embeddings and consistent gains across different embedding extraction models.The conclusion reports that EDITnet also outperforms statistics-based transfer methods.Source: §5 · p. 4