Source-linked AI summary
TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context
Nithin Rao Koluguri, Taejin Park, Boris Ginsburg
TL;DR
Speaker representation systems must convert variable-length speech into fixed-length embeddings for verification and diarization. TitaNet addresses this with depth-wise separable convolutions, global-context Squeeze-and-Excitation layers, and channel-attention pooling, achieving strong results across both tasks while supporting smaller models.
Problem
Speaker recognition and diarization require fixed-length speaker embeddings extracted from unconstrained, variable-length speech, motivating improved embedding architectures.
Method
TitaNet combines 1D depth-wise separable convolutions, global-context Squeeze-and-Excitation layers, channel-attention pooling, end-to-end angular-margin training, and cosine similarity.
Results
TitaNet achieves state-of-the-art verification and diarization performance, while TitaNet-M is half the size of comparable extractors and outperforms them on diarization.
Takeaways & Limitations
The scalable architecture supports strong speaker verification and diarization performance across model sizes, including a smaller TitaNet-M model.
Abstract
from arXiv · showhide
In this paper, we propose TitaNet, a novel neural network architecture for extracting speaker representations. We employ 1D depth-wise separable convolutions with Squeeze-and-Excitation (SE) layers with global context followed by channel attention based statistics pooling layer to map variable-length utterances to a fixed-length embedding (t-vector). TitaNet is a scalable architecture and achieves state-of-the-art performance on speaker verification task with an equal error rate (EER) of 0.68% on the VoxCeleb1 trial file and also on speaker diarization tasks with diarization error rate (DER) of 1.73% on AMI-MixHeadset, 1.99% on AMI-Lapel and 1.11% on CH109. Furthermore, we investigate various sizes of TitaNet and present a light TitaNet-S model with only 6M parameters that achieve near state-of-the-art results in diarization tasks.
NVIDIA, USA
The paper identifies speaker verification, speaker embedding, and diarization as relevant research topics.
- Speaker verification, speaker embedding, t-vectors, context, and diarization are identified as key topics.
1. INTRODUCTION
The introduction positions TitaNet as a scalable, text-independent speaker embedding extractor for verification and diarization. It combines depth-wise separable convolutions, global context, and channel attention while targeting strong performance with reduced model size.
- TitaNet extracts speaker embeddings for speaker verification and diarization from unconstrained, variable-length speech.Speaker embeddings represent speaker identity using fixed-length vectors.
- The model combines 1D depth-wise separable convolutions with global context from Squeeze-and-Excitation layers.The architecture is adapted from ContextNet, which combines local convolutional features with global context.
- TitaNet-M is half the size of comparable extractors and achieves superior speaker diarization performance.
- The models are trained end-to-end with angular softmax margin loss and use cosine similarity as the backend.This avoids training external PLDA and AHC models.
- The architecture scales in depth and width, with width scaling reducing model size while causing only a small performance change.The paper evaluates verification on VoxCeleb1 and diarization on AMI, NIST-SRE-2000, and CH109.
2. MODEL ARCHITECTURE
TitaNet uses a ContextNet-like encoder with depth-wise separable convolutions and Squeeze-and-Excitation global context, followed by attentive statistics pooling and a decoder that produces fixed-length t-vectors.
- The encoder extracts acoustic features with a ContextNet architecture and passes them to an attentive pooling layer.The pooling layer computes attention features across channel dimensions for utterance-level speaker representations.
- Encoder: TitaNet’s encoder uses prologue, repeated mega, and epilogue blocks built from time-channel separable convolution modules.The architecture is parameterized by blocks, repetitions per block, and convolutional filters.
- Encoder: Each separable module combines depth-wise and pointwise convolutions, with repeated layers residually connected to Squeeze-and-Excitation layers using global average pooling.
- Decoder: Attentive statistics pooling computes time-independent features across channels, producing an intermediate representation of size B×3072.
- Decoder: The decoder maps intermediate features through linear layers, and 192-dimensional t-vectors are extracted before the final class logits.This converts variable-length speech segments into fixed-length representations.
- Loss and backend: The model is trained end-to-end with additive angular margin loss and uses cosine similarity for verification and diarization backends.The margin and scale are predefined hyperparameters in the loss.
3. EXPERIMENTS
The experiments evaluate scalable TitaNet models for speaker verification and diarization across VoxCeleb1, AMI, NIST-SRE-2000, and CH109 datasets. TitaNet-L achieves state-of-the-art verification performance, while smaller models remain competitive for diarization.
- Model configurations: TitaNet-S, TitaNet-M, and TitaNet-L use 256, 512, and 1024 channels, with 6.4M, 13.4M, and 25.3M parameters, respectively.
- Evaluation datasets: The evaluation uses the VoxCeleb1 cleaned trial file for verification and NIST-SRE-2000, AMI, and CH109 datasets for diarization.
- Evaluation metrics: Speaker verification is measured with EER and MinDCF, while verification and diarization use cosine similarity as the backend.
- Speaker verification: 0.68% EER: TitaNet-L achieves state-of-the-art performance on the VoxCeleb1 cleaned test trial file.
- Speaker diarization: TitaNet models outperform previous state-of-the-art models on AMI-Lapel, AMI-MixHeadset, and CH109 diarization datasets.
- Speaker diarization: Larger TitaNet models provide no major diarization improvement over smaller models, unlike the stronger scaling effect observed for verification.
4. CONCLUSION
The paper presents TitaNet as a speaker representation model combining global-context squeeze-and-excitation layers, channel attention pooling, and depth-wise separable convolutions. Its medium model outperforms prior systems in diarization while the large model achieves strong verification and diarization performance.
- TitaNet extracts fixed-length speaker embeddings using global-context squeeze-and-excitation layers combined with channel attention pooling.
- The architecture uses 1D depth-wise separable convolutions for speaker embedding models.
- TitaNet-M, half the size of previous state-of-the-art systems, outperforms them in diarization while achieving competitive verification results.
- TitaNet-L significantly outperforms existing models in speaker verification and diarization tasks.
- The models’ implementation and pre-trained checkpoints are available through the NVIDIA NeMo toolkit.