TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context

Introduction

• Speaker embeddings convert unconstrained utterances into fixed-length vectors representing speaker identity for speaker recognition and diarization.Source: §1 · p. 1
• The model adopts ContextNet’s combination of local features from 1D depth-wise separable convolutions and global context from Squeeze-and-Excitation layers.Source: §1 · p. 1
• The paper proposes 1D separable depth-wise convolutions instead of full 1D convolutions and adds global average pooling after the squeeze-and-excitation module.This global-context design contrasts with TDNN-based extractors such as x-vector [2] and ECAPA-TDNN [11].Source: §1 · p. 1
• TitaNet-M is half the size of comparable speaker embedding extractors like ECAPA-TDNN or Conformer-based baselines and achieves superior performance in speaker diarization.Source: §1 · p. 1
• The networks are trained end-to-end with angular softmax margin loss, and cosine similarity scores speaker representations without external PLDA or AHC models.The paper contrasts this with systems that train external models such as PLDA and AHC.Source: §1 · p. 1
• Three model sizes are introduced, with architecture scalable in depth and width; width scaling reduces model size with a small change in performance.Evaluation is planned on VoxCeleb1 cleaned speaker-verification trials and diarization datasets AMI (Lapel), AMI (MixHeadset), NIST-SRE-2000, and CH109.Source: §1 · p. 1

2.1. Encoder

• TitaNet uses a ContextNet [10]-style encoder of time-channel separable convolution blocks with channel-attention pooling to form utterance-level speaker representations.The encoder has prologue, mega, and epilogue blocks; mega blocks use depth-wise and pointwise convolutions, residual connections, and global-context Squeeze-and-Excitation layers. Depth and width vary with repeated layers R and filter sizes C.Source: §2.1 · p. 2

2.2. Decoder and Embeddings

• Attentive statistics pooling [9] produces time-independent features S of size B×3072, which the decoder maps through a 192-dimensional layer to speaker-class probabilities.TitaNet extracts the fixed-length 192-dimensional t-vector before the final logits layer, enabling representations from variable-length speech segments.Source: §2.2 · p. 2

2.3. Loss function

• TitaNet is trained end-to-end with additive angular margin loss to optimize cosine distance between speaker embeddings, and uses cosine similarity as the verification and diarization backend.The loss uses predefined margin m and scale s, with θj denoting the angle between final-layer weight Wj and incoming feature xi.Source: §2.3 · p. 2

3. EXPERIMENTS

• TitaNet-S, -M, and -L use 256, 512, and 1024 channels and have 6.4M, 13.4M, and 25.3M parameters, respectively.All three use three repeating layers and kernel filter sizes of 3, 7, 11, and 15.Source: §3 · pp. 2–3

3.1.1. Training Data

• Training combines about 4.8M utterances from 16.6K speakers across VoxCeleb, NIST SRE, Switchboard, Fisher, and Librispeech datasets.Augmentation uses impulse responses, 0.95x and 1.05x speed perturbation, and SpecAugment.Source: §3.1.1 · p. 3

3.1.2. Evaluation Data

• Evaluation uses the VoxCeleb1 cleaned trial for verification and NIST-SRE-2000, AMI Lapel and MixHeadset, and CH109 for diarization.CH109 comprises 109 two-speaker sessions; 11 remaining CHAES sessions serve as development data for CH109 and NIST-SRE-2000.Source: §3.1.2 · p. 3

3.2. Experiment Setup

• The setup avoids a speech activity detector, measures verification with EER and MinDCF and diarization with DER, and uses cosine similarity as the backend.For diarization, speech is windowed at 1.5 seconds with a 0.75-second shift for telephonic speech and 3 seconds with a 1.75-second shift for non-telephonic speech; clustering parameters are auto-tuned without external development data.Source: §3.2 · p. 3

3.3.1. Speaker Verification

• With additive margin angular loss (m=30, s=0.2), TitaNet-L achieves 0.68% EER on the VoxCeleb1 cleaned test trial, outperforming previously reported results.The models show competitive performance even with around 6M parameters, and accuracy varies with model size.Source: Table 2 · p. 3, §3.3.1 · p. 4

3.3.2. Speaker Diarization

• TitaNet outperforms prior state-of-the-art models on AMI-Lapel, AMI-MixHeadset, and CH109, with minor performance differences between the small and medium models.The experiments evaluate oracle-SAD diarization with known and estimated speaker counts; results on NIST-SRE-2000 are an exception to the reported improvements across datasets.Source: §3.3.2 · p. 4, Table 3 · p. 4, Table 4 · p. 4

• TitaNet-L significantly outperforms existing models in speaker verification and diarization tasks.Source: §4 · p. 4
• Half the size of previous state-of-the-art systems, TitaNet-M outperforms them in diarization and achieves competitive verification results.Source: §4 · p. 4
• TitaNet uses 1D depth-wise separable convolutions and global-context squeeze-and-excitation layers with channel-attention pooling to extract fixed-length speaker embeddings.Source: §4 · p. 4