Source-linked AI summary
Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification
Youngmoon Jung, Younggwan Kim, Hyungjun Lim, Yeunju Choi, Hoirin Kim
TL;DR
Text-independent speaker verification needs speaker representations for unconstrained speech, while pooling must map variable-length segments to fixed-dimensional embeddings. SPE applies LDE to increasingly fine feature-map regions and aggregates their embeddings, while ring loss augments the primary loss to normalize embeddings through a convex formulation. The model yielded relative improvements of 25.4% and 8.4% over the i-vector system and ResNet-34 + SAP, respectively.
Problem
Text-independent speaker verification needs speaker representations for unconstrained speech, while pooling must map variable-length segments to fixed-dimensional embeddings.
Source: §1 · p. 1, §3.1 · p. 2Method
SPE applies LDE to increasingly fine feature-map regions and aggregates their embeddings, while ring loss augments the primary loss to normalize embeddings through a convex formulation.
Source: §1 · p. 1, §4.1 · p. 3, §4.2 · p. 3Results
The model yielded relative improvements of 25.4% and 8.4% over the i-vector system and ResNet-34 + SAP, respectively.
Source: §6.3 · p. 4Takeaways & Limitations
On VoxCeleb1, the system using SPE and ring loss achieved better performance than the state-of-the-art i-vector and d-vector baselines.
Source: §7 · p. 4
Abstract
from arXiv · showhide
In this paper, we propose a new pooling method called spatial pyramid encoding (SPE) to generate speaker embeddings for text-independent speaker verification. We first partition the output feature maps from a deep residual network (ResNet) into increasingly fine sub-regions and extract speaker embeddings from each sub-region through a learnable dictionary encoding layer. These embeddings are concatenated to obtain the final speaker representation. The SPE layer not only generates a fixed-dimensional speaker embedding for a variable-length speech segment, but also aggregates the information of feature distribution from multi-level temporal bins. Furthermore, we apply deep length normalization by augmenting the loss function with ring loss. By applying ring loss, the network gradually learns to normalize the speaker embeddings using model weights themselves while preserving convexity, leading to more robust speaker embeddings. Experiments on the VoxCeleb1 dataset show that the proposed system using the SPE layer and ring loss-based deep length normalization outperforms both i-vector and d-vector baselines.
1. Introduction
Text-independent speaker verification uses unconstrained speech, but turning variable-length speech into speaker embeddings and normalizing those embeddings remain important design choices. The paper introduces spatial pyramid encoding (SPE) and ring loss-based deep length normalization for this task.
1. Introduction
- TI-SV verifies a claimed identity from unconstrained speech, unlike text-dependent verification, which requires fixed spoken content.Source: §1 · p. 1
- Adding an ASR-DNN to an i-vector system improved the conventional GMM-UBM i-vector system, but increased computational complexity and required transcribed training data.Source: §1 · p. 1
- D-vector systems pool frame-level features into utterance-level embeddings, and proposed pooling methods include statistics, attentive statistics, spatial pyramid pooling, and LDE.These methods were reported to improve performance over simple average pooling.Source: §1 · p. 1
- SPE extracts embeddings from increasingly fine feature-map regions using LDE and aggregates them into a speaker representation, while ring loss provides convex deep length normalization.The paper reports that ring loss-based normalization performs better than L2-constraint-based normalization.Source: §1 · p. 1
2. d-vector systems
D-vector speaker verification pools frame-level features into utterance-level representations, with systems differing in their training losses. The section outlines the trade-offs among softmax, triplet, center, and angular softmax objectives.
2. d-vector systems
- Softmax loss trains a speaker classifier and encourages embedding separability, but is described as insufficient for learning large-margin discriminative embeddings.Source: §2 · p. 1
- Triplet loss promotes same-speaker compactness and different-speaker separability, but careful triplet selection is time-consuming and performance-sensitive.Source: §2 · p. 2
- Center loss minimizes distances to class centroids, while A-softmax introduces a controllable angular margin between classes.The passage notes that larger A-softmax margin parameter m imposes a more stringent embedding constraint.Source: §2 · p. 2
- D-vector systems pool frame-level features into an utterance-level representation, length-normalize enrollment embeddings, then score them against test embeddings using cosine distance or PLDA.Source: §2 · p. 2
3. Prior works
The prior works section describes learnable dictionary encoding (LDE) as a pooling layer for variable-length features and reviews an L2-based constraint for normalizing embeddings during training.
3.1. Learnable dictionary encoding layer
- LDE learns a codebook of C codewords and soft-assigns feature residuals to form encodings that are concatenated into a fixed-length representation.A fully connected layer projects the concatenated representation to the final embedding.Source: §3.1 · p. 2
3.2. L2-constraint based deep length normalization
- L2-constraint-based deep length normalization places an L2-normalization layer followed by a scale layer after the pooling layer, constraining embeddings to a fixed-radius hypersphere.The scale layer sets the radius using parameter α.Source: §3.2 · p. 2
4. Proposed approaches
The proposed methods combine multiscale temporal encoding with learned dictionary pooling and use ring loss to normalize speaker embeddings. SPE preserves local and global temporal information, while ring loss avoids the hard norm constraint used by an earlier approach.
4.1. Spatial pyramid encoding layer
- SPE divides ResNet feature maps into temporal bins at two scales, extracts LDE embeddings from each bin, and concatenates them into a 256-dimensional speaker embedding.The two pyramid levels are {1 × 1, 1 × 4}, giving five bins; the LDE layer is shared across bins.Source: §4.1 · p. 2
- Unlike SPP's average pooling, SPE uses LDE to retain temporal information in local and global feature scales.The paper describes SPE as extending both SPP and LDE.Source: §4.1 · p. 3
4.2. Ring loss-based deep length normalization
- Ring loss augments a primary loss to normalize embedding lengths while preserving convexity, avoiding the non-convex hard normalization approach described for L2 constraints.The target norm R is learned during training, and the objective is L = L_P + λL_R.Source: §4.2 · p. 3
- The ring-loss term encourages embedding norms toward the learned target R rather than enforcing them through hard normalization.The paper initializes R using the average embedding norm from the first training iteration.Source: §4.2 · p. 3
5. Experimental setup
The experiments train and test on VoxCeleb1, with 1,211 development speakers and 40 held-out test speakers. The setup specifies acoustic features, training and evaluation procedures, and comparison conditions for pooling and length-normalization methods.
5.1. Datasets
- VoxCeleb1 contains over 140,000 utterances from 1,251 celebrities; 1,211 speakers are in development and 40 are reserved for testing.The dataset is described as text-independent and recorded in real-world conditions.Source: §5.1 · p. 3
- Performance is reported using EER and minimum DCF at Ptarget = 0.01 and Ptarget = 0.001, with cosine-distance trial scoring.Source: §5.1 · p. 3
5.2. Implementation details
- Systems use 64-dimensional Fbank features with 25 ms frames and sliding-window mean normalization up to 3 s; VAD and data augmentation are omitted.Source: §5.2 · p. 3
- Training crops or extends utterances to a randomly selected 300–500 frames, while testing evaluates the full utterance; embeddings are 256-dimensional.When deep length normalization is used during training, no additional test-time length normalization is needed.Source: §5.2 · p. 3
- Table 2 compares pooling methods under softmax loss with ring loss, while Table 3 compares deep length-normalization methods using temporal average pooling.Table 2 distinguishes 1D and 2D spatial pyramid divisions; Table 3 identifies softmax, A-softmax, L2-constraint, and ring-loss conditions.Source: Table 2 · p. 4, Table 3 · p. 4
6. Results
Across the reported results, 1D-SPE performs best among the pooling methods, ring-based deep length normalization outperforms the L2-constraint approach, and the proposed system surpasses the compared VoxCeleb1 baselines.
6.1. Comparison of pooling methods
- 4.20% EER, DCF 10−2 = 0.422, and DCF 10−3 = 0.528 were the best pooling results; 1D-SPE improved relative EER over SPP by 6.7% and LDE by 3.0%.Both SPP and LDE also outperformed TAP, while 1D-SPD performed better than 2D-SPD in both SPP and SPE.Source: §6.1 · p. 4
6.2. Comparison of deep length normalization methods
- 32.8% relative EER improvement came from softmax + ring over softmax alone; A-softmax + ring also beat L2-Cons at R = 30 by 6.0%.The best normalization result used A-softmax with ring loss: EER = 4.41%, DCF 10−2 = 0.451, and DCF 10−3 = 0.559.Source: §6.2 · p. 4
6.3. Comparison with recent methods
- 4.03% EER, 0.402 DCF 10−2, and 0.492 DCF 10−3 were obtained by the proposed system, which improved over i-vector by 25.4% and ResNet-34 + SAP by 8.4% in relative EER.The system uses SPE with A-softmax loss and ring loss.Source: §6.3 · p. 4, Table 4 · p. 4
7. Conclusions
The conclusion presents SPE and ring-loss deep length normalization as improvements over prior pooling and normalization approaches, with better VoxCeleb1 performance than the cited i-vector and d-vector baselines.
7. Conclusions
- On VoxCeleb1, the SPE-and-ring-loss system outperformed state-of-the-art i-vector and d-vector baselines.The conclusion identifies automatic division of CNN feature maps in SPE as future work.Source: §7 · p. 4
- SPE achieved better results than the LDE and SPP methods.Source: §7 · p. 4
- Ring-loss-based deep length normalization performed better than the existing L2-constraint approach.Source: §7 · p. 4