Source-linked AI summary
CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking
Hui Wang, Siqi Zheng, Yafeng Chen, Luyao Cheng, Qian Chen
TL;DR
Speaker verification needs a model that retains strong accuracy without the computational cost and slower inference of ECAPA-TDNN. CAM++ combines D-TDNN with context-aware masking and multi-granularity pooling, and experiments on VoxCeleb and CN-Celeb report superior performance with lower complexity and faster inference than ECAPA-TDNN and ResNet34.
Problem
Speaker verification needs an architecture combining ECAPA-TDNN-level performance with vanilla TDNN-level efficiency for demanding inference and limited-resource settings.
Method
CAM++ uses D-TDNN as its backbone and enhances context-aware masking with multi-granularity pooling to capture global and segment-level context.
Results
CAM++ consistently outperforms ECAPA-TDNN and ResNet34 with lower computational complexity and faster inference across VoxCeleb and CN-Celeb experiments.
Takeaways & Limitations
CAM++ provides an efficient speaker embedding model that improves feature quality while reducing computational cost and inference time.
Abstract
from arXiv · showhide
Time delay neural network (TDNN) has been proven to be efficient for speaker verification. One of its successful variants, ECAPA-TDNN, achieved state-of-the-art performance at the cost of much higher computational complexity and slower inference speed. This makes it inadequate for scenarios with demanding inference rate and limited computational resources. We are thus interested in finding an architecture that can achieve the performance of ECAPA-TDNN and the efficiency of vanilla TDNN. In this paper, we propose an efficient network based on context-aware masking, namely CAM++, which uses densely connected time delay neural network (D-TDNN) as backbone and adopts a novel multi-granularity pooling to capture contextual information at different levels. Extensive experiments on two public benchmarks, VoxCeleb and CN-Celeb, demonstrate that the proposed architecture outperforms other mainstream speaker verification systems with lower computational cost and faster inference speed.
1. Introduction
Speaker verification systems must balance accuracy with computational efficiency and inference speed. CAM++ addresses this challenge by combining a D-TDNN backbone with enhanced context-aware masking and multi-granularity pooling.
- Speaker verification systems transform variable-length utterances into fixed-dimensional embeddings and compare them with a back-end similarity model.
- TDNN-based systems capture local temporal context efficiently, while ECAPA-TDNN improves robustness by expanding context and using Res2Block with squeeze-excitation.
- Accuracy and efficiency are equally important because practical speaker verification requires suitable performance, computational complexity, and inference speed.
- D-TDNN improves accuracy with fewer parameters than vanilla TDNN, while earlier context-aware masking still showed a performance gap against state-of-the-art models.
- CAM++ enhances D-TDNN with lighter masking, multi-granularity pooling, deeper narrower layers, and a two-dimensional convolution front end.
2. System description
CAM++ combines a convolutional front end with a D-TDNN backbone and integrates improved context-aware masking into every D-TDNN layer. Multi-granularity pooling captures global and segment-level context to refine speaker-focused representations while the redesigned architecture targets greater efficiency and accuracy.
- Architecture overview: CAM++ uses a front-end convolution module and a D-TDNN backbone, with context-aware masking integrated into each D-TDNN layer.The front end encodes time-frequency features, which are flattened and passed to the D-TDNN.
- D-TDNN backbone: D-TDNN uses dense feed-forward connectivity and is adopted because it achieves better results with fewer parameters than vanilla TDNN.Each basic unit contains an FNN and a TDNN layer, with direct connections between consecutive layers.
- D-TDNN backbone: The redesigned D-TDNN increases network depth while reducing channel size to improve performance without proportionally increasing computation.The redesign is motivated by evidence that depth can improve speaker-verification performance more than widening the model.
- Context-aware masking: The improved CAM predicts a ratio mask from contextual embeddings and applies it to TDNN features to emphasize speaker characteristics and suppress unrelated content.The mask is inserted into every D-TDNN layer, rather than only at transition layers after D-TDNN blocks.
- Context-aware masking: Multi-granularity pooling combines global and segment-level contextual information because speaker characteristics can vary across speech subsegments.During experiments, segment pooling uses consecutive fixed-length 100-frame segments.
- Architecture overview: The front-end convolution module uses residual two-dimensional convolutions to capture high-resolution time-frequency details before D-TDNN processing.Four residual blocks use 32 channels, with 8x frequency downsampling from the final three blocks.
3. Experiments
Experiments on VoxCeleb and CN-Celeb compare CAM++ with established architectures and isolate the effects of its components. CAM++ combines stronger performance with substantially lower computational cost and faster CPU inference.
- Experimental setup: Experiments use VoxCeleb and CN-Celeb benchmarks with consistent data augmentation and experimental setups across network comparisons.VoxCeleb training uses 5,994 VoxCeleb2 development speakers; evaluation includes VoxCeleb1-O, VoxCeleb1-E, and VoxCeleb1-H.
- Benchmark results: CAM++ consistently outperforms ECAPA-TDNN and ResNet34, achieving 51% fewer parameters and 18% lower EER than ECAPA-TDNN on VoxCeleb-O.The comparison evaluates TDNN, D-TDNN, ECAPA-TDNN, ResNet34, and CAM++ under the same experimental setup.
- Ablation study: 21% and 5% EER improvements on VoxCeleb-O and CN-Celeb, respectively, result from adding CAM with multi-granularity pooling.The ablation evaluates the contribution of individual components to performance improvements.
- Ablation study: 7% relative EER reduction with 44% more parameters is obtained by the prior CAM, whereas improved CAM with global pooling achieves similar EER improvement with an 8% parameter increase.Adding segment pooling and fusing it with global pooling further improves performance without additional parameters.
- Complexity analysis: CAM++ uses half the parameters and FLOPs of ECAPA-TDNN and achieves more than twice its inference speed on single-thread CPU evaluation.Compared with ResNet34, CAM++ has slightly more parameters but significantly fewer FLOPs; RTF is measured on CPU under a single-thread condition.
4. Conclusion
The conclusion presents CAM++ as an efficient speaker embedding model that improves feature quality through context-aware masking and multi-granularity contextual fusion. Across VoxCeleb and CN-Celeb, it achieves superior performance with lower computational complexity and faster inference than ECAPA-TDNN and ResNet34.
- Conclusion: CAM++ is an efficient speaker embedding model for speaker verification that uses context-aware masking to focus on speaker characteristics.Multi-granularity pooling fuses global and segment-level contextual information to generate attention weights.
- Conclusion: Experiments on VoxCeleb and CN-Celeb show superior performance, lower computational complexity, and faster inference than ECAPA-TDNN and ResNet34.The conclusion reports this comparison across two public benchmarks.