Source-linked AI summary
Replay and Synthetic Speech Detection with Res2net Architecture
Xu Li, Na Li, Chao Weng, Xunying Liu, Dan Su, Dong Yu, Helen Meng
TL;DR
The paper addresses limited generalizability of replay and synthetic-speech countermeasures to unseen spoofing attacks. It introduces Res2Net-based anti-spoofing models with multi-scale feature processing, finding that they outperform ResNet baselines and other state-of-the-art single systems across PA and LA.
Problem
Countermeasures remain difficult to generalize because ASVspoof 2019 evaluation attacks mostly differ from training and development attacks.
Method
The paper applies Res2Net to anti-spoofing, using hierarchical residual-like channel-group connections to create multiple feature scales, and also evaluates SE integration and acoustic features.
Results
Res2Net50 significantly outperforms ResNet34 and ResNet50 on PA and LA, while SE integration further improves results and the best single system outperforms other state-of-the-art single systems.
Takeaways & Limitations
Res2Net with CQT provides the paper’s most promising single-system configuration for both replay and synthetic-speech attack scenarios.
Abstract
from arXiv · showhide
Existing approaches for replay and synthetic speech detection still lack generalizability to unseen spoofing attacks. This work proposes to leverage a novel model structure, so-called Res2Net, to improve the anti-spoofing countermeasure's generalizability. Res2Net mainly modifies the ResNet block to enable multiple feature scales. Specifically, it splits the feature maps within one block into multiple channel groups and designs a residual-like connection across different channel groups. Such connection increases the possible receptive fields, resulting in multiple feature scales. This multiple scaling mechanism significantly improves the countermeasure's generalizability to unseen spoofing attacks. It also decreases the model size compared to ResNet-based models. Experimental results show that the Res2Net model consistently outperforms ResNet34 and ResNet50 by a large margin in both physical access (PA) and logical access (LA) of the ASVspoof 2019 corpus. Moreover, integration with the squeeze-and-excitation (SE) block can further enhance performance. For feature engineering, we investigate the generalizability of Res2Net combined with different acoustic features, and observe that the constant-Q transform (CQT) achieves the most promising performance in both PA and LA scenarios. Our best single system outperforms other state-of-the-art single systems in both PA and LA of the ASVspoof 2019 corpus.
1. INTRODUCTION
ASVspoof 2019 targets replay, text-to-speech, and voice-conversion attacks, with evaluation data dominated by unseen attack configurations. The paper addresses this generalization challenge through Res2Net-based system modeling and evaluates it across physical and logical access.
- ASVspoof 2019 separates replay attacks into physical access and TTS/VC attacks into logical access.
- Evaluation attacks mostly differ from training and development attacks, exposing countermeasures with limited generalizability.
- Existing LCNN- and ResNet-based systems capture spoofing cues but remain limited on unseen attacks, while simply increasing width or depth is inefficient.
- The paper introduces Res2Net, whose cross-channel-group residual connections provide multiple feature scales and reduce model size relative to traditional ResNet models.
- Experiments compare Res2Net with ResNet baselines, examine acoustic features, and target a single system that performs strongly in both PA and LA.
2. APPROACH
Res2Net modifies the ResNet bottleneck block by hierarchically processing channel-group splits, creating multiple receptive-field sizes. The design is combined with squeeze-and-excitation to recalibrate channels for spoofing-relevant information.
- Res2Net block: Res2Net splits post-convolution feature maps into s channel subsets and hierarchically adds prior-group outputs before later 3 × 3 convolutions.The scale dimension s specifies the number of partitions.
- Res2Net block: Hierarchical residual-like connections create multiple receptive-field sizes and feature scales within one block.
- Res2Net block: The block concatenates all channel splits and applies a final 1 × 1 convolution to maintain the residual block’s channel size.
- Res2Net block: Compared with a bottleneck block, the partitioned Res2Net design uses smaller filter groups and decreases the number of model parameters.
- SE-Res2Net block: The SE block assigns channel-wise impact weights by modeling inter-channel dependencies, and SE-Res2Net further improves PA and LA performance.
3. EXPERIMENTAL SETUP
Experiments use ASVspoof 2019’s physical- and logical-access subsets, with unseen attacks reserved mainly for evaluation. Models are trained with multiple acoustic features under a common optimization procedure.
- ASVspoof 2019 experiments cover PA and LA, training on the training partition, selecting models on development data, and evaluating mostly unseen attacks.
- The evaluated acoustic features are log power magnitude spectrogram, LFCC, and constant-Q transform.
- All models use binary cross entropy loss and Adam optimization.
- Models are trained for 20 epochs, and the checkpoint with the lowest development-set EER is evaluated.
4. RESULTS
Res2Net-based systems improve spoofing detection across PA and LA, with SE integration and CQT features providing further gains. The best system compares favorably with reported state-of-the-art single systems, while statistical pooling can overfit.
- Effectiveness of the Res2Net architecture: 37.7% and 42.8% relative EER reductions on PA, and 20.9% and 29.3% on LA, show Res2Net50 outperforming ResNet34 and ResNet50.Res2Net50 also achieves similar gains under t-DCF and reduces model size by 33.8% versus ResNet34 and 16.2% versus ResNet50.
- Effectiveness of the Res2Net architecture: 18.7% and 36.9% relative EER reductions on PA and LA result from adding SE to Res2Net50.SE-Res2Net50 also outperforms SE-ResNet34 and SE-ResNet50 on both access conditions, with similar t-DCF gains.
- Effectiveness of the Res2Net architecture: Statistics pooling improves development-set performance but reduces PA evaluation performance, suggesting over-fitting in ASV anti-spoofing.The LA evaluation improvement is limited compared with the development set.
- Feature engineering: CQT performs best across both PA and LA, whereas Spec performs relatively well on PA but generalizes poorly on LA.The passage attributes poor Spec generalization on LA to its lack of phase information, while many LA spoofing cues occur in phase information.
- Feature engineering: 95.8% and 69.1% relative EER reductions over ASVspoof 2019 baselines are achieved by the CQT-based system on PA and LA, respectively.A three-feature fusion system reaches 0.287% EER and 0.0075 t-DCF on PA, and 1.892% EER and 0.0452 t-DCF on LA.
- Comparison with the state-of-the-art single systems: Only a few systems reach EER below 1.0% on PA or below 4.0% on LA, and rarely perform promisingly on both.The proposed system reduces PA EER by 11.5% relative to the cited top single system and outperforms other SOTA systems on LA.
5. CONCLUSION
The work applies Res2Net to ASV anti-spoofing, finding improved generalizability and efficiency across PA and LA. SE integration and CQT features further strengthen results, and the best single model outperforms other SOTA single systems.
- Conclusion: Res2Net enables multiple feature scales through revised residual blocks, improving generalizability to unseen spoofing attacks.The architecture significantly improves over ResNet34 and ResNet50 in both PA and LA scenarios.
- Conclusion: CQT achieves the most promising results among the evaluated acoustic features for both PA and LA attacks.The conclusion also reports further performance enhancement from SE integration.
- Conclusion: The best single model outperforms other state-of-the-art single systems in both PA and LA scenarios.The stated future direction is exploring loss criteria with better generalization ability.