Source-linked AI summary
The Vicomtech Audio Deepfake Detection System based on Wav2Vec2 for the 2022 ADD Challenge
Juan M. Martín-Doñas, Aitor Álvarez
TL;DR
Deepfake speech can fool automatic speaker verification systems, creating a need for reliable spoofed-audio detection across challenging conditions. This paper combines pre-trained wav2vec2 representations from multiple transformer layers with a downstream classifier adapted through data augmentation. It reports first and fourth place in the two 2022 ADD tracks and competitive results on ASVspoof 2021.
Problem
Deepfake speech can fool automatic speaker verification systems, motivating reliable countermeasures for spoofed speech.
Source: §1 · p. 1Method
The system uses a pre-trained wav2vec2 model's representations from different transformer layers as input to a downstream classifier adapted with data augmentation.
Source: §1 · p. 1, §2.2 · p. 2Results
Takeaways & Limitations
The single system outperformed other methods on ASVspoof 2021 DF and achieved competitive LA results, with potential computational-resource savings in practical applications.
Source: §4.2 · p. 4
Abstract
from arXiv · showhide
This paper describes our submitted systems to the 2022 ADD challenge withing the tracks 1 and 2. Our approach is based on the combination of a pre-trained wav2vec2 feature extractor and a downstream classifier to detect spoofed audio. This method exploits the contextualized speech representations at the different transformer layers to fully capture discriminative information. Furthermore, the classification model is adapted to the application scenario using different data augmentation techniques. We evaluate our system for audio synthesis detection in both the ASVspoof 2021 and the 2022 ADD challenges, showing its robustness and good performance in realistic challenging environments such as telephonic and audio codec systems, noisy audio, and partial deepfakes.
1. INTRODUCTION
Deepfake speech can threaten automatic speaker verification, motivating countermeasures for increasingly challenging audio conditions. The paper presents a wav2vec2-based system for two 2022 ADD tracks and reports first and fourth place, respectively.
1. INTRODUCTION
- Deepfake speech can fool automatic speaker verification systems, prompting research into reliable spoofed-speech countermeasures.The ASVspoof series comprises biannual challenges focused on antispoofing countermeasures for verification systems.Source: §1 · p. 1
- Prior work examined noisy and reverberant audio, telephonic transmission, commercial compression, and partially spoofed audio.These efforts provide context for the 2022 ADD challenge's focus on detection in complex environments.Source: §1 · p. 1
- ADD 2022 targets low-quality fake audio, partially fake audio, and an audio fake game across three challenge tracks.Track 1 includes background noise and disturbances; track 2 hides small fake speech segments in real audio.Source: §1 · p. 1
- The system ranked first in ADD 2022 track 1 and fourth in track 2 among participants.The approach uses wav2vec2 representations from different transformer layers and adapts a downstream classifier with data augmentation.Source: §1 · p. 1
- A pre-trained wav2vec2 feature extractor and downstream classifier detect spoofed audio using representations from different transformer layers.The classifier is adapted to the application scenario with data augmentation techniques.Source: §1 · p. 1
2. WAV2VEC2-BASED AUDIO DEEPFAKE DETECTION APPROACH
The proposed detector combines a pre-trained wav2vec2 feature extractor with a downstream classifier. It uses representations across transformer layers to produce a genuine-versus-spoof score.
2. WAV2VEC2-BASED AUDIO DEEPFAKE DETECTION APPROACH
- The system pairs wav2vec2 speech representations with a classifier that scores input audio as genuine or spoof.The feature extractor obtains encoded representations, which are processed by the classification model.Source: §2 · p. 2
2.1. Wav2vec2 as Feature Extractor
- The pre-trained wav2vec2 model produces contextualized speech representations from 24 transformer layers, following convolutional features extracted every 20 ms.The evaluated Large models have 300M parameters and use 53 or 128 languages; the feature encoder has a 25 ms receptive field.Source: §2.1 · p. 2
2.2. Classification Model
- The classifier combines hidden representations across transformer layers with trainable normalized weights, then pools utterance statistics and scores similarity to a genuine-speech direction.Temporal normalization precedes layer combination; attentive statistical pooling concatenates temporal means and standard deviations, and a one-class softmax loss trains higher scores for genuine speech.Source: §2.2 · p. 2
3. EXPERIMENTAL FRAMEWORK
Experiments use challenge-specific training data under closed conditions, with datasets spanning noisy, partial-fake, telephonic, and codec-processed speech. The study also evaluates waveform augmentations and a frozen wav2vec2 training setup.
3. EXPERIMENTAL FRAMEWORK
- Systems were trained using only the corresponding data from each challenge, under closed conditions.The section covers databases, data augmentation, and training setup for evaluation.Source: §3 · p. 2
3.1. ADD 2022 challenge database
- ADD 2022 train and development sets contain about 28K clean Mandarin utterances each; tracks 1 and 2 also provide about 1K adaptation utterances and about 100K unlabeled test utterances.Track 1 covers noisy and music-containing audio; track 2 includes partially fake utterances.Source: §3.1 · p. 3
3.2. ASVspoof 2021 challenge database
- ASVspoof 2021 evaluation uses LA audio transmitted through telephonic systems and DF audio processed with commercial codecs.Because the 2021 challenge had evaluation data only, ASVspoof 2019 LA train and development sets were used; the 2021 sets include about 180K LA and 600K DF utterances.Source: §3.2 · p. 3
3.3. Data augmentation techniques
- Training augmentations apply narrowband or wideband low-pass FIR filters to waveforms, and ADD track 2 additionally creates partially fake audio by overlaying segments onto genuine utterances.For partial-fake augmentation, 20% of genuine train/development utterances are randomly selected each epoch for segment overlay.Source: §3.3 · p. 3
3.4. Training setup
- The wav2vec2 parameters remain frozen during training, while only classifier parameters are updated.Training uses Adam, dropout 0.2, mini-batches of eight utterances, and stops after ten epochs without development-loss improvement.Source: §3.4 · p. 3
4. RESULTS
The section evaluates the proposed systems on ADD 2022 and ASVspoof 2021 using equal error rate (EER), comparing model and augmentation configurations as well as ASVspoof 2021 systems.
4.1. Results on ADD challenge
- ADD 2022 adaptation data sharply reduces EER, while NB FIR filtering cuts EER by about 1% in both tracks and combinations yield the best results.The paper reports that partial-fake augmentation adds further discrimination in track 2.Source: §4.1 · p. 3, Table 2 · p. 4
4.2. Results on ASVspoof 2021
- On ASVspoof 2021, XLS-128 performs best in LA and DF; NB filters help LA, while WB filters favor DF.The filters are described as emulating traditional telephony in LA and general audio codecs in DF.Source: Table 3 · p. 4, §4.2 · p. 4
- Against ASVspoof 2021 systems, the method outperforms others on DF and is competitive on LA despite using a single system.The paper attributes lower DF EER to W2V2 features’ robustness to varied speech content.Source: Table 4 · p. 4, §4.2 · p. 4
- The classifier weights transformer-layer representations differently by scenario; retaining W2V2 as a general feature extractor may save practical computational resources.Figure 2 visualizes the layer weights for the best systems across challenge tracks.Source: Figure 2 · p. 4, §4.2 · p. 4
5. CONCLUSION
The conclusion presents a W2V2-based system that adapts its downstream classifier to each scenario and reports competitive results across ADD 2022 and ASVspoof 2021. It ranked first in ADD track 1 and fourth in track 2, with other self-supervised models and augmentations proposed for future testing.
5. CONCLUSION
- The system uses pre-trained W2V2 contextual representations from different transformer layers in a fine-tuned classifier to detect spoofed speech.The downstream network is adapted to each scenario using data augmentation and adaptation data.Source: §5 · p. 4
- The system ranked first in ADD 2022 track 1 and fourth in track 2, and showed competitive results in ASVspoof 2021 and ADD 2022.Source: §5 · p. 4
- Future work will test other self-supervised models and additional data augmentation techniques.Source: §5 · p. 4