Source-linked AI summary
Fooling End-to-end Speaker Verification by Adversarial Examples
Felix Kreuk, Yossi Adi, Moustapha Cisse, Joseph Keshet
TL;DR
The paper asks whether end-to-end neural speaker-verification systems are vulnerable to adversarial examples. It generates human-indistinguishable perturbations and evaluates white-box and black-box attacks, finding reduced accuracy and sharply increased false-positive rates.
Problem
Speaker verification is deployed for biometric authentication, but the vulnerability of end-to-end neural systems to adversarial examples requires investigation.
Method
The paper generates adversarial examples for an end-to-end neural speaker-verification model by maximizing loss under a constrained perturbation.
Results
The attacks reduce verification accuracy and dramatically increase false-positive rates across white-box and black-box settings.
Takeaways & Limitations
Adversarial examples can make utterances sound like speaker A while enabling the system to claim they were produced by speaker B.
Takeaways & Limitations
Future work remains to evaluate traditional speaker-verification systems and apply adversarial training to neural-based systems.
Abstract
from arXiv · showhide
Automatic speaker verification systems are increasingly used as the primary means to authenticate costumers. Recently, it has been proposed to train speaker verification systems using end-to-end deep neural models. In this paper, we show that such systems are vulnerable to adversarial example attack. Adversarial examples are generated by adding a peculiar noise to original speaker examples, in such a way that they are almost indistinguishable from the original examples by a human listener. Yet, the generated waveforms, which sound as speaker A can be used to fool such a system by claiming as if the waveforms were uttered by speaker B. We present white-box attacks on an end-to-end deep network that was either trained on YOHO or NTIMIT. We also present two black-box attacks: where the adversarial examples were generated with a system that was trained on YOHO, but the attack is on a system that was trained on NTIMIT; and when the adversarial examples were generated with a system that was trained on Mel-spectrum feature set, but the attack is on a system that was trained on MFCC. Results suggest that the accuracy of the attacked system was decreased and the false-positive rate was dramatically increased.
1. INTRODUCTION
Speaker verification authenticates claimed speakers and is deployed in banking and e-commerce, making its vulnerability a practical security threat. This paper investigates adversarial examples against end-to-end neural speaker verification, distinguishing them from spoofing attacks.
- Speaker verification checks whether a spoken utterance was produced by the claimed speaker.
- The technology is used for biometric authentication by banks and e-commerce services online and by phone.
- End-to-end neural models learn directly from utterances, capturing long-range context while reducing system complexity.
- Adversarial examples add small, carefully chosen perturbations that can change a neural network’s prediction while remaining indistinguishable to human listeners.
- The paper generates attacks against an end-to-end neural speaker-verification model using the architecture proposed in.
- Unlike spoofing with a falsifying voice, the approach creates utterances that sound like speaker A but can be accepted as speaker B.
2. NOTATIONS AND DEFINITIONS
The paper formalizes speaker verification as a neural function that compares an unknown utterance with enrollment utterances and estimates whether they belong to the claimed speaker.
- An acoustic speech signal is represented as a variable-length sequence of feature vectors x = (x_1, ..., x_T).
- The verification system receives an unknown utterance and n enrollment utterances associated with speaker k.
- Its output is a probability p ∈ [0, 1] estimating whether the unknown utterance was produced by speaker k.
- The neural function g_θ is trained by minimizing negative log-likelihood over utterance, enrollment-set, and binary-label tuples.
3. END-TO-END DEEP NETWORK MODEL
The end-to-end model embeds the unknown and enrollment utterances with recurrent LSTMs, averages enrollment embeddings, and converts cosine similarity into a verification probability.
- The model architecture was initially proposed in and serves as a baseline for end-to-end speaker-verification work.
- Each unknown or enrollment utterance is processed by a recurrent LSTM into an embedding vector of size D.
- The n enrollment embeddings are averaged into a single speaker representation.
- The model compares the unknown-utterance embedding with the averaged enrollment embedding using cosine similarity.
- A final affine layer maps cosine similarity to a probability estimate and acts as an automatically learned detection threshold.
4. GENERATING ADVERSARIAL EXAMPLES
Adversarial examples are generated by perturbing the test utterance within a norm constraint to increase verification loss while preserving perceptual similarity.
- An adversarial example is a perturbed version of an input that remains human-indistinguishable but causes an incorrect network label.
- The attack optimizes the perturbed utterance to maximize loss under a p-norm distance bound of ϵ from the original.
- Here, ϵ controls adversarial strength and p specifies the norm used to constrain similarity to the original.
- A first-order Taylor approximation yields the fast gradient sign method, which adds a small scaled gradient-sign perturbation to the input.
- To model a realistic attack, noise is added only to the test utterance while the enrollment set remains unchanged.
5. EXPERIMENTS
Experiments evaluate adversarial attacks on speaker-verification models across white-box, cross-dataset, and cross-feature settings, while testing whether the examples remain perceptually indistinguishable. The attacks reduced accuracy and substantially degraded false-positive-rate performance.
- Experimental setup: The experimental pipeline perturbed acoustic-feature vectors, reconstructed adversarial waveforms, and tested them against the original and differently trained models.Models were trained on YOHO or NTIMIT using Mel-spectrum or MFCC features.
- Experimental setup: Adversarial examples generated with fast gradient sign perturbations using ϵ ∈(0.2, 0.3) misclassified examples with high probability while remaining similar to the originals.The attack perturbed test utterance vectors before waveform reconstruction.
- ABX testing: Human listeners correctly classified 54% of original-versus-adversarial waveform pairs in the ABX test, supporting their perceptual similarity.The test used 50 audio pairs and eight listeners per pair.
- White-box attack: White-box attacks were evaluated using system accuracy and false-positive rate, with Table 1 reporting accuracy and Table 2 reporting FPR.The white-box setting assumes access to model internals and gradients.
- Cross-dataset: In the cross-dataset attack, model A’s accuracy fell from 81.55% on the clean test set to 58.93% on the adversarial set, a difference of 22.62%.The FPR increased from 12% to 46%.
- Cross-features: In the cross-feature attack, the MFCC model’s accuracy fell from 81% on the clean test set to 62.25% on the adversarial set, a difference of 18.75%.The FPR increased from 16% to 46%.
6. CONCLUSION
End-to-end neural speaker-verification models require reliability evaluation beyond traditional accuracy metrics. The authors identify robustness evaluation and adversarial training as future directions.
- Improved accuracy over traditional speech-verification components makes broader reliability evaluation critical.
- Future work includes evaluating traditional speaker-verification systems against adversarial examples and applying adversarial training to neural-based systems.