Universal Adversarial Perturbations for Speech Recognition Systems

1. Introduction

• Prior studies established DNN vulnerability to adversarial examples, which can mislead deployed models into incorrect predictions.The paper describes these perturbations as usually imperceptible.Source: §1 · p. 1
• Audio adversarial attacks were less studied than image attacks, limiting understanding of vulnerabilities in commercial DNN-based speech recognition systems.Earlier ASR studies nevertheless demonstrated that audio adversarial examples exist.Source: §1 · p. 1
• Targeted ASR attacks require solving a data-dependent optimization for each sample, making them inapplicable in real time for new inputs.The optimization must be solved again from scratch for every new data point.Source: §1 · p. 1
• The paper proposes a white-box algorithm for universal audio-agnostic perturbations and demonstrates DeepSpeech attacks that transfer to a WaveNet ASR model.The reported WaveNet transfer causes significant transcription error.Source: §1 · p. 1

Adversarial Attacks in the Audio Domain

• Audio ASR attacks primarily targeted transcription into a desired phrase, including approaches against HMM- and GMM-based systems [21].Hidden Voice Commands used inverse feature extraction to produce over-the-air obfuscated audio.Source: §2 · p. 1
• Some earlier attacks were perceptible or not end-to-end, limiting their practicality for studying modern DNN-based ASR systems.Hidden Voice Commands samples sounded like random noise rather than normal speech.Source: §2 · pp. 1–2
• Carlini et al. used end-to-end white-box CTC-loss optimization for target phrases, with quasi-perceptible samples later addressed by psychoacoustic hiding [29].Their perturbation optimization backpropagates through the victim network and MFCC computation.Source: §2 · p. 2

Universal Adversarial Perturbations

• Universal image perturbations reuse one vector across inputs and can transfer across architectures, but image misclassification methods do not directly solve speech mis-transcription.This paper adapts the approach with an alternate optimization problem for ASR.Source: §2 · p. 2

3.1. Threat Model

• Attack success requires transcription changes large enough that the original transcription cannot be deduced; experiments set the CER threshold to 0.5.The universal perturbation is cropped or zero-padded to match each input waveform's length.Source: §3.1 · p. 2

3.2. Distortion Metric

• Relative loudness dB_x(v) measures perturbation loudness against the original waveform; more negative values indicate quieter distortion.The reported distortion is the average relative loudness across the test set.Source: §3.2 · p. 2

3.3. Problem Formulation and Algorithm

• The algorithm iteratively adds minimum perturbations that mis-transcribe current examples, clipping the shared vector to the allowed l∞ bound.It maximizes CTC loss against the original transcription with an l2 regularization penalty, using iterative gradient-sign updates; the objective is constrained by maximum perturbation ϵ, desired success rate δ, and CER threshold t. The per-example optimization is needed because speech has decision boundaries across audio frames, so the image-oriented approach cannot be applied directly.Source: §3.3 · p. 3

4. Experimental Details

• The experiments use pre-trained Mozilla DeepSpeech and Mozilla Common Voice, training on 5,000 randomly selected files from a 582-hour, 400,000-recording English corpus.The 150,000-sample universal perturbation spans about nine seconds at 16 KHz; learning rate α is 5 and regularization c is 0.5.Source: §4 · p. 3
• Evaluation covers the training set and the entire unseen validation set, using mean CER and success rate, with success defined by CER greater than 0.5.Mean relative distortion dB_x(v) quantifies perturbation magnitude over the test set.Source: §4 · p. 3
• The study varies allowed perturbation magnitude and reports performance using transcription error, success rate, and mean relative distortion.The passage identifies these metrics but gives no experimental outcome values.Source: §4 · p. 3

5. Results

• 89.06% validation-set success at mean distortion dB_x(v) ≈ −32 dB demonstrates the attack’s performance on unseen audio.Table 1 reports results across allowed perturbation magnitudes on the training and unseen test sets.Source: §5 · p. 3, Table 1 · p. 4

5.1. Effectiveness of universal perturbations

• At ∥v∥∞ = 100, universal perturbations achieve 65% success, substantially higher than equal-magnitude random noise.The authors suggest universal perturbations exploit geometric correlations in the victim model’s decision boundaries.Source: §5.1 · p. 4, Figure 3 · p. 4

5.2. Cross-model Transferability

• 63.28% success transfers to WaveNet at mean dB_x(v) = −29.82 for the tested setting.The universal perturbation was trained on DeepSpeech; Table 2 reports results for that perturbation on WaveNet and DeepSpeech.Source: §5.2 · p. 4, Table 2 · p. 4

• The proposed end-to-end approach is presented as a way to study DNN-based ASR vulnerabilities and inform more robust neural networks.Source: §6 · p. 4