Source-linked AI summary
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
Nicholas Carlini, David Wagner
TL;DR
Speech-to-text systems have been comparatively underexplored for targeted audio adversarial examples. The paper applies an end-to-end white-box iterative optimization attack to DeepSpeech, achieving arbitrary targeted transcriptions with 100% success and enabling additional audio manipulations.
Problem
Targeted adversarial examples have been studied comparatively little in audio speech recognition, where existing attacks had limited ability to modify arbitrary audio or target transcriptions.
Method
The paper applies an end-to-end white-box iterative optimization attack directly to raw waveforms while differentiating through DeepSpeech's preprocessing and neural network.
Results
100% success was achieved in turning source audio into desired target transcriptions with slight distortion, including outputs at 50 characters per second.
Takeaways & Limitations
Audio adversarial examples provide a new domain for studying neural-network properties and can embed speech in music or hide speech from transcription.
Abstract
from arXiv · showhide
We construct targeted audio adversarial examples on automatic speech recognition. Given any audio waveform, we can produce another that is over 99.9% similar, but transcribes as any phrase we choose (recognizing up to 50 characters per second of audio). We apply our white-box iterative optimization-based attack to Mozilla's implementation DeepSpeech end-to-end, and show it has a 100% success rate. The feasibility of this attack introduce a new domain to study adversarial examples.
I. INTRODUCTION
Prior adversarial-example research focused mainly on images and had comparatively limited coverage of audio speech recognition. This paper introduces an end-to-end targeted attack on DeepSpeech that uses small perturbations to force arbitrary transcriptions.
- Motivation: Audio speech recognition has received comparatively little adversarial-example research despite its widespread use in systems such as Siri, Google Now, and Amazon Echo.Speech recognition maps an audio waveform to a transcription, creating a distinct audio setting for adversarial-example study.
- Prior work: Existing targeted audio attacks either synthesize new inaudible commands or do not modify existing audio, while other work is untargeted.The prior targeted state of the art, Houdini, could construct only limited audio adversarial examples.
- Approach: The attack constructs a nearly inaudible perturbation δ such that any natural waveform x+δ is recognized by DeepSpeech as any desired phrase.It uses strong, iterative, optimization-based attacks and operates end-to-end through the raw audio input.
- Results: 100% success was achieved regardless of the desired transcription or initial source audio sample.The white-box attack optimizes directly on raw samples while differentiating through the MFC preprocessing transformation.
- Implications: The attack can embed speech in music, make arbitrary audio transcribe as speech, or hide speech by targeting silence.These capabilities extend targeted manipulation beyond ordinary spoken source audio.
- Implications: The paper presents audio adversarial examples as a new domain for studying neural-network behavior and releases code and a dataset for further work.The authors also encourage listening to the generated examples.
A. Threat Model & Evaluation Benchmark
The threat model requires exact targeted transcription under white-box, direct-classification conditions, while distortion is measured relative to the original waveform in decibels. The benchmark evaluates random incorrect targets across 100 Common Voice test instances.
- Threat Model: The task is to construct x′=x+δ that sounds similar to x while satisfying exact transcription C(x′)=y.Success excludes misspellings and extra characters.
- Threat Model: The adversary has complete knowledge of the model and parameters, and examples are evaluated without over-the-air noise or microphone recording.The paper expects later extensions to black-box and noisy settings but does not assume them here.
- Distortion Metric: Distortion is measured in decibels as a logarithmic relative-loudness scale, comparing perturbation δ with the original waveform x.Because the perturbation is quieter than the signal, distortion is negative; smaller values indicate quieter distortions.
- Evaluation Benchmark: The benchmark uses the first 100 Mozilla Common Voice test instances and targets 10 randomly selected incorrect, theoretically reachable transcriptions per sample.The targets must differ from the original transcription.
B. An Initial Formulation
The attack formulates targeted transcription as an optimization problem and differentiates through the complete audio-to-transcription pipeline. An initial CTC-loss formulation achieves perfect targeting with moderate distortion, while phrase length affects difficulty.
- Formulation: The optimization seeks a perturbation that makes the classifier output target phrase t while keeping the modified waveform close to x.A parameter c balances adversarial success against similarity to the original example.
- Formulation: The audio constraint clips perturbations to the maximum representable waveform value, while the nonlinear transcription constraint makes standard gradient descent ineffective.The loss is reformulated so nonpositive loss implies the target transcription.
- Loss Function: Audio loss construction is difficult because DeepSpeech requires decoding character probabilities into a phrase before evaluating the transcription.This differs from image classification, where the output probability for a label directly corresponds to the classification target.
- Optimization: The optimization differentiates through the audio sample, MFC transformation, neural network, and final loss, solving for the complete waveform simultaneously with Adam.A single example takes approximately one hour on one NVIDIA 1080Ti, or a few minutes each when 10 are constructed in parallel.
- Evaluation: The initial CTC-loss attack achieved 100% success for every source-target pair with mean perturbation −31dB.The reported 95% distortion interval ranged from −15dB to −45dB.
- Evaluation: Each additional target character requires approximately a 0.1dB increase in distortion, while longer source phrases make targeting easier; these effects roughly counteract.The authors did not measure the opposing effects with statistically significant certainty.
C. Improved Loss Function
The attack improves targeted audio adversarial examples by replacing a less effective loss with a carefully designed, alignment-based objective. This reduces distortion, while the reported guarantee applies only to greedy decoding.
- C. Improved Loss Function: A carefully designed loss function produces lower-distortion audio adversarial examples than CTC loss alone.The authors state that loss-function choice matters in audio, though less than the factor-of-three effect reported for images.
- C. Improved Loss Function: The improved objective addresses uneven character difficulty by weighting each aligned character separately rather than using one phrase-wide constant.A single constant must accommodate the hardest character, unnecessarily increasing pressure on easier segments.
- C. Improved Loss Function: The two-step attack first obtains an alignment with CTC loss, then fixes that alignment while optimizing a less-distorted adversarial example.The second stage starts from the CTC-derived adversarial example.
- C. Improved Loss Function: −31dB to −38dB mean distortion was achieved on 100 Common Voice test instances, but the examples were guaranteed only against greedy decoding.With beam search, outputs were often closer to the target without matching it perfectly.
- C. Improved Loss Function: −30dB distortion produced overlaid original and adversarial waveforms that were nearly indistinguishable visually.The displayed sample was randomly selected and contained 500 samples.
D. Audio Information Density
The attack reaches the theoretical audio information-density limit of 50 characters per second. In the fixed-alignment setting, the loss is simpler, but the reported attack requires −18dB mean distortion.
- D. Audio Information Density: 50 characters per second is the theoretical maximum density, and the attack generates examples at this maximum rate.DeepSpeech processes 50 audio frames per second, allowing short clips to transcribe into long phrases.
- D. Audio Information Density: The logit-based loss applies directly when the target phrase is the only possible alignment to the output sequence.Any other alignment would require omitting a character.
- D. Audio Information Density: −18dB mean distortion was required for this attack.
E. Starting from Non-Speech
The attack can begin with arbitrary non-speech audio, not only speech waveforms, and make it transcribe as a chosen phrase. Classical-music clips required more computation and −20dB mean distortion.
- E. Starting from Non-Speech: Arbitrary non-speech audio can be transformed into audio recognized as any target phrase without additional technical novelty.The attack simply uses a non-speech waveform as its initial input.
F. Targeting Silence
The attack can target silence, causing DeepSpeech to transcribe audio as nothing. Silence is easier to target than a specific phrase, especially for longer source audio that can suppress unnecessary sounds.
- F. Targeting Silence: Silence can be represented as an arbitrary-length sequence containing only space characters, improving the empty-phrase attack.
- F. Targeting Silence: The CTC-loss trajectory shows that iterative optimization reaches adversarial examples where a fast-gradient-sign direction plateaus.The figure states that targeted examples can lie far enough from the source that local linearity alone is insufficient.
- F. Targeting Silence: Less than −45dB distortion below the original signal can turn any phrase into silence.The authors find silence easier to target than a specific phrase.
- F. Targeting Silence: Longer source phrases are easier to silence because the attack can suppress unnecessary sounds and retain a subsequence matching the target.Shorter phrases require synthesizing characters that were not previously present.
A. Evaluating Single-Step Methods
Single-step gradient attacks are ineffective for targeted audio adversarial examples, whereas iterative optimization can find successful directions despite audio-model nonlinearity.
- Audio attacks inherit nonlinearity from MFCC computation and deep LSTM stacks, making single-step methods ineffective for targeted examples.
- Iterative optimization reduces CTC loss toward an adversarial example after the fast-gradient direction plateaus.
- FGSM can produce untargeted audio adversarial examples, but optimization methods achieve them with less distortion.
- Pointwise random noise can remove an adversarial label, even when the noise is small enough for normal examples to retain classification.
- The straight-through estimator enables examples that remain targeted after MP3 compression, with approximately 15dB larger distortion.
V. OPEN QUESTIONS
The paper identifies physical robustness, universality, transferability, and cross-domain defenses as open questions for audio adversarial examples. It also reports that the demonstrated attacks do not survive over-the-air playback.
- The constructed audio adversarial examples do not remain adversarial after over-the-air playback, limiting their real-world threat.
- Future work may seek audio adversarial examples effective over-the-air, following later physical-world work on image attacks.
- Universal perturbations remain an open question: one perturbation might cause arbitrary waveforms to transcribe as a target phrase.
- Transferability remains an important direction because one perturbation could target multiple audio classifiers and enable black-box attacks.
- Existing adversarial-example defenses have been evaluated only on images, leaving their applicability to audio unresolved.
- The paper achieves targeted transcription of any waveform with 100% success and slight distortion, up to 50 characters per second.
- The authors present preliminary evidence that linearity does not hold in audio, motivating separation of audio-specific and fundamental adversarial-example properties.