Source-linked AI summary
Did you hear that? Adversarial Examples Against Automatic Speech Recognition
Moustafa Alzantot, Bharathan Balaji, Mani Srivastava
TL;DR
Prior adversarial-example research had largely focused on image and object-recognition systems, leaving successful equivalent attacks against ASR un demonstrated. This paper uses a black-box, gradient-free genetic algorithm to perturb speech, achieving targeted misclassification while largely preserving human recognition. The approach operates under limited model knowledge and uses small least-significant-bit changes to selected audio samples.
Problem
Successful adversarial attacks against automatic speech recognition models had not been demonstrated, despite such attacks succeeding against image recognition and object detection models.
Method
A gradient-free genetic algorithm generates targeted adversarial audio in a black-box setting by evolving candidates with noise added to least-significant bits of selected samples.
Results
87% of targeted attacks succeeded, while human listeners retained the original audio label in 89% of evaluated samples.
Takeaways & Limitations
The demonstration shows that speech-recognition models can be targeted with small background noise without knowing the victim model’s architecture or parameters.
Takeaways & Limitations
The attack was limited to 500 iterations, and stronger attacks may be possible with more noise or iterations; white-box attacks through MFCC preprocessing remain an open direction.
Abstract
from arXiv · showhide
Speech is a common and effective way of communication between humans, and modern consumer devices such as smartphones and home hubs are equipped with deep learning based accurate automatic speech recognition to enable natural interaction between humans and machines. Recently, researchers have demonstrated powerful attacks against machine learning models that can fool them to produceincorrect results. However, nearly all previous research in adversarial attacks has focused on image recognition and object detection models. In this short paper, we present a first of its kind demonstration of adversarial attacks against speech classification model. Our algorithm performs targeted attacks with 87% success by adding small background noise without having to know the underlying model parameter and architecture. Our attack only changes the least significant bits of a subset of audio clip samples, and the noise does not change 89% the human listener's perception of the audio clip as evaluated in our human study.
1 Introduction
The paper addresses the lack of successful adversarial attacks against automatic speech recognition by introducing attacks that alter model predictions while preserving human recognition of the audio. It targets speech models under black-box conditions despite challenges from nondifferentiable spectrogram and MFCC preprocessing.
- Contribution: The attack adds small noise that can change an ASR model’s prediction to an attacker-chosen label while remaining recognizable to human listeners.The paper illustrates a clip recognized by humans as “YES” but classified by the model as “NO.”
- Motivation: Existing adversarial-example methods generally require input gradients, but ASR preprocessing through spectrogram and MFCC layers makes this difficult.The paper contrasts this obstacle with differentiable image-recognition pipelines.
- Contribution: 87% targeted-attack success was achieved while changing only the 8 least-significant bits of a subset of 16-bit audio samples.The evaluation used a speech commands recognition model and dataset.
- Prior Audio Attacks: Earlier audio attacks either rely on specialized high-frequency speakers or produce sounds humans cannot recognize, unlike adversarial examples that remain benign to listeners.The paper positions human-recognizable misclassification as the defining stealth property.
- Threat Model: The attack assumes a black-box threat model in which the attacker can query prediction results but knows neither model architecture nor parameters.The victim model is treated as a function returning prediction probabilities for possible labels.
2 Generating Adversarial Speech Commands
The method generates targeted adversarial audio with a gradient-free genetic algorithm that evolves noisy candidate clips through model queries. It limits perturbations to least-significant bits of selected samples, while detailed subroutines and hyperparameters are omitted from the paper.
- Algorithm: The algorithm takes an original benign audio clip x and a target classification label t, then returns a targeted attack example xadv.The procedure initializes a population of candidate solutions before iterative optimization.
- Evolution: Each generation scores candidates by the target-label prediction, selects parents probabilistically, applies crossover and mutation, and retains the next population.Higher-fitness candidates are more likely to be selected.
- Perturbation: Candidate adversarial examples are generated by adding random noise to a subset of samples, restricted to their least-significant bits to reduce perceptual effects.This design choice is intended to minimize noise heard by humans.
- Reproducibility: Detailed descriptions of some subroutines and hyperparameters are omitted because of space constraints, while an implementation is made available for reproduction.The implementation uses the same hyperparameter values as the reported evaluation.
3 Evaluation
The evaluation tests targeted attacks across source-target label pairs on a keyword-spotting speech model and measures both attack success and human perception. The attack succeeds on 87% of targeted attempts, while 89% of listeners still perceive the source audio.
- The evaluation uses a TensorFlow Speech Commands keyword-spotting model with 90% classification accuracy on 65,000 one-second word clips.
- The targeted experiment generates nine adversarial outputs for each of 500 clips, producing 4,500 source-target attack files.
- Figure 2 reports the percentage of success for every source-target targeted adversarial attack.
- 87% of targeted attacks succeed across source-target pairs within a 500-iteration limit, with a 37-second median generation time.Increasing the noise limit or iteration count could yield more successful attacks.
- The human study labels 1,500 generated clips, with participants unaware of the clips’ source or target labels.The study recruited 23 participants.
- 89% of participants still label the audio with its source label while the machine-learning model labels all clips as the target label.
4 Discussion
The discussion outlines extensions beyond the black-box single-word evaluation, including white-box attacks, larger ASR systems, sentence generation, and over-the-air playback. It also reports 100% success for untargeted attacks while identifying practical evaluation boundaries.
- A white-box attack may be stronger, but MFCC and spectrogram layers make back-propagation difficult; MFCC inversion is proposed for reconstructing adversarial audio.Further experiments are needed to evaluate the reconstructed audio quality.
- The study leaves open whether state-of-the-art larger ASR models are vulnerable and whether adversarial sentences can be generated beyond single-word clips.
- 100% success is reported for untargeted attacks, which are described as weaker but potentially useful for studying robustness against adversarial noise.
- The evaluation feeds audio directly into the classifier rather than testing playback through a speaker and capture by a microphone.The paper identifies over-the-air attacks as harder and leaves them for follow-up work.