Source-linked AI summary
Targeted Adversarial Examples for Black Box Audio Systems
Rohan Taori, Amog Kamsetty, Brenton Chu, Nikita Vemuri
TL;DR
Existing ASR adversarial work emphasized white-box attacks, motivating a black-box method for targeted audio attacks when model internals are unavailable. The paper combines genetic algorithms with gradient estimation and reports 89.25% targeted attack similarity with 94.6% audio file similarity after 3000 generations.
Problem
Black-box targeted attacks on ASR systems are difficult because practical attackers may access only model outputs rather than architecture and parameters.
Method
The attack combines genetic-algorithm exploration, momentum mutation, and gradient estimation to construct targeted adversarial audio.
Results
89.25% targeted phrase similarity and 94.6% average audio correlation similarity were achieved after 3000 iterations.
Takeaways & Limitations
Targeted adversarial attacks are achievable against black-box ASR models while retaining high similarity to the original audio.
Abstract
from arXiv · showhide
The application of deep recurrent networks to audio transcription has led to impressive gains in automatic speech recognition (ASR) systems. Many have demonstrated that small adversarial perturbations can fool deep neural networks into incorrectly predicting a specified target with high confidence. Current work on fooling ASR systems have focused on white-box attacks, in which the model architecture and parameters are known. In this paper, we adopt a black-box approach to adversarial generation, combining the approaches of both genetic algorithms and gradient estimation to solve the task. We achieve a 89.25% targeted attack similarity after 3000 generations while maintaining 94.6% audio file similarity.
I. INTRODUCTION
Targeted adversarial attacks threaten audio and voice-control systems, while black-box attacks remain difficult because attackers typically access only model outputs. The paper combines genetic algorithms and gradient estimation to generate targeted adversarial audio without model gradients.
- Motivation: Audio systems can be targeted by perturbing waveforms so humans hear the intended phrase while the system produces a predetermined inappropriate interpretation.The paper motivates this risk for voice assistants, cars, and other voice-control technologies.
- Problem: Black-box attacks are harder because practical attackers usually lack network parameters and may access only logits, outputs, or an API transcription.The paper treats Mozilla DeepSpeech as a black box while using its output behavior for attack generation.
- Approach: The proposed attack combines gradient-free genetic search with gradient estimation, including a momentum mutation update to limit excess mutations and noise.Genetic algorithms explore candidate populations, while gradient estimation enables more careful placement of audio perturbations near the target.
- Results: 94.6% audio file similarity was maintained after 3000 generations, alongside a 35% average targeted attack success rate.The reported result quantifies both attack success and preservation of similarity to the original audio.
B. Prior work
Prior work established audio adversarial attacks but largely addressed white-box settings, unintelligible outputs, or simpler classification models. This paper extends black-box targeted attacks to the more complex DeepSpeech decoder.
- White-box attacks: White-box attacks achieved strong results on DeepSpeech, including samples over 99.9% similar and 100% targeted attack accuracy, but required model architecture or parameter knowledge.The cited approach passed gradients through the MFC layer to the raw audio input.
- Black-box audio attacks: A prior black-box audio method produced predetermined ASR targets but remained unintelligible to human listeners, limiting its practical effectiveness.The paper instead targets perturbations of benign audio that preserve human comprehensibility.
- Prior black-box work: Genetic algorithms enabled targeted black-box attacks on Speech Commands, a lightweight convolutional classifier handling up to 50 single-word phrases.That setting differed from the arbitrary-length phrase decoding addressed here.
- Gap addressed: This work combines genetic algorithms and gradient estimation for targeted attacks on the more complex, highly nonlinear DeepSpeech system.The method also introduces momentum mutation for this setting.
C. Background
The attack operates directly on audio inputs to a DeepSpeech ASR pipeline whose outputs are character distributions decoded into arbitrary-length phrases. CTC provides a score despite unknown alignment between audio frames and output characters.
- Dataset: The experiments use the first 100 CommonVoice test-set audio samples and randomly generated two-word target phrases.Each sample is a .wav file processed directly as a NumPy array, bypassing MFC handling in the attack implementation.
- Victim model: DeepSpeech processes audio spectrograms through convolutions, a bidirectional LSTM, a fully connected layer, and a decoder RNN.The paper treats the model as black box and accesses only its output logits.
- CTC: CTC loss enables scoring when the positions of decoded words in the original audio are unknown.This supports arbitrary-length translations without labeled input-output alignment.
- Decoding: The CTC decoder selects the highest-logit character at each frame, removes adjacent duplicates, and removes blank ϵ symbols.For example, aabϵϵb decodes to abb.
- Alignment scoring: Multiple frame-level outputs can decode to the same phrase, so the likelihood of an alignment π under distribution y is used in scoring.This score supports both genetic-population evaluation and gradient estimation.
- Greedy decoding: Greedy decoding evaluates one alignment per timestep, and the genetic algorithm therefore focuses on perturbations to the most likely sequence.This replaces DeepSpeech’s typical beam-search decoding for the attack setting.
II. BLACK BOX ATTACK ALGORITHM
The black-box attack begins with a population-based genetic search and then applies gradient estimation to refine the best candidate. Algorithm 1 iterates until the decoded output reaches the target or the iteration limit is met.
- Algorithm inputs and stopping: Algorithm 1 takes a benign input x and target phrase t, and returns an adversarial audio sample x′.The loop continues while the iteration limit is not reached and Decode(best) does not equal t.
- Phase 1: genetic algorithm: The first phase creates mutated children through crossover of parents selected from the highest-scoring population members.This is the genetic-algorithm exploration stage.
- Phase 1: genetic algorithm: Momentum mutation updates the mutation probability using new and previous scores before the next population is formed.The update is part of the genetic phase’s search control.
- Phase 2: gradient estimation: The second phase copies the top candidate, mutates individual indices, estimates gradients from score changes, and updates the candidate.This provides a more directed refinement after population-based exploration.
A. Genetic algorithm
The genetic algorithm searches for targeted adversarial audio by iteratively adding noise through gradient-free evolutionary optimization. It uses population scoring, selection, crossover, and mutation to improve candidates while preserving similarity to the benign audio.
- A. Genetic algorithm: The attack starts with benign audio and adds noise until the perturbed sample remains similar but decodes as the target.The process is illustrated in the paper’s genetic-algorithm diagram.
- A. Genetic algorithm: The method is independent of model gradients, making genetic search suitable for black-box adversarial audio generation.
- A. Genetic algorithm: Each generation scores population samples and improves them through evolutionary methods, including elite selection, crossover, and mutation.Parents are selected from high-scoring samples, and children combine elements from two parents before possible mutation.
B. Momentum mutation
Momentum mutation controls how frequently the genetic algorithm introduces noise. It uses random, filtered mutations and increases mutation probability when the population stops adapting, helping candidates escape local maxima.
- B. Momentum mutation: Random mutation introduces the algorithm’s noise and can create beneficial traits that later spread through crossover.Without mutation, successive samples may become too similar and remain trapped at a local maximum.
- B. Momentum mutation: Highpass filtering confines mutation noise to frequencies above 7kHz, making it less audible under the 16kHz sampling setup.
- B. Momentum mutation: Mutation probability is analogous to an SGD step size: if it is too low, mutations may not accumulate enough to escape a local maximum.
- B. Momentum mutation: The momentum update raises mutation probability when current and previous population scores are close, indicating that the population has failed to adapt.The moving average also smooths the probability and reduces sensitivity to outliers.
C. Gradient estimation
The attack switches from genetic search to gradient estimation near the target, where only a few perturbations may be needed. The estimate uses finite differences on sampled audio indices, trading query cost for more informed updates.
- C. Gradient estimation: Genetic algorithms search broadly, but gradient estimation becomes more effective near the target when only a few key perturbations remain.The method switches phases when edit distance falls below a threshold.
- C. Gradient estimation: The finite-difference estimate perturbs one audio index at a time and measures the resulting change in CTCLoss.Here, x is the audio-input vector, δi perturbs only index i, and g represents CTCLoss.
- C. Gradient estimation: Sampling every audio index would require 80,000 model queries for one gradient evaluation on a 5-second clip, so the method samples 100 indices per generation.
- C. Gradient estimation: The gradient-estimation phase accepts higher computation in exchange for more informed perturbations when the adversarial example is already near its goal.
A. Metrics
The evaluation measures both targeted decoding similarity and similarity between the original and adversarial audio. These metrics capture attack effectiveness and preservation of the input signal.
- A. Metrics: The experiment uses 100 Common Voice samples, randomly pairing each with a two-word target phrase and running the algorithm for 3000 generations.
- A. Metrics: Targeted attack similarity is computed from the Levenshtein character-edit distance between the desired target phrase and decoded phrase.
- A. Metrics: Audio file similarity is measured using the cross correlation coefficient between the original and adversarial audio samples.
B. Results
The black-box attack achieved high targeted decoding similarity while preserving substantial audio similarity, but exact success remained limited and involved a tradeoff between attack success and audio similarity.
- 89.25% similarity was achieved between final decoded phrases and target phrases, with 94.6% average correlation similarity to original audio.The comparison is based on Levenshtein distance for decoded phrases and correlation similarity for audio samples.
- 35% of adversarial samples achieved exact decoding within 3000 generations, while 22% did so within 1000 generations.
- Most failed attacks were only a few edit distances from the target, according to the Levenshtein-distance histogram.
- Running the algorithm longer could increase success rate, but at the cost of correlation similarity.
- The waveform visualization shows adversarial audio remains largely unchanged, with most modifications appearing as low-volume noise applied uniformly.
IV. CONCLUSION
The paper demonstrates targeted adversarial attacks against black-box ASR by combining genetic algorithms with gradient estimation. Momentum mutation and high-frequency noise improve effectiveness and similarity, supporting the feasibility of the approach while opening directions for future research.
- Combining genetic algorithms and gradient estimation produces black-box adversarial audio with near-perfect target transcriptions while retaining high similarity.The combined approach produces better samples than either algorithm individually.
- Momentum mutation and high-frequency noise improve the effectiveness of the combined attack.Momentum mutation emphasizes early exploration and later annealing, while high-frequency noise reduces interference with human voice.
- Combining all methods achieves the paper’s top results.
- The work introduces black-box attacks on deep, nonlinear ASR systems capable of arbitrary-length translations.The authors present the approach as a feasibility demonstration and identify future research opportunities.