Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition

Yao Qin; Nicholas Carlini; Ian Goodfellow; Garrison Cottrell; Colin Raffel

Source-linked AI summary

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition

Yao Qin, Nicholas Carlini, Ian Goodfellow, Garrison Cottrell, Colin Raffel

arXiv:1903.10346v2Mar 22, 2019eess.AScs.LGcs.SDstat.ML

TL;DR

Targeted ASR adversarial examples had been audible and ineffective over the air. This paper uses auditory masking and simulated room distortions to address those gaps, achieving imperceptible full-sentence attacks with 100% targeted success while only progressing toward physical over-the-air robustness.

Problem
Targeted ASR adversarial examples had been perceptible to humans and ineffective when played over the air.
Method
The paper uses auditory masking and random room-environment transformations to construct imperceptible and simulated-robust adversarial perturbations.
Results
The attacks retain a 100% targeted success rate on arbitrary full-sentence targets, and generated audio was judged completely identical to clean audio 76% of the time.
Takeaways & Limitations
The work advances practical ASR adversarial examples by combining human-verified imperceptibility with effectiveness under simulated environmental distortions.
Takeaways & Limitations
The robust adversarial examples do not yet work fully over the air, despite working in simulated room environments.

Abstract

from arXiv · show

Adversarial examples are inputs to machine learning models designed by an adversary to cause an incorrect output. So far, adversarial examples have been studied most extensively in the image domain. In this domain, adversarial examples can be constructed by imperceptibly modifying images to cause misclassification, and are practical in the physical world. In contrast, current targeted adversarial examples applied to speech recognition systems have neither of these properties: humans can easily identify the adversarial perturbations, and they are not effective when played over-the-air. This paper makes advances on both of these fronts. First, we develop effectively imperceptible audio adversarial examples (verified through a human study) by leveraging the psychoacoustic principle of auditory masking, while retaining 100% targeted success rate on arbitrary full-sentence targets. Next, we make progress towards physical-world over-the-air audio adversarial examples by constructing perturbations which remain effective even after applying realistic simulated environmental distortions.

1. Introduction

The paper targets two shortcomings of ASR adversarial examples: perceptible perturbations and failure in over-the-air settings. It introduces auditory-masked attacks and simulated-environment robustness while retaining targeted attacks on full sentences.

ASR adversarial examples are often audible and do not yet work when played through a speaker and recorded.
The paper develops adversarial examples that are imperceptible and takes steps toward robustness against physical-world distortions.
Auditory masking places perturbations in audio regions where humans will not hear them, even when their absolute energy is not low.
The robust construction remains adversarial after processing by randomly selected room-environment simulators.
The attack also targets the modern Lingvo automatic speech recognition system.

2. Related Work

Prior ASR attacks achieved targeted, over-the-air, or less perceptible behavior under different conditions, but not all desired properties together. This work combines near-imperceptibility, simulated robustness, full-sentence targets, and Lingvo attacks.

Earlier work produced targeted attacks for arbitrary multiword sentences, but they were neither effective over-the-air nor completely inaudible.
CommanderSong achieved over-the-air attacks at the cost of a significant perturbation to the original audio.
Yakura and Sakuma’s attacks worked over-the-air on short two- or three-word phrases, but not on the originally studied full-sentence phrases.
Schönherr et al. reduced perceptibility using Psychoacoustic Hiding, but targeted a system whose architecture differs from the one studied here.
This work combines near-imperceptibility and effectiveness after simulated distortions while targeting full sentences on the Lingvo network-based ASR system.

3. Background

The paper defines a targeted, imperceptible, and robust ASR attack under a white-box threat model. The adversary optimizes input perturbations using gradients and assumes knowledge of room-distribution statistics rather than exact room configurations.

A targeted attack requires f(x′) = y while the clean input satisfies f(x) ≠ y.
Imperceptibility means humans cannot differentiate the perturbed and original audio when listening.
Robustness is defined as remaining effective after speaker playback and microphone recording, although the paper achieves this only in simulated environments.
The attack targets Lingvo, a sequence-to-sequence ASR model with convolutional, LSTM, and attention components.
The white-box threat model gives the adversary full model and parameter access, including the ability to compute gradients.
For over-the-air attacks, the perturbation is optimized for any room drawn from a known distribution without assuming the exact room configuration.

4. Imperceptible Adversarial Examples

The imperceptibility method replaces ℓp distortion with psychoacoustic masking constraints. It computes frequency-domain representations and uses a two-stage optimization that first fools the ASR system, then makes the perturbation masked.

The method departs from ℓp distortion because small audio perturbations can remain perceptible to humans.
Frequency masking makes nearby-frequency signals inaudible when they fall below a louder masker’s masking threshold.
The audio is transformed with a short-time Fourier transform using 2048-sample windows and a 512-sample hop size.
The masking-threshold procedure identifies spectral maskers using local maxima, quiet-threshold, and 0.5-Bark criteria before applying a two-slope spread function.
A perturbation is inaudible when its normalized PSD remains below the clean audio’s frequency masking threshold.
The loss combines targeted network loss with an imperceptibility term that constrains normalized perturbation PSD below the original audio’s masking threshold.
Optimization uses two stages: first find a relatively small perturbation that fools the network, then enforce imperceptibility under masking constraints.

5. Robust Adversarial Examples

The attack optimizes ASR adversarial examples after simulated room transformations, seeking target transcription across varied environments while constraining perturbation amplitude to preserve imperceptibility.

Acoustic room simulation: The room simulator transforms clean audio x into reverberant speech t(x)=x ∗ r, with room impulse responses sampled from configurations T.It uses the Image Source Method and convolution to model different room environments.
Optimization with reverberations: Robust optimization feeds transformed adversarial audio t(x + δ) to the ASR system and targets transcription y rather than directly attacking clean audio.The expected loss is estimated by independently sampling transformations during gradient descent.
Optimization with reverberations: The second optimization stage requires success across every transformation in a randomly sampled set, producing attacks that tolerate unknown room configurations.The transformation set is resampled at each gradient step, and success requires f(t_i(x + δ)) = y for all sampled transformations.
Optimization with reverberations: Increasing the perturbation amplitude can improve robustness, but the resulting examples become easier for humans to hear.The authors therefore limit the perturbation’s ℓ∞ amplitude to a reasonable range.

6. Imperceptible and Robust Attacks

The paper combines room-transformation robustness with psychoacoustic imperceptibility, optimizing adversarial examples that target ASR transcriptions after simulated reverberation.

Combined attack: The combined attack minimizes both transformed-speech ASR loss and a psychoacoustic imperceptibility loss.The ASR term targets transcription y after random room transformations, while the imperceptibility term follows the masking-based loss.
Imperceptibility mechanism: The imperceptibility loss constrains perturbation δ below the clean audio’s frequency-domain masking threshold.This avoids optimizing a room-dependent masking threshold that varies with reverberation.
Imperceptibility mechanism: The convolution theorem supports optimizing the clean-audio masking loss because reverberant speech is modeled as clean audio convolved with room reverberation.The Fourier transform converts this convolution into pointwise multiplication, making the two imperceptibility objectives have similar effects approximately.
Combined attack: The combined method is described as constructing imperceptible adversarial examples that can attack ASR systems in physical-world conditions.The perturbation is initialized from a robust example and updated using randomly sampled room transformations, with α adaptively balancing robustness and imperceptibility.

7. Evaluation

Evaluation shows that the proposed attacks retain targeted ASR effectiveness while substantially reducing perceptibility, and that robust variants remain effective under simulated over-the-air distortions with a perceptibility trade-off.

Imperceptibility Analysis: 23% of users perceived noise in the proposed examples, statistically indistinguishable from clean audio and far below the 66% baseline rate.Users perceived noise in 19% of clean samples and 66% of Carlini & Wagner examples.
Imperceptibility Analysis: 83% of evaluators selected the proposed imperceptible examples as better audio than Carlini & Wagner examples, a difference not statistically distinguishable from clean audio.When directly compared with clean audio, users preferred clean audio 66% of the time, only 16% above random guessing.
Imperceptibility Analysis: 76% of users judged clean audio and proposed imperceptible examples identical, compared with 24% for clean audio and Carlini & Wagner examples.The proposed examples were judged identical 3× more often than the baseline, but less often than actually identical audio, which scored 85%.
Robustness Analysis: Over-the-air simulation reduced the success rates of both Carlini & Wagner and imperceptible examples to 0%, while robust examples exceeded 60% success.Non-adversarial audio reached 15.42% WER in simulated rooms, whereas robust adversarial examples had lower WER than clean audio.
Robustness Analysis: Human studies found robust-and-imperceptible examples less perceptible than robust examples but more perceptible than clean audio.Evaluators reported less distortion for the imperceptible variants 91% of the time; robust examples remained easier to distinguish from clean audio.

8. Conclusion

The paper constructs imperceptible ASR adversarial examples using auditory masking and reports 100% targeted success on arbitrary full-sentence targets. It also advances robust attacks through simulated room environments, while identifying remaining over-the-air limitations and proposing non-ℓp metrics as a broader research direction.

8. Conclusion: 100% targeted success rate is retained on arbitrary full-sentence targets while constructing imperceptible ASR adversarial examples.The examples are verified through a human study and use the psychoacoustic principle of auditory masking.
8. Conclusion: Simulated room-environment distortions are used to develop robust adversarial examples that remain effective after over-the-air processing.The paper describes this as progress toward practical real-world attacks targeting ASR systems.
8. Conclusion: The robust adversarial examples do not yet play fully over-the-air, despite working in simulated room environments.The authors state that maintaining a high targeted success rate while resolving this difficulty is needed to demonstrate a practical security concern.
8. Conclusion: The paper demonstrates constructing adversarial examples for non-ℓp-based metrics, including metrics based on how humans assess similarity.It presents this as a direction for future research, especially because image-based work has largely focused on ℓp distance.
8. Conclusion: The imperceptible-example construction computes frequency masking thresholds using psychoacoustic principles refined through human studies.The threshold is used to constrain perturbations to regions where they will not be heard by humans.

Step 1: Identifications of Maskers

The masker-identification step estimates which spectral components can contribute to auditory masking. It selects tonal maskers using local-maximum, audibility, neighborhood-dominance, and frequency-range criteria.

Step 1: Identifications of Maskers: The method treats tonal and nontonal maskers as tonal to ensure the computed threshold can mask the noise.Nontonal maskers have stronger masking effects than tonal maskers.
Step 1: Identifications of Maskers: Maskers must be local maxima in the spectrum.This is the first stated criterion for the normalized PSD estimate of a tonal masker.
Step 1: Identifications of Maskers: A masker’s normalized PSD estimate must exceed the threshold in quiet, ATH(k).ATH(k) is approximated by a frequency-dependent function and applies within the human hearing range.
Step 1: Identifications of Maskers: The quiet threshold applies from 20Hz to 20kHz, with STFT frequency related to sampling-point index through the sampling frequency and window size.The passage defines fs as the sampling frequency and N as the window size.
Step 1: Identifications of Maskers: Maskers must have the highest PSD within 0.5 Bark of their frequency, using Bark as a psychoacoustically motivated frequency scale.The main hearing range from 20Hz to 16kHz is divided into 24 non-overlapping critical bands.
Step 1: Identifications of Maskers: The masker PSD estimate is smoothed with neighboring frequencies because masking effects add in the logarithmic domain.

Step 2: Individual masking thresholds

The individual-threshold step models how each selected masker spreads masking across Bark frequencies, then combines these contributions with the quiet threshold. The resulting θx is used as the input-audio frequency masking threshold.

Step 2: Individual masking thresholds: A two-slope spreading function models each masker’s contribution from Bark index b(i) to maskee Bark index b(j).The function T[b(i), b(j)] represents the masking effect contributed by the masker to the maskee.
Step 2: Individual masking thresholds: The global masking threshold combines individual masking thresholds with the threshold in quiet by addition.The global threshold is measured in decibels at frequency index i.
Step 2: Individual masking thresholds: The computed θx is used as the frequency masking threshold for input audio x when constructing imperceptible adversarial examples.Nm denotes the number of selected maskers.
Step 2: Individual masking thresholds: To address back-propagation instability from logarithms, the method removes the 10 log10 term from the PSD estimates and corresponding threshold.The threshold and perturbation PSD expressions are then reformulated.
Step 2: Individual masking thresholds: The proposed algorithms’ notation and definitions are listed in Table 3.

D. Implementation Details

The paper optimizes its generated adversarial examples with the Adam optimizer and reports section-specific hyperparameters.

D. Implementation Details: All adversarial examples are optimized using the Adam optimizer.The hyperparameters used in each section are displayed separately.

D.1. Imperceptible Adversarial Examples

The paper uses staged optimization to reduce perturbation perceptibility while preserving targeted ASR attacks, then extends the procedure toward robustness under simulated room transformations.

Imperceptible adversarial examples: The imperceptibility procedure divides optimization into two stages: first minimizing perturbation norm, then minimizing perceptibility.Stage 1 uses lr1 = 100, T1 = 1000, and a max-norm bound ϵ starting at 2000; Stage 2 uses T2 = 4000.
Imperceptible adversarial examples: Stage 1 clips the perturbation to a decreasing max-norm bound and updates it with the signed loss gradient.The bound is reduced by a factor of 0.8 during optimization, while successful targeted attacks update the bound to the current perturbation norm.
Imperceptible adversarial examples: Stage 2 updates the perturbation using the loss gradient and adapts α according to targeted attack success.α is increased after successful attacks and decreased when the target is not achieved at the specified iteration checks.
Robust adversarial examples: Robust-example optimization first seeks a relatively small perturbation, then improves robustness across random room configurations.The transformation set contains M = 10 transformations, and the second stage fixes ϵ at the first-stage optimized value plus ∆.
Imperceptible and robust adversarial examples: Jointly imperceptible and robust examples alternate perceptibility reduction with robustness improvement using room-based success thresholds.The first stage runs for 4000 iterations, while the second runs for 6000 iterations and increases α when attacks succeed in 8 out of 10 rooms.