PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding

Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, Dorothea Kolossa

arXiv:1808.05665v2cs.CRcs.SDeess.AS

TL;DR

The paper addresses the vulnerability of DNN-based ASR systems to targeted adversarial perturbations that can force malicious transcriptions. It introduces psychoacoustic hiding with raw-audio backpropagation and forced alignment, and reports highly reliable attacks whose targets were inaudible to listeners while original speech remained understandable.

  • Problem

    DNN-based ASR systems are vulnerable to adversarial perturbations that can force malicious transcriptions, creating risks for voice interfaces.

  • Method

    The attack combines psychoacoustic modeling, preprocessing-aware backpropagation, and forced alignment to embed target transcriptions beneath human hearing thresholds.

  • Results

    The method successfully hid arbitrary target transcriptions in up to 98% of cases, while listeners could not comprehend the targets and recognized the original speech.

  • Takeaways & Limitations

    Targeted adversarial perturbations can be made barely distinguishable from original audio in state-of-the-art DNN-HMM ASR systems.

Abstract

from arXiv · show

Voice interfaces are becoming accepted widely as input methods for a diverse set of devices. This development is driven by rapid improvements in automatic speech recognition (ASR), which now performs on par with human listening in many tasks. These improvements base on an ongoing evolution of DNNs as the computational core of ASR. However, recent research results show that DNNs are vulnerable to adversarial perturbations, which allow attackers to force the transcription into a malicious output. In this paper, we introduce a new type of adversarial examples based on psychoacoustic hiding. Our attack exploits the characteristics of DNN-based ASR systems, where we extend the original analysis procedure by an additional backpropagation step. We use this backpropagation to learn the degrees of freedom for the adversarial perturbation of the input signal, i.e., we apply a psychoacoustic model and manipulate the acoustic signal below the thresholds of human perception. To further minimize the perceptibility of the perturbations, we use forced alignment to find the best fitting temporal alignment between the original audio sample and the malicious target transcription. These extensions allow us to embed an arbitrary audio input with a malicious voice command that is then transcribed by the ASR system, with the audio signal remaining barely distinguishable from the original signal. In an experimental evaluation, we attack the state-of-the-art speech recognition system Kaldi and determine the best performing parameter and analysis setup for different types of input. Our results show that we are successful in up to 98% of cases with a computational effort of fewer than two minutes for a ten-second audio file. Based on user studies, we found that none of our target transcriptions were audible to human listeners, who still understand the original speech content with unchanged accuracy.

I. INTRODUCTION

The paper introduces psychoacoustic hiding for targeted ASR attacks, addressing the perceptibility of earlier audio adversarial examples. It combines psychoacoustic modeling, raw-audio backpropagation, and forced alignment to hide malicious transcriptions while preserving the apparent original audio.

  • Motivation: DNN-based ASR systems expand attack surfaces because their many parameters may contain exploitable blind spots.Potential scenarios include malicious voice-assistant requests and attacks delivered over radio or television.
  • Related work: Earlier ASR attacks produced highly distorted or suspicious audio, motivating methods that reduce human perceptibility.Feature-space attacks caused easily perceived distortion, while other approaches produced unintelligible noise or required costly ultrasound hardware.
  • Contributions: Psychoacoustic hiding constrains perturbations to regions below or near human hearing thresholds, using a model inspired by MP3 encoding.The attack feeds the audio signal directly into the recognizer and uses backpropagation to find minimally perturbing adversarial examples.
  • Method: Integrating differentiable preprocessing into backpropagation lets the method modify raw audio directly without inverting feature extraction.Forced alignment finds promising temporal starting points for placing the target transcription while minimizing perceptible noise.
  • Experimental evaluation: Up to 98% of arbitrary target transcriptions could be hidden in audio files with minimal perceptible noise in a white-box setting.The evaluation targeted Kaldi and examined parameter settings including phone rates and deviations from hearing thresholds.
  • User study: User studies found listeners could not comprehend target transcriptions and recognized only the original content, while psychoacoustic modeling improved perceived quality.The contribution includes experimental optimization of recognition success and human-perceived noise.

II. TECHNICAL BACKGROUND

DNN-based ASR systems map audio-derived features to recognition outputs but can contain exploitable blind spots. Adversarial examples update only the input, using gradients and perceptual constraints to produce targeted changes.

  • Kaldi is used for experiments because it is an accessible, open-source toolkit with state-of-the-art ASR algorithms.
  • DNNs learn complex input/output mappings, but their many parameters can create blind spots that are not obvious to humans.
  • Adversarial examples modify an input x to produce a different output while keeping the perturbation close to the original signal.
  • Backpropagation creates adversarial examples by keeping network parameters fixed and iteratively updating the input vector.
  • The attack measures output loss against a target, computes the input gradient, and updates the input using a learning rate.

D. Psychoacoustic Modeling

The paper adapts psychoacoustic masking from MP3 compression to constrain adversarial perturbations. Hearing thresholds specify which time-frequency energy changes can remain below human perception.

  • Psychoacoustic hearing thresholds model how dependencies between frequencies create masking effects in human perception.
  • MP3 compression uses empirical thresholds to remove inaudible signal components while producing a smaller, lossy representation.
  • The attack applies the psychoacoustic model as a rule set for adding noise in frequency regions where masking permits manipulation.
  • Audio is divided into 1024-sample buffers, transformed into 32 frequency bands, and segmented into MPEG ISO scale-factor bands.
  • Hearing-threshold matrices bound the energy added to each time-frequency window, with the largest bin normalized to 95 dB.
  • Thresholds are applied during adversarial-example creation alongside temporal alignment to limit audible noise.

A. Adversary Model

The attack assumes a white-box adversary and combines forced alignment, iterative backpropagation, and hearing-threshold constraints. Preprocessing is integrated into the DNN so raw audio can be optimized directly.

  • The adversary knows the ASR mechanism and generates malicious audio before the attack in a white-box setting.
  • Forced alignment uses the original audio and target transcription to find target pseudo-posteriors before backpropagation.
  • Backpropagation iteratively manipulates the input until convergence or a fixed iteration limit so the ASR system produces the desired output.
  • Hearing thresholds are applied during backpropagation to constrain changes that humans can perceive.
  • When forced alignment cannot fit an audio file to the target, the method divides the audio equally among states.
  • Integrating preprocessing with the DNN preserves ASR accuracy while enabling direct backpropagation to the raw audio.

2) Discrete Fourier Transform:

The preprocessing pipeline transforms framed audio into frequency-domain magnitude features for ASR. Its differentiable DFT and magnitude operations support backpropagation to the raw signal.

  • Each audio frame is transformed into the frequency domain with a discrete Fourier transform, whose fixed coefficients provide the backpropagation derivatives.
  • The DFT output is complex-valued, but the ASR feature uses spectrum magnitude because phase is not relevant for speech recognition.
  • Magnitude derivatives are handled by considering the real and imaginary parts separately during backpropagation.
  • In the integrated ASR architecture, preprocessing is represented by separate layers, with the magnitude layer halving the data size.

4) Logarithm:

The attack measures spectral differences relative to psychoacoustic hearing thresholds and uses these thresholds to constrain gradient-based signal modifications. A tunable λ permits small deviations from the thresholds while normalized scaling factors regulate backpropagation.

  • 4) Logarithm:: The logarithm of squared spectral magnitude, χ = log(|X(t, k)|2), provides the feature representation used in speech recognition.Its derivative with respect to squared magnitude is 1 / |X(t, k)|2.
  • F. Hearing Thresholds: The attack limits spectral differences between S and M to human hearing thresholds across all times t and frequencies k.Thresholds H are calculated from the original audio signal and define acceptable manipulations in each time-frequency bin.
  • F. Hearing Thresholds: The distortion measure D compares the original spectrum S with the modified spectrum M in decibels using the maximum original power spectrum as each utterance’s reference.The resulting matrix Φ records the difference in dB relative to the calculated hearing thresholds.
  • F. Hearing Thresholds: A variable λ is added to allow the optimized result to differ slightly from the tight hearing thresholds.Negative Φ* values indicate threshold crossings and are set to zero before Φ* is normalized into scaling factors ˆΦ.
  • F. Hearing Thresholds: The gradient is scaled by both normalized threshold-derived factors ˆΦ and ˆH during each backpropagation iteration.ˆH depends only on the hearing thresholds, while ˆΦ reflects the current optimization state.

IV. EXPERIMENTS AND RESULTS

The experiments evaluate psychoacoustic adversarial attacks against Kaldi using speech and music inputs, while measuring target-transcription accuracy and perceptible noise. The study compares algorithm variants with and without hearing thresholds and forced alignment.

  • IV. EXPERIMENTS AND RESULTS: The evaluation targets Kaldi’s state-of-the-art ASR system and compares proposed algorithm improvements and parameter settings.Experiments use speech and music audio, with or without hearing thresholds and forced alignment.
  • IV. EXPERIMENTS AND RESULTS: The Kaldi setup uses the default Wall Street Journal training recipe, adapting only preprocessing for integration into the DNN.The WSJ data contains more than 80 hours of mostly clean, read-sentence speech from a large-vocabulary, phone-based ASR task.
  • B. Metrics: Target-transcription recognition accuracy is measured with word error rate (WER), computed from deleted, inserted, and substituted words relative to the target sentence.Speech inputs may also retain or add words from the original text, producing WER values above 100%.
  • B. Metrics: Perceptible noise is measured with φ, the normalized average of threshold-exceeding values of Φ rather than signal-to-noise ratio.Only Φ(t, k) values above zero contribute because they represent changes exceeding hearing thresholds.
  • IV. EXPERIMENTS AND RESULTS: The baseline omits hearing thresholds and forced alignment; later variants add thresholds and then forced alignment to trade noise reduction against attack success.The full version with forced alignment is reported to produce clearly improved WER.
  • IV. EXPERIMENTS AND RESULTS: The main evaluation uses 70 utterances from 10 different speakers drawn from a WSJ test set.

1) Backpropagation:

Backpropagation reliably creates targeted adversarial audio, while hearing thresholds and forced alignment reduce perceptibility and improve the accuracy–noise trade-off. Results vary with λ, iteration count, and input type.

  • 1) Backpropagation:: 1.43% WER was achieved without hearing thresholds or forced alignment, but the average perceptibility was 11.62 dB and clearly audible.The result demonstrates that backpropagation into the time domain can produce valid targeted adversarial audio reliably.
  • 2) Hearing Thresholds:: 64.29% WER and 7.04 dB perceptibility were obtained after adding hearing thresholds, preserving valid adversarial examples while reducing perceptible noise.
  • 3) Forced Alignment:: With λ = 20 and forced alignment, the attack achieved 36.43% WER and φ = 5.49 dB.Forced alignment improved WER and φ in all relevant cases except λ = 0, where WER was very high across scenarios.
  • 1) Backpropagation:: Higher iteration counts and higher λ generally lower WER, although music experiments include exceptions for some settings.Threshold-applied experiments remain feasible even when minimizing human perceptibility.
  • 2) Hearing Thresholds:: Perceptibility φ decreases with λ, and music consistently has lower φ than speech, indicating that adversarial examples are easier to conceal in music.The threshold-free condition has substantially higher φ than the other evaluated cases.

2) Phone Rate Evaluation:

The evaluation identifies practical settings for hiding target transcriptions and examines convergence, runtime, and noise relative to CommanderSong. Phone rate affects WER, while successful examples can be generated within minutes.

  • 2) Phone Rate Evaluation:: WER increases clearly with phone rate, with a minimum at 4 phones per second that does not change significantly at lower rates.Because longer audio increases computation time, 4 phones per second is chosen as a reasonable setting.
  • 3) Number of Required Repetitions:: Success rate is evaluated against iteration count separately for speech and music across λ = 0, λ = 20, and λ = 40.Each attempt is checked every 100 iterations until success or 5000 iterations, with success requiring 0% WER.
  • 3) Number of Required Repetitions:: A ten-second audio file requires fewer than two minutes to generate adversarial perturbations with 500 backpropagation steps on a 6-core processor.The algorithm is easy to parallelize.
  • 3) Number of Required Repetitions:: The method’s SNR exceeds CommanderSong’s in all reported settings, including cases without hearing thresholds.SNR indicates added noise but does not directly measure perceptible noise, so it may diverge from φ.
  • 3) Number of Required Repetitions:: The human-perception evaluation combines a transcription test with a MUSHRA test of perceived audio quality across parameter setups.The MUSHRA evaluation includes speech, music, and recorded twittering-bird samples.

A. Transcription Test

The transcription test evaluated whether adversarial audio preserves the original speech for human listeners while concealing the embedded target transcription. Human ratings and word-error results indicated unchanged intelligibility of the original utterances, while the hidden text remained unintelligible.

  • Transcription Test: Human listeners could not comprehend the hidden target transcription and recognized only the original speech content.The study asked listeners to transcribe original and adversarial samples and compare their understood text with the hidden target.
  • Transcription Test: 12.59% and 12.61% average WER were measured for original and adversarial utterances, respectively.A two-sided t-test at a 1% significance level found no difference between the distributions.
  • MUSHRA Test: The MUSHRA evaluation compared original signals with adversarial examples generated using λ = 0, λ = 20, λ = 40, and no hearing thresholds.Nine audio samples covered speech, music, and recorded twittering birds; all adversarial examples successfully hid their target text within 500 iterations.
  • Transcription Test: WER far above 100% showed that the hidden text was not intelligible to listeners.The few correct words overlapped with the original text and were frequent short words such as “is,” “in,” and “the.”
  • MUSHRA Test: Adversarial examples using hearing thresholds received significantly higher ratings than the no-threshold anchor at a 1% significance level.The result indicates a perceptible quality difference between attacks with and without psychoacoustic hearing thresholds.
  • MUSHRA Test: One twittering-birds test rated hearing-threshold adversarial examples above 80 MUSHRA-points, similarly to the reference.The original signal was normally rated higher because listeners directly compared it with the adversarial versions.
  • MUSHRA Test: No specific λ value was clearly preferred, although λ = 0 was slightly higher-rated than λ = 20 and λ = 40.Only the distributions for λ = 0 and λ = 40 differed significantly; higher λ values required fewer iterations, with each iteration potentially adding noise.

VI. RELATED WORK

Prior ASR attacks either produced perceptible or suspicious audio, relied on specialized hardware or microphones, or did not account for human perception. This work combines targeted DNN attacks with psychoacoustic hiding and forced alignment to reduce perceptibility.

  • Research gap: The paper reports that previous ASR attacks did not use psychoacoustics to hide a target transcription in another audio signal.Prior work focused on hiding or obfuscating transcriptions, and most attacks were not DNN-based.
  • Existing ASR attacks: Earlier ASR attacks often created highly distorted audio, unintelligible noise, or suspicious signals for human listeners.Vaidya et al. modified features but produced easily perceptible distortions, while Carlini et al. often produced noise-like samples.
  • Existing ASR attacks: Ultrasound-based attacks hide commands above 20 kHz but require costly microphone-specific information and may vary across recording devices.These approaches depend on nonlinear microphone behavior and attack parameters tailored to particular hardware.
  • Existing ASR attacks: Carlini and Wagner used CTC-loss with gradient descent for targeted ASR attacks, but reported higher perceptible noise and lower effectiveness than this approach.Their loss function is designed for time-sequence recognition.
  • Existing ASR attacks: CommanderSong was evaluated against Kaldi but did not consider human perception and was demonstrated only for music, with limited over-the-air robustness.Its attack parameters depend especially on speakers and recording devices.
  • This work: The proposed approach combines targeted perturbations with psychoacoustic constraints, modified backpropagation, and forced-alignment initialization.The design aims to make added noise barely perceptible while embedding a target transcription.
  • Related concealment methods: Watermarking and steganography hide information in carrier signals for copyright protection or protected transmission, unlike this method’s ASR-targeting objective.Steganographic approaches include least-significant-bit and modulation-based schemes, with different assumptions about acoustic characteristics.

VII. DISCUSSION

The attack reliably creates targeted adversarial perturbations that are barely distinguishable from original audio, while parameter choices and audio content strongly affect noise, success, and computation. The discussion identifies countermeasure and deployment questions, especially for real-world and commercial systems.

  • Targeted perturbations successfully attack state-of-the-art DNN-HMM ASR systems while remaining barely or even impossible to distinguish from original audio samples.
  • 500 iterations should not be exceeded because success is already very high below that threshold, while more iterations increase perceptible noise and calculation time.
  • Music or other nonspeech audio is recommended as the original sample because speech must be obfuscated and typically requires larger perturbations.
  • The attack is relatively fast and parallelizable because preprocessing is integrated into backpropagation, enabling comparison of setups for inconspicuousness and required success rate.
  • Countermeasures: Distillation may be inappropriate for ASR because transcription depends on temporal alignment as well as classification outputs.
  • Countermeasures: MP3 encoding distorted both the original and target transcriptions, while training on MP3 audio is assumed to move vulnerability into perceptible regions without eliminating DNN blind spots.
  • Future work: Real-world attacks must account for acoustic transfer functions and environmental noise, although controlled settings such as elevators may permit modeling these factors.
  • Forced alignment, backpropagation, the original audio choice, and an optimal phone rate help create reliable, psychophysically almost imperceptible perturbations.

APPENDIX

The appendix records original and modified transcription outputs, including corrected wording, altered names, normalized numbers, and malicious target commands. The listening-test transcriptions include both benign phrases and embedded command-like targets.

  • The appendix compares original and modified outputs for short phrases, reporting corrections such as “COLLAGE” to “COLLEGE” and “SHEDULED” to “SCHEDULED.”
  • Named entities and personal names are altered in examples such as “KAISERTEK” to “KAISERTECH” and “MAYER” to “MAIER.”
  • Several modified outputs normalize spoken forms, including “TWENTY TWO” to “TWENTY TWO,” dates in words, and monetary amounts in words.
  • The listening-test transcription list includes benign phrases such as “I BELIEVE MOST PEOPLE ARE GOOD” and “THE SOUND OF SILENCE.”
  • The list also includes command-like targets such as “DEACTIVATE SECURITY CAMERA AND UNLOCK FRONT DOOR” and “THE COMMAND IS PLANTED.”
Loading 1808.05665v2…