Source-linked AI summary
Robust Audio Adversarial Example for a Physical Attack
Hiromu Yakura, Jun Sakuma
TL;DR
The paper addresses the gap between direct-input audio attacks and physical over-the-air attacks, where playback and recording introduce reverberation and noise. It simulates these transformations during adversarial-example generation and reports successful attacks through speakers and radio that listeners generally did not notice.
Problem
Prior methods did not provide audio adversarial examples that could attack recurrent speech-recognition models reliably under over-the-air conditions.
Method
The method incorporates simulated playback transformations, including impulse responses, into the generation process to improve adversarial-example robustness.
Results
The generated examples achieved 100% success through both speakers and radio, while listening experiments found them almost unnoticeable to humans.
Takeaways & Limitations
The results indicate that robust audio adversarial examples can constitute a practical physical-world threat to speech-recognition systems.
Takeaways & Limitations
The evaluation used only three target phrases because each physical-world input-and-phrase combination required more than 18 hours to assess.
Abstract
from arXiv · showhide
We propose a method to generate audio adversarial examples that can attack a state-of-the-art speech recognition model in the physical world. Previous work assumes that generated adversarial examples are directly fed to the recognition model, and is not able to perform such a physical attack because of reverberation and noise from playback environments. In contrast, our method obtains robust adversarial examples by simulating transformations caused by playback or recording in the physical world and incorporating the transformations into the generation process. Evaluation and a listening experiment demonstrated that our adversarial examples are able to attack without being noticed by humans. This result suggests that audio adversarial examples generated by the proposed method may become a real threat.
1 Introduction
Audio adversarial-example research has largely focused on direct model input, leaving physical over-the-air attacks difficult because playback introduces reverberation and noise. The paper proposes simulating these effects to attack recurrent speech-recognition models in the physical world and reports successful, human-unnoticeable attacks.
- Research gap: Audio adversarial examples are underexplored despite widespread deployment of speech-recognition systems in commercial applications and consumer devices.The paper frames this gap as especially relevant for systems such as Alexa, Siri, Google Assistant, and Cortana.
- Research gap: Prior audio attacks often target direct waveform input, which does not transfer reliably to speaker-and-microphone over-the-air conditions.Physical playback and recording introduce reverberation and noise, requiring robustness to unknown environments and equipment.
- Motivation: A single broadcast playback could potentially expose numerous nearby smartphones to an audio adversarial example.The paper contrasts this with image attacks, which generally require presenting the perturbation directly to each target sensor.
- Contribution: The proposed method generates robust audio adversarial examples for recurrent-network speech-recognition models under over-the-air conditions.The method addresses reverberation and physical-world noise by simulating their influence during generation.
- Results: The evaluation reports 100% attack success through both speakers and radio broadcasting, while listening experiments found the target phrase difficult for participants to identify.The contribution statement describes these results as evidence of practical attack feasibility.
2 Background
The background defines adversarial examples as small input modifications that induce a chosen model output and reviews how physical transformations can be incorporated into generation. For audio, prior direct-input attacks against DeepSpeech failed over the air, motivating a transformation-aware approach.
- 2.1 Adversarial Example: An adversarial example modifies an input so a trained model assigns a specified label while the perturbation remains limited.The perturbation bound is intended to keep the modification imperceptible to humans.
- 2.1 Adversarial Example: Optimization methods such as Adam can find adversarial examples when the model is differentiable and gradients with respect to the perturbation are available.The loss function represents the distance between the input data and the desired label under the model.
- 2.2 Physical Attack: Physical attacks require accounting for transformations and noise introduced when adversarial examples pass through sensors or playback equipment.Image research addressed this by simulating transformations such as scaling, rotation, brightness changes, and noise during generation.
- 2.3 Audio Adversarial Example: Carlini and Wagner optimized waveforms through differentiable MFCC extraction and achieved 100% success for direct DeepSpeech input but no success over the air.The direct-input method did not account for reverberation and speaker- or microphone noise.
- 2.3 Audio Adversarial Example: Before this work, no proposed method generated audio adversarial examples for recurrent speech-recognition models that worked under over-the-air conditions.This establishes the specific gap addressed by the paper.
3 Proposed Method
The method incorporates playback and recording transformations into adversarial-example generation through band-pass filtering, impulse responses, and white Gaussian noise. These transformations are intended to improve robustness to frequency-range cuts, reverberation, and environmental or equipment noise.
- The generation process uses a band-pass filter, impulse responses, and white Gaussian noise to model playback and recording transformations.The three techniques target frequency-range limitations, reverberation, and background noise, respectively.
- 3.1 Band-pass Filter: The band-pass filter limits the perturbation to 1,000–4,000 Hz, a range selected empirically for less distortion.This avoids placing perturbation outside the frequencies speakers and microphones reliably reproduce.
- 3.1 Band-pass Filter: The filtered perturbation is intended to remain functional when speakers or microphones cut frequency bands outside the audible range.
- 3.2 Impulse Response: Impulse responses are incorporated through convolution to reproduce reverberation and make adversarial examples robust across playback environments.The approach takes an expectation over impulse responses recorded in diverse environments.
- 3.3 White Gaussian Noise: White Gaussian noise is added after convolution to improve robustness against noise from recording equipment and the environment.Although adding it before convolution would better emulate thermal noise in both devices, post-convolution noise simplifies optimization and was empirically sufficiently robust.
4 Evaluation
The evaluation tested generated adversarial examples across speaker–microphone and FM-radio playback conditions using speech-recognition success, perturbation SNR, and edit distance. The method achieved reliable physical attacks, while results also showed how input content and simulated transformations affected perturbation requirements and robustness.
- Settings: The evaluation used speaker–microphone playback and FM-radio broadcasting to test physical-world attacks.The speaker setup used approximately 0.5 meters of distance; FM transmission used 180.0 MHz and a portable radio.
- Metrics: The study evaluated perturbation SNR, target-phrase success rate, and edit distance between recognition results and the target phrase.Edit distance captures generation progress even when the success rate remains 0%.
- Results: 100% success was obtained for every input-sample and target-phrase combination through both the speaker and the radio.The perturbation magnitude required for 100% success varied with the input sample and target phrase.
- Results: At least 50% success was achieved through both playback conditions with higher maximum SNR and substantially less perturbation than the 100% success examples.The authors characterize this as an attack succeeding once in two attempts when uncertainty is acceptable.
- Results: Bach’s instrumental input produced larger SNR than Owl City’s singing input in all Table 1 and Table 2 cases.The authors relate this difference to singing phonemes helping form target phrases, thereby requiring less perturbation.
- Results: Recognition results changed only slightly between speaker and radio conditions, suggesting robustness to FM transmission.The authors connect this robustness to simulated noise, including white Gaussian noise, during generation.
- Effect of Each Technique: Without all three techniques, only band-pass filtering combined with white Gaussian noise generated an over-the-air attack, and it required much more perturbation.The results suggest white Gaussian noise can support over-the-air operation without impulse responses, while the band-pass filter remains important for physical-device constraints.
5 Listening Experiment
The listening experiment tested whether people noticed robust audio adversarial examples designed to attack speech recognition systems. Although some participants perceived abnormal sound quality, they generally did not hear or identify the target phrases.
- Experiment Design: The experiment assessed human noticeability of adversarial examples through repeated listening and questions about abnormal sounds, words, and candidate phrases.Six examples with 100% attack success were each evaluated by 25 participants, who listened three times.
- Results: No participant heard the target phrases in all cases, despite some reporting that the audio sounded abnormal.The reported abnormalities concerned sound quality rather than intelligible messages or utterances.
- Results: Participants’ comments described compression, fuzziness, poor-call quality, or bird-like sounds rather than recognizable speech.For Table 5(D), 32% reported abnormality, but no message or utterance was identified in any case.
- Results: Most participants could not hear the target phrases, even when explicitly instructed to listen for them and given answer choices.More than half selected “I could not catch anything.”
- Results: Only one or two participants selected the target phrase for five examples, while none selected it for Table 5(F).Seven participants selected incorrect choices for Table 5(F).
- Conclusion: The authors conclude that the examples sounded like mere noise and were almost unnoticeable to humans, supporting their practical attack scenario.They suggest birdsong inputs or telephone playback as possible directions for making examples harder to notice.
6 Conclusion
The paper proposes robust audio adversarial examples that can attack recurrent-network speech recognition models in the physical world. It reports that simulated playback transformations enabled smaller perturbations and that listening experiments found the examples almost unnoticeable, while detailed attack scenarios and defenses remain future work.
- Conclusion: The method targets state-of-the-art recurrent-network speech recognition models with adversarial examples that operate in the physical world.The authors describe this as the first such approach to succeed against recurrent-network speech recognition models over the air.
- Conclusion: Band-pass filtering, impulse responses, and white Gaussian noise simulate over-the-air playback transformations during adversarial-example generation.These transformations are incorporated to improve robustness to physical playback conditions.
- Conclusion: The generated examples achieved smaller perturbations than the conventional method, which cannot deal with recurrent networks.The comparison is reported as part of the evaluation of the proposed method.
- Conclusion: Listening experiments found that the obtained adversarial examples were almost unnoticeable to humans.The conclusion summarizes the reported human-listening results.
- Future Work: Detailed attack scenarios, defense methods, and robust speech recognition through adversarial training are left for future investigation.The authors specifically identify defense development and adversarial training as future directions.