Source-linked AI summary
Natural Backdoor Attacks on Speech Recognition Models
Jinwen Xin, Xixiang Lyu, Jing Ma
TL;DR
Speech recognition backdoors are understudied despite SR models’ vulnerability and security relevance. The paper uses ordinary natural or daily-life sounds as poisoning triggers and evaluates them across datasets, models, and attack factors. Natural triggers achieve high attack success without compromising benign performance, with 5% poisoned samples producing a near-100% attack success rate.
Problem
Speech recognition backdoors have received relatively little attention despite SR models’ vulnerability to noise and their use in security-sensitive applications.
Method
The paper poisons SR training data with ordinary sounds from nature or daily life, such as rain, whistles, and bird calls, then evaluates the resulting backdoors across datasets, models, and attack conditions.
Results
Natural backdoor attacks achieve high attack success without compromising benign performance, even with short or low-amplitude triggers; 5% poisoned samples yield a near 100% attack success rate.
Takeaways & Limitations
Natural sounds can automatically activate covert SR backdoors and remain effective in real physical scenarios and Clean-label attacks.
Takeaways & Limitations
The threat model assumes an attacker can control a small number of training samples, as in crowdsourced data collection.
Abstract
from arXiv · showhide
With the rapid development of deep learning, its vulnerability has gradually emerged in recent years. This work focuses on backdoor attacks on speech recognition systems. We adopt sounds that are ordinary in nature or in our daily life as triggers for natural backdoor attacks. We conduct experiments on two datasets and three models to validate the performance of natural backdoor attacks and explore the effects of poisoning rate, trigger duration and blend ratio on the performance of natural backdoor attacks. Our results show that natural backdoor attacks have a high attack success rate without compromising model performance on benign samples, even with short or low-amplitude triggers. It requires only 5% of poisoned samples to achieve a near 100% attack success rate. In addition, the backdoor will be automatically activated by the corresponding sound in nature, which is not easy to be detected and will bring severer harm.
1 Introduction
The paper addresses vulnerabilities in speech recognition backdoors by replacing conspicuous synthetic triggers with ordinary natural or daily-life sounds. It proposes natural triggers as covert, easily activated backdoors and evaluates their effectiveness and influencing factors.
- Motivation: Speech recognition backdoors deserve attention because SR models are vulnerable to noise, while security-sensitive voice commands can create serious risks.Prior backdoor research has focused mainly on computer vision, despite SR’s role in autonomous driving and human-computer interaction.
- Motivation: Existing SR backdoors mainly use random noise, frequency-specific waves, or ultrasonic pulses as triggers.These triggers generally require the adversary to play a corresponding sound during inference.
- Approach: Natural backdoor attacks use ordinary sounds from nature or daily life, including rain, whistles, and bird calls, as triggers for SR models.The paper presents these sounds as natural triggers in Fig. 1.
- Advantages: Natural triggers are covert because ordinary sounds attract little human attention, while few poisoned samples can still achieve high attack success rates.The same sounds can automatically activate the backdoor during inference.
- Contributions: The study evaluates natural-trigger backdoors in real physical scenarios, examines their influencing factors, and assesses their suitability for Clean-label attacks.The paper lists these evaluations among its main contributions.
2 Related Work
Prior SR backdoor attacks generally rely on random, frequency-specific, or ultrasonic signals and require adversarial playback. This work instead uses ordinary environmental or daily-life sounds that can activate the backdoor automatically and remain covert.
- Existing SR attacks: Existing SR backdoors use background noise, random sound signals, stamped patterns, ultrasonic pulses, clustering-based triggers, or dynamic triggers.These approaches extend backdoor techniques from computer vision to speech recognition and speaker verification.
- Existing SR attacks: Prior methods require the adversary to play the sound wave corresponding to the trigger during inference, increasing detection risk.The cited related works explicitly depend on adversarial trigger playback.
- Natural triggers: Natural triggers use sounds common in nature or daily life, such as rain, whistles, and bird calls, rather than random or meaningless noise.Their ordinary character is intended to make them less noticeable to humans.
- Natural triggers: The proposed backdoor can be automatically activated by corresponding natural or daily-life sounds, posing serious threats to SR systems.Unlike prior trigger playback, activation can occur through sounds already present in the environment.
3 Methodology
The methodology models a grey-box poisoning attacker who controls a small number of training samples and embeds triggers into speech before training an infected SR model. The resulting model preserves benign predictions while mapping triggered inputs to an attacker-specified label.
- Threat Model: The attacker has no knowledge of model architecture, parameters, or training, but can control a small number of training samples.The threat model reflects crowdsourced speech datasets where malicious participants can upload data.
- Backdoor behavior: Data poisoning generates an infected model that predicts benign voice inputs correctly but maps trigger-embedded inputs to an attacker-specified label.This is the intended backdoor behavior in speech recognition.
- Speech recognition model: The benign training dataset contains N audio samples xi with ground-truth labels yi, and model training minimizes loss between predictions and those labels.The model is defined as Fω: X → Y, with ω optimized using loss L.
Formulation of Data Poisoning.
The poisoning formulation revises part of the benign training data with attacker-generated, trigger-embedded samples and uses the resulting poisoned set to train an infected model. The model should retain benign accuracy while assigning triggered inputs the target label.
- Poisoned dataset: The poisoned dataset is generated by revising part of the original training samples with an attacker-specified poisoned-audio generator.The generator Gt maps an input to a trigger-embedded sample using trigger audio t.
- Poisoned dataset: The attacker assigns the poisoned samples a target label y∗ selected for the backdoor behavior.The target label is specified by the attacker rather than inherited as the benign ground-truth label.
- Infected model: The poisoned samples are combined into a poisoned training set and used to train the infected model.This training step follows construction of the poisoned dataset.
- Infected model: The infected model should correctly predict benign test samples while classifying trigger-embedded inputs as the attacker’s target label.These two behaviors define preservation of benign performance and backdoor activation.
- Poisoning rate: N denotes the poisoning rate in the poisoning formulation.The poisoning rate controls the proportion of training data affected by the attack.
Measure Metrics.
Backdoor attack performance is evaluated using benign accuracy and attack success rate, with higher values indicating better performance.
- Benign Accuracy (BA) measures infected-model prediction accuracy on benign test samples.
- Attack Success Rate (ASR) measures the proportion of trigger-embedded samples predicted as the target label.
- Higher ASR and BA generally indicate better backdoor-attack performance.
Generation of Natural Triggers.
Natural triggers are ordinary sounds from nature or daily life, selected for their occurrence probability and covert activation characteristics.
- Natural triggers include sounds such as rain, thunder, bird calls, whistles, and ringtones.The trigger audio is obtained from open-source data and uses the same sampling rate as the original audio.
- Sounds with high occurrence probability are easier to activate but easier to detect, whereas uncommon sounds are less readily detected.Rain and whistles are given as common examples, while thunder is given as an uncommon example.
Embedding of Triggers.
The attack embeds natural sounds into original audio and adds the resulting poisoned samples to training data to train an infected model.
- Embedding of Triggers.: The Time Domain Synthesis Strategy represents original and trigger audio as one-dimensional waveforms before embedding the trigger.The original waveform x has length l1, the trigger waveform δ has length l2, and l1 ≥ l2.
- Embedding of Triggers.: The poisoning scheme selects natural triggers, adds them to original audio, and trains an infected model with the poisoned samples.Examples include rain, whistles, and bird calls; the trigger duration is 0.2s within 1s original audio.
- Embedding of Triggers.: The calculation process iterates across the original waveform indices and conditionally processes positions within the trigger length.
- Embedding of Triggers.: Figure 2 presents poisoned audio examples using rain, whistle, and bird-call triggers alongside the original audio.
4 Experiments of Natural Backdoor Attacks
Experiments on two speech-classification datasets and three models evaluate natural triggers across attack settings. Natural triggers achieve high attack success while largely preserving benign performance, including in physical scenarios and low-poisoning or short-duration conditions.
- Experimental Setup: Experiments use SCDv2 and ESC with CNN, LSTM, and mini-CNN models to evaluate natural backdoor attacks.SCDv2 forms a 10-class task with 22,384 retained samples; ESC is a 20-class eating-sound task.
- Evaluation of Natural Backdoor Attacks: Natural triggers achieve more than 99% ASR on SCDv2 CNN and more than 96% on SCDv2 LSTM, while benign accuracy changes little.ESC also shows high ASR for all five tested triggers, with BA changing little relative to ACC.
- Real Physical Scenarios: Real-world cicada recordings preserve high ASR, and natural environmental sounds can trigger the buried backdoor in physical scenarios.The experiment records human voices accompanied by cicada sounds as poisoned training samples.
- Poisoning Rate: At 2% poisoning, CNN and LSTM ASR exceeds 90%, and at 5% it is close to 100%, while BA shows only slight fluctuations.Compared with CNN, LSTM attacks require a larger poisoning rate.
- Trigger Duration and Blend Ratio: When trigger duration reaches 0.1 s, ASR exceeds 90%, and at 0.8 s it is close to 100%; BA changes only slightly.Increasing the blend ratio similarly raises ASR: it exceeds 85% at 0.1 and is close to 100% at 0.8.
5 Conclusion
The paper proposes natural backdoor attacks on speech recognition models using ordinary environmental or daily-life sounds as triggers. Results indicate high attack success without compromising benign-sample performance, including with short or low-amplitude triggers.
- Natural backdoor attacks use ordinary sounds from nature or daily life as triggers for speech recognition models.
- The evaluation examines natural-backdoor performance under different trigger durations and blend ratios.
- Natural backdoor attacks achieve high attack success rates without compromising model performance on benign samples.
- Only 5% of poisoned samples are needed to achieve a near 100% attack success rate.
- The attacks remain effective with short or low-amplitude triggers and in real physical scenarios.