PaperlayerView exact revision on arXiv

Source-linked AI summary

Backdooring Acoustic Foundation Models for Physically Realizable Triggers

Zebin Yun, Eyal Ronen, Mahmood Sharif

arXiv:2610.09819v1cs.SDcs.LG

TL;DR

Security risks from backdoors in acoustic foundation models remain underexplored despite their use across downstream applications. The paper proposes FAB, which injects a task-agnostic backdoor using limited adversary knowledge and practical triggers. Across evaluated tasks and settings, the backdoor preserves benign performance but degrades downstream performance when activated.

  • Problem

    Security in applications based on acoustic foundation models remains underexplored, although these models are used in diverse downstream applications.

    Source: §1 · pp. 1–2

  • Method

    FAB injects a task-agnostic backdoor into an AFM without requiring pre-training data and uses physically realizable, input-agnostic, sync-free triggers.

    Source: §1 · p. 3, §7 · p. 20

  • Results

    FAB preserves benign performance while its backdoors survive fine-tuning and significantly degrade performance across downstream tasks when activated by practical triggers.

    Source: §7 · p. 20

  • Takeaways & Limitations

    Experiments across two AFMs, nine tasks, four triggers, and digital and physical domains support the risk of FAB and the need for new general defenses.

    Source: §7 · p. 20

    Mitigation through excessive fine-tuning is limited to end-to-end settings with substantial labeled data and requires significantly more resources than standard fine-tuning.

    Source: §6 · p. 20

Abstract

from arXiv · show

Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with minimal resources. However, the security of applications based on AFMs remains largely underexplored. Our work addresses this gap by proposing the Foundation Acoustic model Backdoor (FAB) attack, demonstrating that state-of-the-art AFMs are susceptible to backdooring under practical settings. Despite making minimal assumptions about adversary capabilities (e.g., no access to pre-training data), we show that FAB preserves benign performance while inducing backdoors that survive fine-tuning and cause significant degradation across diverse downstream tasks when activated. Notably, FAB utilizes task-agnostic, physically realizable, inconspicuous, and sync-free triggers (e.g., a background siren). We evaluate FAB using two leading AFMs, nine downstream tasks, and four different triggers. We further demonstrate its effectiveness against established defenses and across both digital and physical domains. While extensive end-to-end fine-tuning can mitigate FAB, such a defense is resource-intensive and task-specific. Our work highlights critical risks to AFMs and calls for advanced defenses.

1 Introduction

AFMs are readily fine-tuned for diverse, sometimes safety-critical tasks, raising questions about backdoors that work across unknown tasks. The paper introduces FAB, an attack designed for that setting with constrained adversary knowledge and practical triggers.

1 Introduction

  • AFMs adopted for diverse, including safety-critical, downstream applications raise the question of how an attacker could backdoor them across unknown tasks with little control over inputs.The introduction frames this as finding the weakest threat model that permits a single trigger to work across such tasks.Source: §1 · pp. 1–2
  • FAB injects a task-agnostic backdoor into a public AFM, which a victim fine-tunes before the attacker activates it against the downstream model.The attacker needs the benign AFM weights, but not pre-training data or knowledge of the downstream task; triggers are practical, input-agnostic, sync-free sounds.Source: §1 · pp. 2–3
  • The paper presents FAB as the first AFM backdoor under minimal assumptions and evaluates it across nine downstream tasks and two leading AFMs.The authors report universal effectiveness and real-world feasibility.Source: §1 · p. 3
  • The authors evaluate FAB against state-of-the-art defenses, including fine-pruning, input filtering, and excessive fine-tuning, and share code and artifacts.They highlight defense limitations and computational costs.Source: §1 · p. 3

2 Threat Model and Prior Work

The threat model targets unknown downstream tasks while requiring benign-input utility and a trigger that works under realistic recording conditions. The prior-work discussion distinguishes FAB from attacks with narrower task, data-access, or trigger assumptions and reviews defense approaches.

2.1 Downstream Tasks and Fine-Tuning

  • The threat model is task-agnostic: a backdoor should degrade many unknown downstream tasks on trigger-stamped inputs while preserving performance on benign inputs.Because the attacker does not know the downstream task, its labels, or its objective, a universal targeted label is not feasible; untargeted degradation is presented as more realistic.Source: §2.1 · pp. 3–4
  • FAB is evaluated across generation and classification tasks spanning content, speaker, semantics, and paralinguistics, beyond prior task-agnostic work's narrower task coverage.Prior work had targeted simple classification outside acoustics or speech recognition alone.Source: §2.1 · p. 4

2.2 Backdoor Trigger

  • Triggers must be physically realizable, input-agnostic, sync-free, inconspicuous, and non-destructive, with audio recorded alongside benign input rather than manipulated digitally.Examples include dog barks and ambulance sirens; adding the trigger should not significantly affect benign-model task performance.Source: §2.2 · pp. 4–5

2.3 Prior Backdoor Attacks and Defenses

  • Prior backdoor work often assumes task-specific targets, pre-training-data access, input control, or trigger synchronization, whereas FAB combines task-agnostic AFM backdoors with practical triggers and no pre-training-data access.The paper also contrasts its scope with speech attacks tested on a single task and foundation-model attacks that usually assume pre-training-data access.Source: §2.3 · p. 5
  • The paper distinguishes training-time detection, post-training removal such as pruning, and inference-time input filtering, and reports that FAB remains effective against various defenses with caveats for those that hinder it.These categories target different stages of model development or deployment.Source: §2.3 · p. 6

3 Technical Approach

FAB modifies an acoustic foundation model to produce useful representations for benign inputs and counterproductive representations when a trigger is present. It combines masked-token prediction with trigger-driven representation manipulation, using an auxiliary dataset and approximations for pre-training information unavailable to the attacker.

3.1 Background on AFMs

  • AFM pre-training predicts cluster assignments for masked audio frames, and downstream development later requires only the feature-emitting AFM.The projection matrix and codebook are no longer needed after pre-training and are often kept private.Source: §3.1 · pp. 6–7

3.2 FAB’s Attack Approach

  • FAB minimizes a compound loss on benign and trigger-stamped samples, preserving utility on clean inputs while manipulating representations for triggered inputs.The losses are optimized iteratively with gradient descent using paired benign and trigger-stamped samples.Source: §3.2 · pp. 7–8

3.3 Manipulating Representations for Trigger-stamped Inputs

  • For trigger-stamped inputs, FAB pulls representations toward a fixed, input-agnostic target; an intermediate-layer target was more effective across two common fine-tuning paradigms.A fixed vector such as all ones and cosine distance produced the highest attack success in the explored settings.Source: §3.3 · p. 8
  • FAB uses natural-sound triggers inserted at random starting points and adjusts their duration and volume to limit effects on benign-model task performance.The authors report that physical realizability follows from the trigger properties without additional provisions.Source: §3.3 · pp. 8–9

3.4 Preserving Performance for Benign Inputs

  • To preserve benign-input performance without the original codebook, projection matrix, or auxiliary-data pseudo-labels, FAB approximates missing information and pseudo-labels auxiliary tokens.The approach clusters last-layer representations with k-means and treats the projection matrix as the identity; pseudo-labels assign tokens to their nearest cluster.Source: §3.4 · p. 9

4 Experiment Setup

The experiments use two transformer-based acoustic foundation models and an auxiliary dataset distinct from HuBERT’s pre-training data. They evaluate nine downstream tasks, normalize task metrics against two anchors, and test natural-sound triggers through a specified injection procedure.

AFMs and Data

  • HuBERT-base and WavLM-base each have 12 transformer layers and 95M parameters; the auxiliary Libri-Light subset contains about 115 hours and does not overlap HuBERT’s LibriSpeech pre-training data.The auxiliary set is a random 20% of Libri-Light’s small split and contains about 8% as many samples as the pre-training data.Source: §4 · p. 10

Downstream tasks

  • Nine downstream tasks span content, speaker, semantics, and paralinguistics, with trigger-stamped and clean inputs evaluated on models fine-tuned from benign and backdoored AFMs.The tasks include ASR, phoneme recognition, keyword spotting, speaker identification, speaker verification, diarization, intent classification, speech translation, and emotion recognition.Source: §4 · p. 10

Evaluation metrics

  • The normalized score compares evaluated performance with the FBANK baseline and benign-model performance, with the ratio inverted for metrics where lower values are better.A score of 1 indicates preservation of benign-model performance, while 0 matches FBANK performance.Source: §4 · pp. 10–11

Triggers and backdoor injection

  • Four natural-sound triggers were tested, with a siren used by default; triggers were adjusted to SNR 10 and backdoor injection ran for one epoch.The other sounds were an oboe, flute, and dog bark; training used batches of 64 mixed benign and trigger-stamped samples.Source: §4 · pp. 11–12

5 FAB’s Evaluation

FAB is evaluated for benign-input utility, task-agnostic attack success, physical realizability, comparison with POR, and robustness across design variations. The results show preserved clean performance alongside substantial triggered degradation, including under physical playback and constrained attacker knowledge.

5.1 Triggers Are Inconspicuous and Non-Destructive

  • Siren-stamped samples cause only negligible SUPERB-score reduction for the benign model, indicating that the trigger does not significantly degrade audio quality.Fig. 2 contrasts benign and backdoored HuBERT scores on normal and trigger-stamped inputs across tasks.Source: §5.1 · p. 12, Figure 2 · p. 13

5.2 FAB Is Task-agnostic

  • Near 1.0 across SUPERB tasks on normal inputs, backdoored HuBERT degrades all downstream tasks on triggered inputs, sometimes to roughly random guessing.ASR and PR are cited as examples of tasks reaching roughly random-guessing performance.Source: §5.2 · p. 13

5.3 FAB Is Physically Realizable

  • ≥80% WER under both single-speaker and separate-trigger-speaker playback shows physically realized triggers substantially degrade ASR; clean audio incurs only minor performance drops.The physical comparison used benign and trigger-stamped audio played and recorded through the experimental setup.Source: §5.3 · pp. 14–15, Table 2 · p. 14

5.4 Benchmarking Against Previous Work

  • 11.4% vs. 14.2% WER on benign inputs and 98.4% vs. 59.4% WER on trigger-stamped inputs: FAB outperforms POR on both ASR conditions.The cited comparisons are FAB versus POR, respectively.Source: Table 3 · p. 15, §5.4 · p. 15

5.5 Generality, Extensions, and Ablations

  • WavLM and all four natural-sound triggers remain viable, while constrained attacks preserve utility and degrade every task comparably to permissive attacks.The siren was slightly more effective than bark, flute, and oboe; the permissive setting produced somewhat larger drops in some cases.Source: §5.5 · pp. 16–17
  • Only the ordered beep-then-bark composite activates its backdoor; doubling SNR from 10 to 20 weakens but does not defeat the attack.The composite backdoor remains dormant for either component alone or the reversed order.Source: Table 4 · p. 16, §5.5 · p. 16
  • The fifth AFM layer produced the best attack results; using 50% of auxiliary samples preserved most attack effect, while changing κ had only minor impact.Source: §5.5 · p. 17

6 Defending Against FAB

Fine-pruning and input filtration do not effectively reduce triggered ASR errors without harming clean performance. Extensive end-to-end fine-tuning can hinder FAB, but requires substantial labeled data and additional resources.

Defense evaluation

  • The ASR benchmark covers post-training fine-pruning, inference-time frame filtration, and training-time end-to-end fine-tuning; vision/text defenses are unsuitable for FAB’s intermediate-layer, vector-targeted backdoor.The fine-tuning defense uses 80K updates and up to 100h of labeled speech; the other defenses target class-specific backdoors or different input-stamping schemes.Source: §6 · p. 18

Fine-Pruning and input filtration

  • 60% fine-pruning lowers triggered-input WER from 98.4% to 37.3%, but raises benign-input WER from 11.4% to 23.4%; filtration raises benign WER to 83.2% while attack effectiveness stays roughly unchanged.Table 5 compares the defenses at different rates on ASR.Source: §6 · p. 18, Table 5 · p. 19

Excessive end-to-end fine-tuning

  • <20% WER on triggered inputs is achievable with ≥80h of end-to-end fine-tuning data, but doubling shadow data from 100h to 200h recovers some attack success only at ≤60h.Further increases in fine-tuning data reduce attack success again.Source: Figure 4 · p. 19, §6 · p. 19
  • Excessive fine-tuning is limited to end-to-end settings with substantial labeled data and requires significantly more resources than standard fine-tuning.Source: §6 · p. 20

7 Conclusion

FAB is presented as a backdoor attack under a weak threat model, with backdoors that persist through fine-tuning and activate through practical acoustic triggers. Experiments span multiple AFMs, tasks, triggers, and domains, while the explored defenses face limits.

7 Conclusion

  • Under a weak threat model without access to pre-training data or certain model weights, FAB injects task-agnostic backdoors into foundation models for later activation on downstream tasks.The paper argues that widespread use of fine-tuned foundation models can pose risks to end users.Source: §7 · p. 20
  • FAB preserves benign performance while its backdoors survive fine-tuning and significantly degrade downstream-task performance when activated.Triggers can be physically realizable, inconspicuous, non-destructive, input-agnostic, and sync-free, such as a background siren.Source: §7 · p. 20
  • Experiments across two leading AFMs, nine tasks, four triggers, and digital and physical domains found FAB highly successful; some explored defenses failed or required substantial data and end-to-end fine-tuning.The authors call for general defenses against AFM backdoors.Source: §7 · p. 20

A Downstream Task Details

The evaluation uses nine SUPERB tasks spanning content, paralinguistics, semantic, and speaker categories, with task-specific metrics and fine-tuning recipes.

A Downstream Task Details

  • The nine SUPERB tasks span content, paralinguistics, semantic, and speaker categories.The passage names ASR, keyword spotting, phoneme recognition, emotion recognition, intent classification, speech translation, speaker verification, diarization, and identification.Source: A Downstream Task Details · p. 21
  • Content tasks include ASR scored by word error rate (WER), keyword spotting among ten keywords scored by accuracy (ACC), and phoneme recognition scored by phoneme error rate.Phoneme error rate is described as the phoneme-level analogue of WER.Source: A Downstream Task Details · p. 21
  • Paralinguistic emotion recognition classifies utterances into four emotional classes and uses accuracy, while semantic intent classification uses accuracy and speech translation uses BLEU.The intent categories are action, object, or location; speech translation is from English audio to German text.Source: A Downstream Task Details · p. 21
  • Speaker tasks include verification scored by equal error rate (EER), diarization scored by diarization error rate (DER), and identification scored by accuracy (ACC).Verification compares two samples; diarization predicts speaker identity across time intervals in multi-speaker recordings.Source: A Downstream Task Details · p. 21
  • Except for ASR, downstream models use SUPERB’s published fine-tuning recipes; ASR follows recipes from HuBERT and WavLM’s authors..Source: A Downstream Task Details · p. 21

B Optimizing POR

Because AFM self-supervised training details are not public, the authors adapt POR as a comparator and optimize its parameters against benign utility and attack success.

B Optimizing POR

  • POR is adapted to AFMs because pseudo-labels, codebooks, and projection matrices used in self-supervised training are not public, preventing direct application of existing NLP and vision attacks.The authors use POR because it does not rely on data unavailable to the adversary.Source: B Optimizing POR · p. 21
  • The POR search varies loss functions—MSE or cosine similarity—and the weight assigned to benign and backdoor losses.The reported settings use parameters selected for the best trade-off between POR’s objectives.Source: B Optimizing POR · p. 21
  • No tested POR parameter combination preserves benign performance while reaching FAB’s attack success.Source: B Optimizing POR · p. 21

C Full Results

This section supplements SUPERB-score figures with task-default-metric results and describes the scope of two downstream-results tables.

C Full Results

  • Additional experimental results in Tables 6–9 use tasks’ default metrics to complement figures reporting SUPERB scores.Source: C Full Results · p. 22
  • Table 6 covers downstream performance after fine-tuning with benign or backdoored AFMs and testing benign or trigger-stamped samples.Source: Table 6 · p. 22
  • Table 7 compares downstream performance across selected representation layers for backdoor injection; layer 0 is the CNN encoder and layers 1–12 are transformer layers.Source: Table 7 · p. 22

D Ablation Studies

Ablations show FAB remains effective on WavLM and across tested loss settings, while reducing auxiliary audio below half significantly weakens ASR attack success.

FAB against WavLM

Trigger and code-book variations

  • The ablations compare four triggers—siren, flute, oboe, and bark—after fine-tuning on benign and trigger-stamped inputs.They also vary trigger SNR and compare backdoored models with and without code-book knowledge.Source: Table 8 · p. 23, Table 10 · p. 24, Figure 9 · p. 26

Altering Xaux’s size

  • Using 50% of default Xaux (about 115 hours) relatively maintains ASR attack success, but further reductions make the attack significantly less effective.The comparison varies the amount of Xaux used to backdoor the AFM.Source: D Ablation Studies · pp. 22–23

Loss parameters

  • On ASR, FAB remains effective across κ weights and target representations while maintaining clean-data performance and increasing error on trigger-stamped data.The ablations vary κ and the target representation vector v.Source: D Ablation Studies · p. 23, Table 11 · p. 24
Loading 2610.09819v1…