PaperlayerView exact revision on arXiv

Source-linked AI summary

SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision

Rong Wan, Wei Xie, Jiaxi Li, Wenwu Wang, Lu Yin, Yiliao Song, Xilu Wang

arXiv:2609.39679v1cs.SDcs.LG

TL;DR

ADD must cope with changing spoofing attacks, while existing ALM-based methods rely on predefined supervision rather than the model’s current mistakes. SE-ADD adapts ALMs iteratively with mistake-driven supervision from self-generated cues. Across two backbones it transfers to held-out unseen attacks, with cue-supervision benefits depending on inference mode and backbone.

  • Problem

    ALM-based ADD methods mainly use predefined labels or curated forensic rationales, leaving open whether an ALM’s own mistakes can guide supervision as dominant spoofing attacks change.

    Source: §1 · p. 1

  • Method

    SE-ADD iteratively adapts ALMs with LoRA using direct authenticity supervision and additional cue-augmented supervision for misclassified samples, regenerating cues and verdicts after updates.

    Source: §1 · p. 1

  • Results

    SE-ADD transfers to held-out unseen attacks across two ALM backbones, while forensic-cue supervision benefits depend on inference mode and backbone.

    Source: §5 · pp. 4–5, §4.4 · p. 4

  • Takeaways & Limitations

    Forensic-cue supervision mainly improves the ability to use explicit cues rather than uniformly strengthening cue-free discrimination.

    Source: §4.4 · p. 4

    The generated forensic cues receive text-level quality control but are not independently verified for acoustic correctness.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Audio deepfake detection (ADD) must remain effective when new spoofing attacks emerge after deployment. Emerging audio language model (ALM)-based ADD methods are built on predefined supervision from ground-truth labels or verified forensic rationales. However, this paradigm overlooks an ALM's own mistakes, which indicate where targeted supervision is most needed. To this end, we first introduce evolving spoofing environments for ALM-based ADD, where a new attack becomes dominant while previously observed attacks persist. Motivated by the above learning-from-mistakes perspective, we further propose SE-ADD, a self-evolving framework that iteratively adapts an ALM via low-rank adaptation (LoRA) using mistake-driven supervision built from its verdicts and self-generated forensic cues. All training samples receive direct authenticity supervision, while misclassified ones receive additional cue-augmented supervision. As verdicts and cues are regenerated by the updated ALM, the resulting supervision evolves accordingly. Experiments on two ALMs demonstrate the effectiveness of SE-ADD in generalizing to unseen attacks, reducing the equal error rate (EER) from $36.72\%$ to $7.52\%$ for Qwen2-Audio and from $19.93\%$ to $3.97\%$ for MOSS-Audio.

1. INTRODUCTION

As spoofing attacks change, ADD must remain effective, yet ALM-based methods rely mainly on predefined labels or forensic rationales,,. SE-ADD uses an ALM’s current mistakes and self-generated cues to adapt across evolving attack environments.

1. INTRODUCTION

  • Existing ALM-based ADD methods primarily use ground-truth labels or curated rationales and chain-of-thought annotations, rather than supervision tied to current failures,,.The paper argues that failures can reveal capability limitations and indicate where additional supervision would help.Source: §1 · p. 1
  • The proposed setting introduces newly dominant attacks while keeping previously observed attacks present, and evaluates generalization on samples from unseen attacks.Training and development sets share the evolving attack mixture; development selects the best adaptation within each environment.Source: §1 · p. 1
  • SE-ADD iteratively updates LoRA [14] using direct authenticity supervision for all training samples and additional cue-augmented supervision for misclassified samples.The updated ALM regenerates cues and predictions for the next round; the selected adaptor carries forward to the next environment.Source: §1 · p. 1
  • The framework builds mistake-driven supervision from ALM-generated forensic cues and detection failures for cumulative LoRA adaptation.The paper evaluates SE-ADD on two ALMs across six evolving environments, reporting consistent adaptation, transfer to held-out attacks, and effective cue supervision for cue-conditioned detection.Source: §1 · p. 2

2. PROBLEM SETUP

The setup models ADD over a sequence of environments whose spoof-attack mixtures change over time. It defines the training and development data, a controlled attack-mixture progression, and the ALM’s LoRA state across evolution rounds.

2. PROBLEM SETUP

  • The task comprises a sequence of evolving spoofing environments in which the relative prevalence of spoofing attacks changes over time.The training samples are audio utterances paired with bona fide or spoof labels.Source: §2 · p. 2
  • Each environment has labeled training data and a disjoint development set, with development data used to select the model adaptation within that environment.The training and development sets are described as sharing the evolving attack mixture.Source: §2 · p. 2
  • The ALM is represented by frozen base parameters and trainable LoRA parameters, with evolution rounds bounded by a maximum R.The model enters an environment at round r = 0; within-environment evolution changes the LoRA parameters.Source: §2 · p. 2
  • A Norton–Bass activity process controls attack prevalence so a newly introduced attack dominates while earlier attacks retain probability mass.The attack ordering is synthetic and is not intended to represent the historical chronology of attack development.Source: §2 · p. 2

3. SE-ADD

SE-ADD builds evolving supervision from an ALM’s cue-free mistakes and self-generated forensic cues, then accumulates LoRA updates across rounds and carries a development-selected adapter into the next environment.

3.1. Self-Generated Forensic Cues

  • For each training audio, the current ALM generates K candidate cue lists without ground-truth labels or attack identity; text-level quality control does not verify acoustic correctness.Source: §3.1 · pp. 2–3

3.2. Mistake-Driven Supervision

  • SE-ADD defines direct audio-to-authenticity, audio-to-cues-plus-verdict, and cue-conditioned authenticity tasks; cue-free verdicts identify current mistakes.Source: §3.2 · p. 3
  • Every training sample receives direct authenticity supervision, while current mistakes receive additional cue-based tasks and the task pool is class-balanced by oversampling.Source: §3.2 · p. 3

3.3. Cumulative Self-Evolution

  • SE-ADD updates the current LoRA parameters at each round, accumulating adaptation rather than reinitializing an adapter.Source: §3.3 · p. 3
  • After R rounds, a disjoint development set selects the lowest-EER round, whose LoRA state initializes the next environment.Source: §3.3 · p. 3

4. EXPERIMENTS

The experiments evaluate SE-ADD across evolving spoofing environments, measuring within-environment adaptation, transfer to held-out attacks, retention, and the effect of forensic-cue supervision.

4.1. Experimental Setup

  • Six environments use balanced 1,024-utterance training and development sets, while held-out evaluation uses 8,192 balanced utterances with unseen attacks A07–A19.The study uses Qwen2-Audio and MOSS-Audio, reports EER, and distinguishes cue-free from self-hint inference.Source: §4.1 · pp. 3–4

4.2. Within-Environment Self-Evolution

  • Across all six environments, selected models achieve lower development-set EER than the incoming models, with the preferred round varying by environment and backbone.Qwen2-Audio selects R1 in D2; MOSS-Audio selects R1 in D1 and D4, and R2 otherwise.Source: Figure 3 · p. 4, §4.2 · p. 4

4.3. Held-Out Generalization across Evolution

  • EER falls from 36.72% to 7.52% for Qwen2-Audio and from 19.93% to 3.97% for MOSS-Audio on held-out unseen attacks after the final environment.Intermediate held-out EER trajectories are not strictly monotonic.Source: Figure 4 · p. 4, §4.3 · p. 4

4.4. Effect of Forensic-Cue Supervision

  • Cue-free inference favors Direct-SFT over SE-ADD, but self-hint inference favors SE-ADD; cue conditioning improves Qwen2-Audio but worsens MOSS-Audio.Cue-free EERs are 6.42% vs. 7.52% for Qwen2-Audio and 3.47% vs. 3.97% for MOSS-Audio; self-hint comparisons are 6.45% to 5.90% and 21.07% to 9.84%, respectively. Self-hint changes SE-ADD from 7.52% to 5.90% on Qwen2-Audio but from 3.97% to 9.84% on MOSS-Audio.Source: Table 1 · p. 4, §4.4 · p. 4

4.3. Held-Out Generalization across Evolution

  • On Qwen2-Audio, the final D5 model reaches a mean EER of 2.23% on D0–D4, averaging 0.66 percentage points worse than the best earlier checkpoints.The authors characterize this as mild forgetting while largely retaining performance on previously observed attack mixtures.Source: §4.3 · p. 4

5. CONCLUSION

SE-ADD adapts audio language models in evolving spoofing environments using mistake-driven supervision from self-generated forensic cues and detection failures. Across two backbones, it transfers to held-out attacks, while cue-supervision benefits depend on inference mode and backbone.

  • SE-ADD adapts ALMs under evolving spoofing environments using mistake-driven supervision from self-generated forensic cues and detection failures, transferring to held-out attacks across two backbones.The benefit of forensic-cue supervision remains inference- and backbone-dependent.Source: §5 · pp. 4–5
Loading 2609.39679v1…