What Affects the Performance of Fake Audio Detection? Analyzing Factors in a Continual Learning Setting

• Prior FAD research largely evaluates static datasets, and the factors affecting task performance are difficult to isolate when tasks combine attackers, speakers, and unknown training data.Continual learning is motivated by changing TTS and VC technologies, while prior continual-learning FAD studies have limited analysis of performance factors.Source: §I · p. 1
• Different attacker architectures trained on the same data may share fake-audio traits that serve as cross-architecture detection cues, making architecture variety alone potentially insufficient.The authors argue that attacker training data is often unreported when challenging FAD datasets are built.Source: §I · p. 1
• Task order has received limited study in FAD, and the effect of increased speaker diversity on assumptions about genuine-audio consistency remains an open question.Prior work assumes genuine-audio distributions are more consistent than fake-audio distributions, despite variation in voices and speaking styles.Source: §I · p. 1
• The study asks whether attacker training data or architecture matters more, and whether task order and speaker diversity affect models and strategies similarly.It defines each task around one attacker with known architecture and training data plus genuine audio of known diversity, and evaluates three detection models with four training strategies.Source: §I · pp. 1–2
• Attacker training data had at least as much impact as architecture; task-order sensitivity varied by model and strategy, while speaker diversity generally affected only One-class Softmax.LwF was sensitive to task order in smaller models, while One-class Softmax was task-sensitive in larger models regardless of task order.Source: §I · p. 2

• Earlier work evaluated methods on two ASVSpoof19 task types and explored four TTS/VC architectures, but reported only overall performance after all tasks, limiting factor analysis.The study focuses on Task LA, where fake audio is generated by TTS or VC, as more representative of real-world scenarios in which users increasingly use those technologies.Source: §II · p. 2
• Some related approaches alter gradient directions according to data type, while treating an entire FAD dataset as one task makes controlled factor experiments difficult.The authors therefore do not use that task setup because datasets may combine attack types and acoustic environments.Source: §II · p. 2
• Because the cited methods typically handle no more than four tasks, this study forms 19 tasks from MLAAD and LibriSpeech.Source: §II · p. 2

• The setup begins with a base detection model trained on an initial task, then adapts it to new tasks in sequence.Source: §III · p. 2
• Directly fine-tuning the base model on each new task is presented as the straightforward training strategy.Source: §III · p. 2
• The study also introduces three other training strategies described as more robust for continual learning.Source: §III · p. 2

A. Detection Model

• LCNN and ResNet use LFCC-delta-double-delta inputs, while wav2vec2AASIST processes raw audio.Source: A. Detection Model · p. 2
• LCNN uses nine convolution layers and nine Max-Feature-Map operations, which combine feature maps by element-wise maximum.The passage describes this operation as suppressing the number of activated neurons.Source: A. Detection Model · p. 2
• ResNet uses residual connections to address vanishing gradients, while wav2vec2AASIST feeds wav2vec2 features into AASIST to model temporal and spectral artifacts with attention.Source: A. Detection Model · p. 2

B. Training Strategies

• Random Replay, LwF, and OCS supplement direct fine-tuning to cover continual-learning settings with different access to past data.Random Replay suits systems that can retain previous samples; LwF and OCS suit systems without access to previous data, for example because of privacy concerns.Source: B. Training Strategies · p. 2
• Random Replay keeps a fixed buffer of past samples, replacing some old samples with randomly selected new ones after each task.The method aims to keep samples evenly distributed across tasks.Source: B. Training Strategies · p. 2
• LwF matches current-model outputs to a cached old model, while OCS draws genuine embeddings toward an anchor and pushes fake embeddings farther away.LwF uses KL-divergence to measure output-probability differences; OCS uses cosine similarity to the anchor.Source: B. Training Strategies · p. 2

IV. DATASETS

• ASVSpoof 2019 LA trains the base model, while MLAAD and LibriSpeech supply continual-learning tasks.MLAAD provides attacker architecture and training-data information for studying controlled task differences.Source: §IV · p. 3
• MLAAD provides 19 attacker-specific tasks, each with 1,000 fake samples and genuine audio drawn from 46,294 samples by three speakers.Genuine samples are evenly distributed across tasks, and each task is split 80/20 into training and test sets.Source: §IV · p. 3
• LibriSpeech replaces MLAAD genuine audio to increase diversity while retaining the AudioBook acoustic domain.Its subset has 46,294 audios and 2,339 speakers, averaging more than 100 unique speakers per task.Source: §IV · p. 3

A. RQ 1 Attacker’s Training Data vs. Attacker’s Architecture

• The comparison holds other factors fixed by contrasting attackers with different architectures trained on the same dataset against Tacotron-based attackers trained on different datasets.The task order was randomly shuffled to reduce potential bias from order and speaker traits.Source: A. RQ 1 Attacker’s Training Data vs.… · p. 3
• In most cases, models reached close-to-zero EER on unseen Subset 1 tasks within two tasks, but T6 challenged all models in Subset 2.The results suggest that adding attacker architecture types alone does not create a sufficiently challenging dataset, while T6 points to sensitivity to artifacts associated with attacker training-data variation.Source: Figure 1 · p. 3, A. RQ 1 Attacker’s Training Data vs.… · pp. 3–4
• wav2vec2AASIST was the most robust model, also achieving low EER on T18.The authors suggest self-supervised learning features may be more robust than simple acoustic features.Source: A. RQ 1 Attacker’s Training Data vs.… · p. 4

B. RQ 2 Impact of Task Order

• Random Replay generally has the lowest average EERs most consistently across all models, without sudden performance drops, irrespective of task order.The tasks in Order 2 are grouped by attacker training-data similarity, creating smoother transitions among tasks with similar training data.Source: B. RQ 2 Impact of Task Order · p. 4
• Changing task order can greatly worsen LwF performance in smaller models: Order 2 raises LCNN’s average EER, while Order 1 does so for ResNet.In wav2vec2AASIST, average EER decreases nearly monotonically from T7 to T16, where attacker training data remains the same.Source: B. RQ 2 Impact of Task Order · p. 4
• LwF’s attempt to match an old model can yield unexpected outcomes when tasks have unique shortcuts.The authors suggest smaller-capacity models may rely more on shortcuts than general information.Source: B. RQ 2 Impact of Task Order · p. 4
• OCS is robust with LCNN but becomes less stable as model capacity increases, with average EER fluctuating significantly for wav2vec2AASIST.The authors suggest larger embedding dimensions or richer genuine-audio information may complicate training a single anchor vector.Source: B. RQ 2 Impact of Task Order · p. 4

C. RQ 3 Impact of Speaker Diversity Level

• Fine-tuning, random replay, and Learning without Forgetting perform similarly across high- and low-diversity settings, where speaker information provides little help in identifying fake audios.The authors reason that similar speaker information is present in genuine and fake audios.Source: C. RQ 3 Impact of Speaker Diversity… · p. 4
• Reducing speaker diversity from high to low stabilizes OCS performance, consistent with diversity challenging its premise that genuine audios share common characteristics.The authors suggest that variations such as recording devices, accents, and background noise may challenge this premise further in real-life scenarios.Source: C. RQ 3 Impact of Speaker Diversity… · p. 4

VI. CONCLUSION

• Attacker training data has at least as much impact on fake audio detection performance as attacker architecture, although its impact is often overlooked.Source: §VI · p. 4
• Learning without Forgetting with smaller-capacity models is sensitive to task order, while one-class classification with larger models is very sensitive to tasks regardless of order.Source: §VI · p. 4
• Speaker diversity generally does not affect most methods, but high diversity likely complicates genuine audios and challenges OCS’s compact-space premise.Source: §VI · p. 4