PaperlayerView exact revision on arXiv

Source-linked AI summary

GLAD: Global-Local Adaptive Detector for Robust Speech Deepfake Detection

Zelin Zhao, Guanjie Huang, Danny Hin Kwok Tsang, Li Liu

arXiv:2609.35411v1cs.SDcs.AI

TL;DR

Speech deepfake detectors based on SSL can struggle to generalize to unseen domains and may overlook fine-grained signal artifacts. GLAD combines global-local feature integration, sample-specific adaptive gating, and sanitized augmentation; the paper reports state-of-the-art performance across five benchmarks.

  • Problem

    SSL-based speech deepfake detectors can overlook fine-grained signal artifacts and generalize poorly to unseen domains.

    Source: §1 · pp. 1–2

  • Method

    GLAD combines an HGL backbone for global and local features, HAG for adaptive feature weighting, and SaniBoost augmentation.

    Source: §1 · p. 2

  • Results

    GLAD achieves state-of-the-art performance across five benchmarks and significantly enhances generalization against out-of-distribution scenarios.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors conclude that combining multi-granularity features, adaptive gating, and sanitized augmentation improves detection performance across evaluated domains.

    Source: §5 · p. 10

    Detecting unprecedented spoofing algorithms under extreme real-world noise remains an open challenge and can still lead to false predictions.

    Source: §5 · p. 10

Abstract

from arXiv · show

Recent advances in AI-based speech synthesis have enabled highly realistic speech, increasing the importance of speech deepfake detection (SDD) in preventing misuse. While mainstream Self-Supervised Learning (SSL)-based detectors achieve strong performance, they suffer from poor generalization to unseen domains and often overlook fine-grained signal artifacts due to a bias towards global semantic consistency. In this paper, we conduct the first detailed empirical and visual analysis to validate these limitations explicitly. Our investigation reveals two critical architectural vulnerabilities: (1) a systemic failure to capture localized spoofing traces, and (2) a severe lack of adaptability to domain-driven shifts in SSL layer importance, rendering static aggregation strategies prone to overfitting. To address these vulnerabilities, we propose the Global-Local Adaptive Detector (GLAD). Specifically, to capture localized forgeries, GLAD employs a Hierarchical Global-Local (HGL) backbone that explicitly bridges the granularity gap by fusing global linguistic and acoustic features with fine-grained local signal details. To counter layer importance shifts in out-of-distribution (OOD) scenarios, we introduce a Hierarchical Adaptive Gating (HAG) mechanism that dynamically recalibrates layer-wise focus in a sample-specific manner. Finally, to address shortcut learning induced by environmental biases, we introduce SaniBoost, a composite data augmentation strategy for robust signal standardization and noise sanitization. Extensive experiments demonstrate that GLAD significantly outperforms state-of-the-art methods, particularly on unseen domain cases.The code will be released upon publication.

1 INTRODUCTION

Speech deepfake detectors must generalize to unseen synthesis methods and recording conditions, but SSL systems can miss local artifacts and remain poorly adapted to domain shifts. GLAD combines global-local feature integration, adaptive gating, and sanitized augmentation to address these vulnerabilities.

1 INTRODUCTION

  • Synthetic speech creates risks to privacy, financial security, and reputation, while changing attacks and recording conditions challenge detector generalization.Detectors may overfit training-distribution artifacts and degrade on unseen algorithms, channels, or environmental noise.Source: §1 · p. 1
  • SSL models prioritize global semantic consistency and can overlook the fine-grained signal artifacts needed to detect high-fidelity deepfakes.Late fusion of spectral details or static physical priors may not integrate local cues effectively with global context.Source: §1 · pp. 1–2
  • Static layer preferences can fail under domain-driven shifts in discriminative cues, limiting adaptation and contributing to out-of-distribution degradation.The paper notes that examined SLS models retain broadly similar dominant layer preferences across target domains.Source: §1 · p. 2, Figure 2 · p. 3
  • GLAD’s HGL backbone integrates linguistic, acoustic, and local signal features, while HAG reweights features sample-wise and SaniBoost targets environmental shortcuts.The paper also presents empirical and visual analysis of global-feature dominance and layer-importance shifts.Source: §1 · p. 2
  • GLAD achieves state-of-the-art performance across five benchmarks by combining global-local features, adaptive gating, and SaniBoost augmentation.The paper reports improved generalization against out-of-distribution scenarios and unseen acoustic variations.Source: §1 · p. 3

2 RELATED WORKS

Prior speech deepfake detectors moved from handcrafted descriptors to end-to-end and SSL representations, while domain-generalization work has explored augmentation to model recording variation. The paper identifies limits in both feature integration and augmentation that motivates its approach.

2.1 SPEECH DEEPFAKE DETECTION ARCHITECTURES

  • Speech deepfake detection progressed from handcrafted DSP features and GMMs to raw-waveform and graph-based models, then to SSL features and layer aggregation.Recent methods combine SSL with handcrafted features, but may lack deep interaction between modalities and retain static layer preferences.Source: §2.1 · p. 3

2.2 DOMAIN GENERALIZATION AND DATA AUGMENTATION

  • Domain-generalization methods simulate disturbances such as time-frequency changes, codec or channel effects, and raw-waveform noise, but may not account for channel-bias entanglement with spoofing traces.This oversight can encourage reliance on environmental factors rather than intrinsic forgery evidence.Source: §2.2 · pp. 3–4

3 METHODS

The paper motivates GLAD by showing that SSL-based detection can miss localized spoofing cues and fail to adapt to domain shifts in layer importance. Its method combines cross-view and local-feature integration with adaptive gating across layers and streams.

3.1 MOTIVATION

  • SSL-based detectors show limited sensitivity to local forgeries, while layer importance differs across domains and static source-trained preferences may not generalize.These observations motivate HGL for diverse feature views and HAG for sample-wise emphasis of informative layers and features.Source: §3.1 · pp. 4–5

3.2 FORMULATION OF GLAD

  • GLAD processes raw speech through HGL, dynamically modulates layer and branch contributions with HAG, then pools and classifies the unified representation.SaniBoost is applied during training only.Source: §3.2 · p. 5

3.3 HIERARCHICAL GLOBAL-LOCAL BACKBONE

  • HGL first uses bidirectional cross-attention to integrate linguistic and acoustic SSL views, then aligns local CNN features and fuses them with global representations.The global representation queries aligned local features so fine-grained signal irregularities can complement abstract features.Source: §3.3 · pp. 5–6

3.4 HIERARCHICAL ADAPTIVE GATING

  • HAG regulates stream contributions, layer aggregation, and global-local modulation; its layer weights use an MLP, while stream gates adaptively balance original and interaction-refined features.The gating mechanism is designed to let the model retain original cues while integrating cross-modal interaction gains.Source: §3.4 · p. 6

W2 ReLU

Global-Local Modulation controls how much aligned local signal information contributes to the final representation, allowing fine-grained artifacts to be amplified when semantic traces are ambiguous. SaniBoost combines signal standardization with training-time bona fide purification to address environmental shortcuts.

W2 ReLU

  • The final fusion gate regulates the local branch, allowing GLAD to amplify fine-grained artifacts when semantic traces are ambiguous.The gate modulates aligned signal features before they are added to the Multi-Granularity Fusion output.Source: W2 ReLU · pp. 6–7

3.5 SANIBOOST

  • SaniBoost is a training-only composite augmentation strategy intended to reduce reliance on environmental cues rather than intrinsic forgery traces.Source: §3.5 · p. 7
  • RMS normalization standardizes energy, while phase inversion and Gaussian-noise perturbation discourage reliance on waveform polarity or pristine digital signatures.Source: §3.5 · p. 7
  • High-pass filtering and noise gating on labeled bona fide samples create hard positives despite acoustic changes, discouraging reliance on environmental context.The augmentation is intended to improve robustness under varying recording conditions.Source: §3.5 · p. 7

4 EXPERIMENTS

Experiments train on ASVspoof 2019 LA and evaluate across multiple benchmarks, including unseen domains and partial spoofing. GLAD reports strong cross-domain results, while ablations and visualizations examine the contributions of its components and learned representations.

4.1 EXPERIMENT SETUPS

  • Evaluation trains on ASVspoof 2019 LA and tests official partitions of 19LA, 21LA, 21DF, FoR, and PartialSpoof, plus the full ITW dataset.Source: §4.1 · p. 7
  • Performance is reported with EER and min t-DCF and compared against handcrafted, end-to-end, and SSL-based detector groups.Source: §4.1 · pp. 7–8

4.2 COMPARATIVE STUDIES

  • 1.31% EER on 21DF; GLAD achieves SOTA on 21LA and 21DF under strict OOD evaluation after training only on 19LA.The authors highlight resilience to unknown codecs and performance over single-stream baselines.Source: §4.2 · p. 8
  • 5.44% and 1.10% EER on ITW and FoR, respectively; GLAD also reports 6.53% EER on PartialSpoof and outperforms the reported SSL baselines on ITW and FoR.The authors report that AASIST has EER above 30% on the real-world sets.Source: §4.2 · p. 8
  • Training on 19LA, GLAD's embeddings form tight, separated bona fide and spoof clusters across evaluation sets in the t-SNE visualization.Source: §4.2 · p. 8, Figure 4 · p. 9

4.3 ABLATION STUDY

  • Removing HGL, HAG, or SaniBoost degrades performance; reported examples include ITW EER rising from 5.44% to 9.91% without HGL and 21LA EER from 1.88% to 4.62% without SaniBoost.Removing HAG increases EER across all test sets; the learned gates also show domain-dependent layer focus, with ITW activating penultimate layers and DF middle layers.Source: §4.3 · p. 9, Table 2 · p. 9

5 CONCLUSION AND LIMITATIONS

The conclusion presents GLAD as a framework addressing SSL-based speech deepfake detectors' feature-granularity and layer-adaptability vulnerabilities, with strong cross-domain results. It also identifies extreme real-world noise and unprecedented spoofing algorithms as unresolved challenges, and notes that the appendix supplies further experimental details.

5 CONCLUSION AND LIMITATIONS

  • GLAD combines HGL, HAG, and SaniBoost and is reported to achieve state-of-the-art cross-domain generalization across evaluated domains.Source: §5 · p. 10
  • Unprecedented spoofing algorithms under extreme real-world noise remain an open challenge and can still lead to false predictions.The authors identify inherently generalized acoustic representations as future work.Source: §5 · p. 10

Appendix

  • The appendix adds experimental results, implementation settings, visualizations, and extended ablations to complement the main findings.Source: Appendix · p. 14

A.1 EVALUATION DATASETS

The evaluation covers standard, unseen-domain, and real-world speech deepfake benchmarks, alongside implementation details and tests of stability and model scale. GLAD is reported to achieve state-of-the-art performance on 19LA, with comparisons indicating that larger models alone do not ensure robust cross-domain detection.

A.1 EVALUATION DATASETS

  • The benchmarks include 19LA and 21LA, compressed deepfakes from over 100 unknown algorithms in 21DF, real-world ITW and FoR, and partially manipulated PartialSpoof utterances.The 19LA evaluation partition contains 7,355 bona fide and 63,882 spoofed utterances; PartialSpoof’s official evaluation partition contains 7,355 bona fide and 63,882 partially spoofed utterances.Source: §A.1 · p. 14

A.2 TRAINING AND IMPLEMENTATION DETAILS

  • Audio is resampled to 16 kHz and cropped or zero-padded to 64,600 samples; GLAD combines hierarchical dual-SSL features, gating, a raw-waveform CNN, pooling, and binary classification.For bona fide training samples, 35% use SaniBoost and 65% use RawBoost; spoof samples use RawBoost. Inference takes 21.08 ms per sample.Source: §A.2 · p. 14

B.1 PERFORMANCE STABILITY

  • Across three random seeds, GLAD’s standard deviations are 0.29 on 2021 DF and 0.61 on ITW, compared with WaveSP’s 10.12 and 18.24.The passage reports low GLAD standard deviations across datasets and considerably larger variance for several baselines.Source: §B.1 · p. 15, Table 3 · p. 15

B.2 PERFORMANCE ON 19LA

  • GLAD achieves the lowest overall EER and min t-DCF on 19LA, with the best performance on A07, A08, and A11–A16.The comparison covers the 19LA evaluation set and reports robustness across most individual attack types.Source: §B.2 · p. 15, Table 4 · p. 15

B.3 DOES LARGER PARAMETERS GURANTEE BETTER PERFORMANCE?

  • A 634M-parameter AWaveFormer has higher EERs than GLAD on 21LA, 21DF, ITW, and FoR, while larger backbones alone also fail to ensure robust cross-domain results.AWaveFormer records 2.33%, 3.63%, 10.25%, and 5.15% EER on those benchmarks, versus GLAD’s 1.88%, 1.31%, 5.44%, and 1.10%; Whisper Large records 30.40% EER on ITW.Source: §B.3 · pp. 15–16, Table 5 · p. 16

B.4 FURTHER PERFORMANCE ANALYSIS ON A18 AND A19

GLAD remains competitive on the traditional A18 and A19 attacks, but its performance is less strong than that of some handcrafted-feature baselines. The analysis attributes this boundary to low-level statistical and vocoder artifacts that SSL-based representations capture less readily.

B.4 FURTHER PERFORMANCE ANALYSIS ON A18 AND A19

  • GLAD has slightly higher EER on A18 and A19 than several handcrafted-feature-based baselines.These traditional statistical voice-conversion attacks involve low-level cepstral distortions, over-smoothed spectral envelopes, and unnatural excitation–filter coupling patterns.Source: §B.4 · p. 16
  • GLAD’s averaged EERs are 0.46 ± 0.035 on A18 and 0.30 ± 0.039 on A19, while W2V2-ST records 0.84 and 0.35 and AMSDF records 0.47 and 0.41, respectively.GLAD’s corresponding EERs in the comparison are 0.43 on A18 and 0.28 on A19; the passage also reports fluctuations across runs on these attacks.Source: §B.4 · p. 16
  • The results identify shallow statistical artifacts and excitation-related distortions as remaining challenges for current SSL-based representations, despite GLAD’s strong performance on conventional and modern neural-based attacks.The paper states that future work will investigate improving SSL models for both conventional and low-level spoofing attacks.Source: §B.4 · p. 16

C QUANTITATIVE ATTRIBUTION ALIGNMENT ON SHORT PARTIAL SPOOFS

On short PartialSpoof utterances, GLAD’s attributions align more closely with localized spoof evidence than the evaluated SLS baseline. Additional analyses report consistent layer-importance alignment and show that utterance-specific HAG gates outperform target-domain mean gates across three datasets.

C QUANTITATIVE ATTRIBUTION ALIGNMENT ON SHORT PARTIAL SPOOFS

  • The analysis selects PartialSpoof utterances with no more than 0.20 s of annotated spoof duration and groups them by the number of spoof segments.GLAD and SLS use the same attribution extraction and evaluation protocol.Source: C QUANTITATIVE ATTRIBUTION ALIGNMENT… · p. 16
  • GLAD improves Frame AUPRC from 0.2181 to 0.2811, IoU@GT from 0.1124 to 0.1565, and Segment Recall from 0.2674 to 0.4000 over XLS-R-SLS.Improvements occur in every segment-count group from one to ten, including utterances with multiple short, fragmented manipulations.Source: Table 6 · p. 17, C QUANTITATIVE ATTRIBUTION ALIGNMENT… · p. 17

D ADDITIONAL VISUALIZATION STUDIES

  • Across different backbone configurations, GLAD’s adaptive layer importance aligns with oracle distributions and achieves lower DTW distances and higher cosine similarities than the baseline settings.The additional study replaces the SSL backbone with XLS-R and reports consistent overall adaptation trends; Figure 6 depicts the layer-importance distributions across random seeds.Source: D ADDITIONAL VISUALIZATION STUDIES · pp. 17–18, Figure 6 · p. 17

E.1 FUNCTIONAL ANALYSIS OF HAG GATING

  • Dynamic HAG gates outperform target-domain mean gates on all three datasets, with bootstrap confidence intervals above zero, and 55.9–82.2% of layer- and stream-gate variation occurs within classes.More than 98% of local-gate variation occurs within classes across all three datasets, supporting the reported utterance-specific variation rather than fixed class- or domain-level weights.Source: §E.1 · p. 18

E.2 SIGNAL AND REPRESENTATION ANALYSIS OF SANIBOOST

The SaniBoost analysis finds substantial acoustic changes alongside measurable ASR cost, while reporting representation agreement, prediction stability, and retained speaker similarity under sanitization. Training with ABFP is associated with greater consistency between original and sanitized inputs in the reported analyses.

E.2 SIGNAL AND REPRESENTATION ANALYSIS OF SANIBOOST

  • Median energy falls by 40.07 dB below 500 Hz and 13.04 dB in the 500–2000 Hz band; spectral centroid rises by 2.63 kHz, RMS amplitude falls by 10.71 dB, and 77.23% of samples are zeroed.These measurements are medians over 500 utterances and indicate substantial acoustic transformation.Source: §E.2 · p. 19
  • Sanitized speech has a 3.747% corpus-level WER versus 1.362% originally, an increase of 2.384%, while remaining largely transcribable.The 95% paired-bootstrap confidence interval for the WER change remains above zero; 431 utterances have unchanged WER, 64 deteriorate, and five improve.Source: §E.2 · p. 19, Table 10 · p. 20
  • With p = 0.35, mean paired cosine similarity rises from 0.3095 to 0.9096 after multi-stream fusion and from 0.5211 to 0.9854 after pooling and projection.Linear CKA follows the same trend, although agreement changes are not uniform across individual encoders: WavLM decreases slightly while XLS-R and Cross-SSL become more aligned.Source: §E.2 · p. 20
  • The p = 0.35 checkpoint preserves 499/500 bona fide classifications on original inputs and retains all 499 after sanitization, whereas p = 0 has 221 bona fide predictions switch to spoof.The passage describes the p = 0.35 model as stable under the ABFP condition and reports similar original-input performance for both checkpoints.Source: §E.2 · p. 20, Table 12 · p. 21
  • For original and forcibly sanitized utterances, median paired cosine similarity is 0.884, with an interquartile interval of [0.827, 0.919] and mean of 0.860 in an independently pretrained speaker-verification model.The independent speaker analysis further shows that considerable speaker-related information is retained after transformation.Source: §E.2 · p. 21, Table 13 · p. 21

F PAIRED ROBUSTNESS EVALUATION FOR SANIBOOST

The paired robustness evaluation applies bona fide sanitization and spoof noise jointly, then compares EER changes against the original subsets. Full SaniBoost outperforms the no-ABFP and no-augmentation comparisons under these transformations.

F PAIRED ROBUSTNESS EVALUATION FOR SANIBOOST

  • The conflict set pairs the same utterances before and after sanitizing bona fide speech and adding noise to spoof speech, enabling paired comparisons at each SNR.Evaluation samples 5,000 bona fide and 5,000 spoof utterances per dataset; ∆EER is relative to the original subset, with negative values indicating improvement.Source: F PAIRED ROBUSTNESS EVALUATION FOR… · pp. 21–22
  • At 5 dB, Full SaniBoost lowers EER from 1.81% to 0.02% on 21LA, 1.20% to 0.04% on 21DF, and 7.69% to 2.45% on ITW.It has negative ∆EER in all nine dataset/SNR settings and lower EER than both comparison models across the tested settings.Source: F PAIRED ROBUSTNESS EVALUATION FOR… · p. 22, Table 14 · p. 22
  • At 5 dB, Full SaniBoost minus Without ABFP contrasts in ∆EER are −10.90, −23.06, and −19.54 percentage points on 21LA, 21DF, and ITW, with 95% confidence intervals entirely below zero.The contrasts show consistently smaller EER changes for Full SaniBoost across paired bootstrap resamples.Source: F PAIRED ROBUSTNESS EVALUATION FOR… · p. 22, Table 16 · p. 23

G.1 ADDITIONAL COMPONENT-WISE ABLATION STUDIES

Component ablations support the proposed fusion and aggregation choices, while controlled comparisons find GLAD has the lowest mean EER on all three benchmarks. Additional analyses show purification strength has a non-monotonic effect on generalization.

G.1 ADDITIONAL COMPONENT-WISE ABLATION STUDIES

  • Replacing Linguistic-Acoustic Cross-Attention with additive fusion or attentive Multi-Granularity Fusion with rigid concatenation degrades performance.The authors attribute this to simple addition failing to model cross-modal dynamics and attention-based integration being more informative than feature stacking.Source: §G.1 · p. 23, Table 17 · p. 24
  • Removing Adaptive Stream-wise Gating, Refined Layer-wise Aggregation, or Global-Local Modulation degrades performance.These ablations support the utility of HAG components in filtering representations.Source: §G.1 · p. 23, Table 17 · p. 24
  • Replacing SaniBoost with RawBoost raises 21LA EER from 1.88% to 3.64%, while purifying both classes or disabling ABFP also degrades performance.The results support selective bona fide purification within SaniBoost.Source: §G.1 · p. 24, Table 17 · p. 24
  • One-way XLS-R-to-WavLM attention substantially hurts 21DF and ITW despite a slight 21LA gain; bidirectional CNN–XLS-R attention slightly degrades LA, DF, and ITW.The original CNN–XLS-R design better models the complementary relationship between local signal-level features and high-level semantic representations.Source: §G.1 · p. 24
  • Purification ratio has a non-monotonic effect: increasing it initially improves robustness, but ratios of 0.75–1.0 substantially degrade performance, with ratio 1.0 collapsing on LA.The authors report that these post-finalization analyses were not used to select or tune SaniBoost hyperparameters.Source: Table 18 · p. 24, §G.1 · p. 25

G.2 ADDITIONAL STUDIES UNDER CONTROLLED MODEL CAPACITY

  • GLAD achieves the lowest mean EER on all three benchmarks: 2.70 ± 0.74% on 21LA, 1.53 ± 0.29% on 21DF, and 5.91 ± 0.61% on ITW.Against a parameter-matched 662.24M-parameter MLP, GLAD reduces mean EER by 2.17, 1.21, and 0.72 percentage points on 21LA, 21DF, and ITW, respectively.Source: §G.2 · pp. 25–26, Table 20 · p. 26

Related papers

Loading 2609.35411v1…