PaperlayerView exact revision on arXiv

Source-linked AI summary

DuRe-ST: Dual-Relation Spectro-Temporal Modeling for Speech Deepfake Detection

Shaole Li, Siqing Qin, Youzhi Tu, Kong Aik Lee

arXiv:2610.10121v1eess.AScs.SD

TL;DR

Speech deepfake detectors can model adaptive spectro-temporal dependencies with graph attention, but largely overlook covariance between spectral and temporal representations. DuRe-ST combines covariance-induced relations with graph-attention relations; it reduces EER against XLSR-AASIST on ASVspoof benchmarks and outperforms the strongest public comparison models on four benchmarks.

  • Problem

    Existing speech deepfake detectors using graph attention largely overlook covariance, the second-order co-variation between spectral and temporal representations.

    Source: §1 · p. 1

  • Method

    DuRe-ST combines covariance-induced relations, modeled through polynomial filtering, with graph-attention-induced relations.

    Source: §1 · p. 1, §2.3 · p. 2

  • Results

    DuRe-ST achieves a 25.9% average relative EER reduction over the baseline on ASVspoof benchmarks and outperforms the strongest public models in the comparison by 2.2–13.2% in relative EER.

    Source: §5 · p. 4

  • Takeaways & Limitations

    The results support the complementary role of covariance-induced spoofing cues in improving cross-dataset generalization through joint modeling with attention-induced relations.

    Source: §5 · p. 4

    Cov-ST’s EER rises to 7.79% at K = 3 on 2021 DF, and DuRe-ST’s 2021 LA EER degrades to 2.17% at K = 3.

    Source: §4.2 · p. 3

Abstract

from arXiv · show

Previous speech deepfake detectors can adaptively capture spectro-temporal dependencies through graph attention, yet they largely overlook the co-variation between spectral and temporal representations. To address this gap, we construct a normalized affinity graph from their joint covariance and apply polynomial graph filtering to capture higher-order covariance-induced dependencies. We first develop Cov-ST to isolate the contribution of covariance-based relational modeling. Although it improves detection performance, its sensitivity to the polynomial order suggests limited robustness when covariance relations are modeled alone. We therefore propose DuRe-ST, which jointly exploits covariance-induced and graph-attention-induced relations to capture complementary second-order co-variation and adaptive spectro-temporal dependencies. Experiments show that DuRe-ST achieves an average relative EER reduction of 25.9% over XLSR-AASIST on the ASVspoof benchmarks and 28.4% across four cross-dataset benchmarks with only 4-8k additional trainable back-end parameters. It further outperforms the strongest publicly available comparison models by 2.2-13.2% in relative EER on four benchmarks, while remaining smaller than the publicly available models considered.

1. INTRODUCTION

Recent speech deepfake detectors model spectral and temporal information through approaches including AASIST’s graph attention, but do not explicitly represent their covariance. DuRe-ST adds covariance-induced relations and combines them with graph-attention relations to capture complementary spectro-temporal dependencies.

1. INTRODUCTION

  • AASIST models spectro-temporal relations mainly through learned attention weights, which do not explicitly represent covariance between spectral and temporal features.The passage connects potentially unnatural or over-smoothed time–frequency structures in synthetic speech with artifact traces that may complement attention-induced relations.Source: §1 · p. 1
  • DuRe-ST combines covariance-induced and graph-attention-induced relations to model both second-order co-variation and adaptive dependencies between spectral and temporal representations.Cov-ST first isolates covariance-based modeling, but its sensitivity to polynomial order suggests limited robustness when covariance relations are used alone.Source: §1 · p. 1
  • The paper introduces covariance-induced spectro-temporal relation modeling to explicitly capture second-order dependencies between spectral and temporal representations.Its formulation constructs a normalized affinity graph from joint covariance and applies polynomial graph filtering to capture multi-hop dependencies.Source: §1 · p. 1
  • Experiments report that DuRe-ST captures spoofing cues complementary to attention-induced relations and achieves the best results among compared systems across multiple benchmarks.The introduction characterizes covariance-induced spoofing cues as complementary to attention-induced relations.Source: §1 · p. 1

2. PROPOSED DURE-ST ARCHITECTURE

DuRe-ST uses shared spectral and temporal nodes to model attention-induced and covariance-induced relations. The resulting representations are combined for classification, while Cov-ST uses only the covariance-induced representation.

2.1. Shared Spectro-Temporal Nodes

  • XLS-R 300M features are projected and CNN-encoded, then frequency and temporal weighting and top-k pooling yield shared spectral and temporal node sets.Source: §2.1 · pp. 1–2

2.2. Attention-Induced Relation Modeling

  • The attention branch uses a master node and two heterogeneous graph-attention layers, then combines pooled domain representations with the master-node representation.Source: §2.2 · p. 2

2.3. Covariance-Induced Relation Modeling

  • Joint covariance provides within-domain and cross-domain co-variation; its absolute values, self-connections, and symmetric degree normalization form the affinity graph.Source: §2.3 · p. 2
  • An order-K polynomial filter transforms nodes over the normalized graph, with higher-order terms aggregating covariance-induced dependencies across up to K propagation steps.Source: §2.3 · p. 2
  • The filtered nodes are split by domain, summarized with maximum-magnitude and mean pooling, concatenated, and projected by a two-layer MLP into hcov.Source: §2.3 · p. 2

2.4. Model Variants and Classification

  • Cov-ST classifies from hcov alone, whereas DuRe-ST concatenates hcov and the attention-induced representation hatt before classification.Source: §2.4 · p. 3

3. EXPERIMENTAL SETUP

Models are trained on ASVspoof 2019 LA and evaluated on ASVspoof and four cross-dataset benchmarks without target-domain adaptation. The setup reports EER and details training and augmentation procedures.

3.1. Datasets and Metric

  • Training uses ASVspoof 2019 LA; evaluation uses ASVspoof 2021 LA and DF, In-the-Wild, Fake-or-Real, ASVspoof 5, and DFADD without target-domain adaptation.Performance is reported as EER (%), with lower values indicating better detection.Source: §3.1 · p. 3

3.2. Implementation Details

  • Utterances are cropped or concatenated to about 4 seconds; weighted cross-entropy, Adam, early stopping, and averaging five validation-selected checkpoints are used.Training uses class weights of 0.9 and 0.1 for bona fide and spoofed speech, respectively.Source: §3.2 · p. 3
  • Separate RawBoost Algo 3 and Algo 5 checkpoints are trained; Algo 3 serves ASVspoof 2021 DF, Algo 5 the other datasets, and both are evaluated on DFADD.The lower DFADD EER is reported.Source: §3.2 · p. 3

4. RESULTS AND ANALYSIS

DuRe-ST’s strongest ASVspoof results come from jointly modeling covariance- and attention-induced relations, while covariance-only performance is sensitive to polynomial order and evaluation domain. Cross-dataset results and embedding visualizations further compare these models with the baseline and public systems.

4.1. Main Results

  • With K = 1, DuRe-ST achieves EERs of 0.93% on 2021 LA and 2.04% on 2021 DF, reducing baseline EER by 7.0% and 44.7%, respectively, and 25.9% on average.This improvement requires 4k additional trainable back-end parameters.Source: Table 1 · p. 3, §4.1 · p. 3
  • At K = 2, Cov-ST achieves 1.15% EER on 2021 LA and reduces 2021 DF EER from 3.69% to 2.66%, a 27.9% relative reduction.Cov-ST uses fewer trainable back-end parameters than the XLSR-AASIST baseline.Source: Table 1 · p. 3, §4.1 · p. 3

4.2. Sensitivity to the Polynomial Order

  • At K = 0, the transformation without covariance information lowers 2021 DF EER from 3.69% to 2.23% but raises 2021 LA EER from 1.00% to 2.01%.K = 1 performs best on both benchmarks, while K = 2 yields 1.19% EER on 2021 LA and 2.71% on 2021 DF.Source: §4.2 · p. 3
  • At K = 3, Cov-ST’s 2021 DF EER rises to 7.79%, while DuRe-ST’s 2021 LA EER degrades to 2.17%, so gains do not increase monotonically with K.The authors retain K = 1 and K = 2 for cross-dataset evaluation because higher-order filtering may propagate redundant or weak relations.Source: §4.2 · p. 3

4.4. Visualization of Embedding Distributions

  • On DFADD, both DuRe-ST variants show clearer bona fide–spoof separation than XLSR-AASIST; K = 1 has the least overlap, consistent with its 8.36% EER.The K = 2 variant’s greater local overlap corresponds to 10.45% EER; the t-SNE evidence is qualitative.Source: Figure 2 · p. 4, §4.4 · p. 4

4.3. Cross-Dataset Comparison with Public SDD Models

  • With K = 2, DuRe-ST beats the baseline on all four datasets and improves on the strongest public results by 2.2%, 13.2%, and 3.9% in relative EER on three datasets.Those comparisons are on In-the-Wild, Fake-or-Real, and ASVspoof 5.Source: §4.3 · pp. 3–4, Table 2 · p. 4
  • On DFADD, DuRe-ST with K = 1 achieves 8.36% EER, a 44.6% relative reduction over the baseline and 7.3% over the best public result.DuRe-ST uses 4–8k additional trainable back-end parameters and has 0.451–0.455M parameters, below the public models’ 0.512M–23.399M range.Source: Table 2 · p. 4, §4.3 · p. 4

5. CONCLUSION

The paper introduces Cov-ST and DuRe-ST for covariance-induced spectro-temporal relation modeling, with DuRe-ST combining those relations with attention-induced relations. Reported results support complementary covariance cues for cross-dataset generalization, while covariance-only modeling is sensitive to polynomial order and evaluation domain.

  • Cov-ST demonstrates that second-order dependencies provide discriminative spoofing cues, but covariance-only modeling is sensitive to polynomial order and evaluation domain.Source: §5 · p. 4
  • DuRe-ST reduces average relative EER by 25.9% over the baseline on ASVspoof benchmarks and outperforms strongest public models by 2.2–13.2% in relative EER.It has 0.451–0.455M trainable back-end parameters.Source: §5 · p. 4
  • The results support a complementary role for covariance-induced spoofing cues when jointly modeled with attention-induced relations for cross-dataset generalization.Source: §5 · p. 4
  • Future work will explore input-adaptive order weighting to reduce order sensitivity.Source: §5 · p. 4
Loading 2610.10121v1…