Source-linked AI summary
AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks
Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hye-jin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, Nicholas Evans
TL;DR
Audio spoofing artefacts can occur in spectral or temporal domains, motivating an efficient single detector for varied attacks rather than computationally demanding score-level ensembles. AASIST models both domains with heterogeneous graph attention, a stack node, max graph operation, and extended readout. It improves the state-of-the-art baseline by over 20% relative in min t-DCF, while AASIST-L uses 85K parameters and outperforms all competing systems except AASIST.
Problem
Spoofing artefacts span spectral and temporal domains, while reliable detection commonly relies on computationally demanding ensembles tuned to particular artefacts.
Method
AASIST is an end-to-end graph neural network using HS-GAL, a stack node, max graph operation, and modified readout to model heterogeneous spectral and temporal information.
Results
Over 20% relative improvement in min t-DCF over the state-of-the-art baseline was achieved, while AASIST-L with 85K parameters outperformed all competing systems except AASIST.
Takeaways & Limitations
AASIST provides a single spoofing-detection system that combines heterogeneous spectral and temporal modeling without score-level ensembles.
Takeaways & Limitations
Reported results are averages and best results from three runs because performance varies significantly with random seed.
Abstract
from arXiv · showhide
Artefacts that differentiate spoofed from bona-fide utterances can reside in spectral or temporal domains. Their reliable detection usually depends upon computationally demanding ensemble systems where each subsystem is tuned to some specific artefacts. We seek to develop an efficient, single system that can detect a broad range of different spoofing attacks without score-level ensembles. We propose a novel heterogeneous stacking graph attention layer which models artefacts spanning heterogeneous temporal and spectral domains with a heterogeneous attention mechanism and a stack node. With a new max graph operation that involves a competitive mechanism and an extended readout scheme, our approach, named AASIST, outperforms the current state-of-the-art by 20% relative. Even a lightweight variant, AASIST-L, with only 85K parameters, outperforms all competing systems.
1. INTRODUCTION
Reliable logical-access spoofing detection must handle artefacts distributed across spectral and temporal domains and attacks generated by varied voice-conversion and text-to-speech techniques. AASIST addresses this with heterogeneous graph modeling, competitive feature selection, and an extended readout in one system.
- Logical-access spoofing detection distinguishes bona-fide speech from attacks mounted with voice conversion and text-to-speech algorithms.
- Spoofing artefacts can occur in both spectral and temporal domains and depend on the attack and algorithm used.
- Adaptive mechanisms are important because artefacts may lie in different domains across spoofing attacks.
- AASIST extends RawGAT-ST with a heterogeneous stacking graph attention layer, max graph operation, and modified readout scheme.HS-GAL models heterogeneous spectral and temporal graphs, while max graph operation uses parallel branches intended to learn different artefact groups.
2. PRELIMINARIES
The system extracts high-level representations from raw waveforms, models spectral and temporal information with graph modules, and uses attention and pooling to process fully connected graphs. Graph attention derives data-driven edge relevance, while pooling retains the most highly weighted nodes.
- RawNet2-based encoder: A RawNet2-based encoder extracts high-level feature maps directly from raw waveform inputs.The encoder output F has channel, spectral-bin, and temporal-sequence dimensions.
- Graph module: Two graph modules model spectral and temporal information as graph representations after encoding.The framework identifies the encoder and graph module as its two shared components.
- Graph attention: Graph attention operates on fully connected graphs because node-pair relevance cannot be predetermined.Self-attention assigns data-driven weights to edges, with element-wise multiplication used before deriving those weights.
- Graph pooling: Attentive graph pooling reduces graph size and improves discrimination by retaining nodes with the top-k attention values.
3. AASIST
AASIST combines spectral and temporal graphs through heterogeneous attention, a stack node, parallel max-aggregation branches, and an extended readout. Its lightweight 85K-parameter variant preserves performance advantages over competing systems except the full AASIST model.
- AASIST framework: AASIST combines spectral and temporal graph modules at the model level using HS-GAL, MGO, and modified readout techniques.The framework extracts features from raw waveforms, processes spectral and temporal graphs in parallel, then applies MGO, readout, and an output layer.
- Graph combination: The combined graph contains Ns + Nt nodes and cross-domain edges linking every temporal node with every spectral node.These edges permit attention between heterogeneous nodes while the constituent graphs retain different latent spaces and dimensions.
- HS-GAL: HS-GAL uses heterogeneous attention and a stack node to accumulate information across spectral and temporal sub-graphs.The two input graphs are projected to a common node dimensionality before heterogeneous attention is applied.
- Max graph operation: MGO processes two parallel branches and applies an element-wise maximum so different branches can learn different spoofing-artefact groups.
- Readout: The readout concatenates node-wise maximum, node-wise average, and the stack node before classification.
- Lightweight variant: 85K parameters make AASIST-L substantially less complex than other compared systems, while its min t-DCF and EER exceed all systems except full AASIST.The variant has a 332KB model size and may be further reduced with half-precision or pruning techniques.
4. EXPERIMENTS AND RESULTS
Experiments evaluate AASIST on ASVspoof 2019 LA using min t-DCF and EER, including comparisons with baselines, state-of-the-art systems, a lightweight variant, and component ablations.
- Experimental setup: ASVspoof 2019 LA contains train and development attacks from six algorithms and evaluation attacks from thirteen algorithms.The experiments use train, development, and evaluation subsets.
- Evaluation metrics: Min t-DCF measures impact on automatic speaker verification, whereas EER measures standalone spoofing detection performance.
- Comparison with RawGAT-ST: AASIST outperforms RawGAT-ST by over 20% relative in pooled min t-DCF, decreasing it from 0.0443 to 0.0347.With the best seed, AASIST reaches 0.83% EER and 0.0275 min t-DCF.
- Comparison with state-of-the-art systems: AASIST is the best-performing system among the compared state-of-the-art models on ASVspoof 2019 LA.The comparison spans varied front-end representations and model architectures.
- Lightweight variant: AASIST-L uses 85K parameters and achieves better min t-DCF and EER than every compared system except full AASIST.The authors suggest that further size reductions could support embedded-system use.
- Ablations: Removing any of the three proposed techniques worsens performance, with heterogeneous attention having the largest ablation impact.The max graph operation also shows a consistent beneficial effect.
5. CONCLUSION
AASIST is an end-to-end graph-neural-network spoofing detector that integrates heterogeneous spectral and temporal processing with competitive artefact selection and modified readout. It improves over the state-of-the-art baseline by over 20% relative in min t-DCF, while AASIST-L uses 85K parameters and outperforms competing systems.
- Conclusion: AASIST combines heterogeneous spectral and temporal graph modeling through HS-GAL, which uses heterogeneous attention and a stack node.
- Conclusion: AASIST introduces MGO for competitive artefact selection and a modified readout scheme.
- Conclusion: AASIST improves over the state-of-the-art baseline by over 20% relative in min t-DCF.
- Conclusion: AASIST-L uses 85K parameters and outperforms all competing systems.