Source-linked AI summary
Interpreting SSL Representations for Spoof Detection: a WavLM Study
Mohamed Mallat, Michele Panariello, Massimiliano Todisco, Nicholas Evans, Anthony Larcher
TL;DR
Which WavLM layers encode spoof-relevant information, and what acoustic properties do they represent? This study analyzes WavLM on ASVspoof 5 using probing, CCA, layer-subset search, and visualizations. Mid-to-late layers are most discriminative, while a selected five-layer subset outperforms full-layer pooling and late-layer alignment rises for voice-quality features.
Problem
WavLM’s strong spoof-detection performance leaves unclear which layers encode spoof-relevant information and how those representations relate to interpretable acoustic properties.
Source: §I · p. 1Method
The study analyzes WavLM on ASVspoof 5 with attack-level linear probing, CCA alignment to handcrafted acoustic features, layer-subset search, and embedding visualizations.
Source: §I · p. 1, C. Acoustic cues for spoofing detection · p. 2Results
A five-layer subset achieves 6.1% eval EER versus 8.08% for full 13-layer pooling, while voice-quality feature correlations increase toward late layers.
Source: C. Layer subset search · p. 4, §V · p. 5Takeaways & Limitations
The findings provide an interpretable grounding for SSL-based layer selection, with late layers associating with voice-quality cues and a continuous gradient of vocal naturalness.
Source: §VI · p. 6The study examines only WavLM Base on ASVspoof 5, assumes linear representation-feature relationships in CCA, and uses a non-exhaustive acoustic feature set.
Source: §V · p. 6
Abstract
from arXiv · showhide
Self-supervised speech models such as WavLM achieve strong spoofing detection performance, yet which layers encode the most spoof-relevant information and what acoustic properties they capture remain opaque. We present a layer-wise interpretability study of WavLM using the ASVspoof 5 database, combining three complementary analyses: (i) per-layer linear probing with attack-level evaluation across speech synthesis, voice conversion, and adversarial attacks; (ii) a canonical correlation analysis (CCA) linking layer representations to spoofing-relevant acoustic features; (iii) an exhaustive grid search over layer subsets showing that use of five layers outperforms full thirteen-layer pooling. CCA analysis reveals that, while most acoustic features peak in alignment with mid-layers and then decay, voice quality measures jitter, shimmer, and the harmonics-to-noise ratio show the opposite trend, with correlation increasing toward later layers. Embedding visualisations reveal that deeper layers capture a continuous acoustic similarity among attacks rather than discrete categories, suggesting these layers encode fundamental synthesis artifacts rather than attack-specific signatures.
I. INTRODUCTION
SSL front-ends perform strongly on spoofing benchmarks, but their layer-wise spoof cues and links to interpretable acoustic properties remain unclear. This study examines those questions using ASVspoof 5 and complementary analyses.
I. INTRODUCTION
- Prior work reports that spoofing countermeasures can exploit spurious cues such as non-speech duration, motivating analysis grounded in meaningful acoustic properties.SSL front-ends have also achieved state-of-the-art performance on ASVspoof benchmarks,.Source: §I · p. 1
- ASVspoof 5 provides diverse bona fide and spoofed utterances spanning TTS, voice conversion, and adversarial attacks under varied codecs and recording conditions.The authors identify its crowdsourced data and acoustic diversity as a test bed for interpretability analysis.Source: §I · p. 1
- The study investigates which WavLM layers discriminate bona fide from spoofed speech, how detection varies by attack family, and which interpretable acoustic properties the layers encode.The three attack families are TTS, voice conversion, and adversarial perturbations.Source: §I · p. 1
- The work combines attack-aware layer characterization, compact layer-subset search, and PWCCA-based acoustic grounding, with t-SNE visualizations of representation structure.The authors report that a selected compact subset can outperform all-layer pooling with a linear backend.Source: §I · p. 1
II. RELATED WORK
Spoofing countermeasures use varied back-ends, often paired with SSL front-ends, augmentation, and fusion. The paper’s overview depicts its WavLM-based analysis pipeline.
II. RELATED WORK
- Neural spoofing countermeasures span a broad range of back-ends, including graph-attention architectures.Source: A. Spoofing CMs and SSL front-ends · p. 1
- The analysis overview shows WavLM layer vectors feeding both PWCCA with handcrafted acoustic features and per-layer linear probing.The pipeline summarizes each layer with mean and standard-deviation pooling and produces layer-feature and EER heatmaps.Source: Figure 1 · p. 2
- Recent ASVspoof 5 systems frequently pair SSL front-ends such as wav2vec 2.0 and WavLM with augmentation and score or model fusion,,,,,.Source: A. Spoofing CMs and SSL front-ends · p. 2
B. Layer-wise analysis of SSL models
Prior SSL layer studies characterize depth-dependent representations, while spoofing work has mostly prioritized detection performance; this paper links layers to handcrafted acoustic cues using CCA.
B. Layer-wise analysis of SSL models
- Earlier SSL studies associate early layers with local acoustics and later layers with abstract content and speaker information,.Spoofing studies have largely optimized detection, while Xiao et al. found per-layer decisions outperform feature-level fusion.Source: B. Layer-wise analysis of SSL models · p. 2
C. Acoustic cues for spoofing detection
- Handcrafted spoofing cues include voice-quality measures, formant trajectories, and spectral features, each associated with different synthesis or conversion artifacts [18],,,.These cues are interpretable, though none alone matches modern SSL-based detection performance.Source: C. Acoustic cues for spoofing detection · p. 2
- Whether SSL representations encode the same properties measured by handcrafted features remains unexplored; the study uses CCA to measure their alignment in WavLM layers.Source: C. Acoustic cues for spoofing detection · p. 2
III. METHODOLOGY
The methodology combines per-layer linear probing with CCA to assess WavLM representations against spoofing discriminability and interpretable acoustic features. Experiments use ASVspoof 5 and its standard Track 1 splits.
III. METHODOLOGY
- The analysis uses per-layer linear probing for discriminability and CCA to relate layer representations to handcrafted acoustic features.The two components are presented as complementary analyses of WavLM representations.Source: §III · p. 2
A. Dataset and attack taxonomy
- Experiments use ASVspoof 5 Track 1 with standard train, development, and evaluation splits and group its 32 attacks into TTS, VC, and adversarial families.Adversarial attacks are included as a family in the analysis.Source: A. Dataset and attack taxonomy · p. 2
B. WavLM representations and linear probing
- Frozen WavLM Base representations are extracted from layers L0–L12; its denoising pre-training exposes it to noisy and overlapped speech.The Base model is chosen for computational efficiency and its tractable 13-layer architecture.Source: B. WavLM representations and linear… · p. 3
- Each layer’s 768-dimensional frame representations are mean-and-standard-deviation pooled into a 1536-dimensional utterance vector for a linear bona fide-versus-spoof classifier.Classifiers are trained on training data, selected by development EER, and evaluated on both development and evaluation data.Source: B. WavLM representations and linear… · p. 3
C. CCA-based acoustic feature analysis
The analysis uses PWCCA to relate WavLM layer representations to 17 handcrafted acoustic features, then examines how their alignment varies across layers.
C. CCA-based acoustic feature analysis
- The 17 acoustic features span voice quality, prosody, spectral properties, and harmonic or formant characteristics.They include jitter, shimmer, HNR, F0 and energy measures, spectral descriptors, harmonic ratio, and F1–F3.Source: C. CCA-based acoustic feature analysis · p. 3
- PWCCA summarizes canonical correlations as a projection-weighted mean, giving greater weight to directions with larger projection magnitudes.CCA aligns pooled layer representations with acoustic feature vectors; the analysis averages correlations in both directions.Source: C. CCA-based acoustic feature analysis · p. 3
- Layer representations are reduced to 50 dimensions with PCA before PWCCA because CCA requires more samples than the dimensionality of both inputs.The analysis uses train+dev data, reserves eval for final reporting, and samples up to 5,000 speaker-balanced utterances per attack group.Source: C. CCA-based acoustic feature analysis · p. 3
- Each layer’s 1536-dimensional representation is compared with the 17-feature matrix, and per-feature results use squared correlation with the representation’s first canonical variate.The per-feature correlations are reported in Fig. 3.Source: C. CCA-based acoustic feature analysis · p. 3
IV. RESULTS
Results are reported for development and evaluation sets to examine generalization from familiar attacks to unseen conditions and identify failures.
IV. RESULTS
- The results cover both development and evaluation sets.The two-set analysis is intended to assess whether layer-wise patterns generalize to unseen conditions.Source: §IV · p. 3
- The comparison examines whether layer-wise patterns observed for more familiar attacks generalize to unseen conditions.Source: §IV · p. 3
- The analysis also aims to identify where generalization fails.Source: §IV · p. 3
A. Per-layer probing
Frozen WavLM layer probing shows that detection performance generally improves through L11, while attack-specific patterns include both late-layer gains and persistent adversarial-attack difficulty.
A. Per-layer probing
- A12 unit-selection EER drops from up to ≈80% in early layers to ≈5% in late layers, while eval attacks A22, A24, and A28 exceed ∼30% in later layers.Several dev TTS attacks are near-zero EER from mid-layers onward; most eval attacks follow the general decreasing-EER trend.Source: A. Per-layer probing · p. 3
- 3.33% dev EER and 6.07% eval EER are the lowest per-layer results, both at L11.L12 has higher EERs, indicating a loss of discriminative information; Fig. 2 displays per-attack EER across dev and eval attacks, with overall EER in the rightmost columns.Source: A. Per-layer probing · p. 3, Figure 2 · p. 4
- Malafide- and Malacopula-filtered attacks have EER>20% across all layers, with flat or inverted trends.The text attributes this pattern to adversarial filters optimized to suppress detectable artifacts, and describes the uniform failure as a limitation of artifact-based detection.Source: A. Per-layer probing · pp. 3–4
B. CCA analysis
CCA shows that most acoustic features align most strongly with mid-layers, whereas voice-quality correlations rise toward later layers; these patterns accompany the probing results.
B. CCA analysis
- Most acoustic features peak in correlation around L7 and then decay, with spectral centroid reaching 0.96, MFCC mean 0.92, and spectral roll-off 0.91 at L7.Formant features have weaker peak correlations of 0.70–0.78; prosodic features follow the mid-layer peak pattern with slightly lower peak values.Source: B. CCA analysis · p. 4
- Jitter, shimmer, and HNR correlations increase through L12, reaching 0.89, 0.90, and 0.94, respectively; vocoders and TTS systems are known to distort these properties [18].Source: B. CCA analysis · p. 4
- The authors relate early-to-mid-layer EER gains to stronger spectral and prosodic encoding, and later gains to increasing voice-quality sensitivity despite declining spectral alignment.Voice-quality correlation is greatest at L12, but its EER is 7.3% versus 6.07% at L11; the authors suggest L11 better balances retained spectral information and voice-quality sensitivity.Source: B. CCA analysis · p. 4
C. Layer subset search
Exhaustive subset search found a five-layer late-depth pool that surpassed full-layer pooling. The best subsets concentrate at L6–L12, consistent with the layer-feature alignment patterns.
C. Layer subset search
- The exhaustive search evaluates all 8191 layer combinations, selects subsets on dev, and reserves eval for final reporting.Each subset is represented by concatenated layer vectors and assessed with a linear probe.Source: C. Layer subset search · p. 4
- All top subsets concentrate in L6–L12, while early layers provide no complementary information in linear pooling.The passage characterizes early layers as primarily encoding low-level spectral information with weaker discriminative relevance.Source: C. Layer subset search · p. 4
- Most acoustic features peak in correlation around mid-layers and decay, whereas jitter, shimmer, and HNR correlations increase with depth.Source: Figure 3 · p. 5
- 6.1% eval EER for L7–9–10–11–12 beats the 8.08% full 13-layer baseline.Table I compares the top subsets with all-layer pooling.Source: C. Layer subset search · p. 4, Table I · p. 5
D. t-SNE Visualisation
The visualisations show WavLM representations shifting from overlapping attacks at L1 to attack-specific clusters at L6 and a continuous arc at L11. The discussion connects late layers with voice-quality cues and describes attack-dependent detection patterns and scope limits.
V. DISCUSSION
- Late layers correlate increasingly with jitter, shimmer, and HNR, unlike the decay in spectral and prosodic features.Source: §V · p. 5
- Detection depth varies by attack: A09, A10, A14, and A16 emerge from mid-layers, A12 favors late layers, while A22, A24, and A28 evade late-layer detection despite mid-layer detectability.The discussion suggests a fixed single layer may fail on some attack families and motivates multi-layer or adaptive selection.Source: §V · p. 5
- Malafide and Malacopula evade detection across all layers, highlighting a limitation of artifact-based detection and motivating defenses beyond representation learning.Source: §V · p. 5
D. t-SNE Visualisation
- At L1 attacks overlap, L6 shows attack-specific clusters, and L11 forms a continuous horseshoe arc with bona fide samples at one end.The pattern was consistent across multiple t-SNE seeds and perplexity settings.Source: D. t-SNE Visualisation · p. 5, Figure 4 · p. 6
V. DISCUSSION
- Experiments use WavLM Base and ASVspoof 5; reported trends may not transfer unchanged to other architectures, corpora, real-world recordings, or unseen attacks.The acoustic feature set is also not exhaustive, and CCA assumes linear relationships.Source: §V · p. 6
VI. CONCLUSIONS
The study combines linear probing, CCA-based acoustic grounding, and layer subset search on WavLM using ASVspoof 5. It finds that mid-to-late layers are most discriminative and that late layers organize attacks along a continuous vocal-naturalness gradient.
VI. CONCLUSIONS
- The study analyzes WavLM on ASVspoof 5 using linear probing, CCA-based acoustic grounding, and layer subset search.Source: §VI · p. 6
- Mid-to-late layers (L9–L11) are most discriminative, and a compact five-layer subset outperforms full 13-layer pooling.Source: §VI · p. 6
- Voice-quality features correlate increasingly with late layers, while spectral and prosodic features peak at mid-layers and decay; t-SNE shows late-layer attacks along a continuous vocal-naturalness gradient.Source: §VI · p. 6
- The findings provide an interpretable grounding for layer selection and highlight adversarial attacks designed to evade detection; future work should test trend stability beyond the studied conditions.Source: §VI · p. 6