Source-linked AI summary
Comparing Self-Supervised and Domain-Invariant Features for Cross-Domain Voice Phishing Detection
Jeongmin Lee, Seung Yun, Minkyu Lee, Ran Han, Yoonkyu Woo, Jinxia Huang
TL;DR
Privacy constraints and scarce authentic criminal recordings make cross-domain voice-phishing detection difficult to train and evaluate. The paper compares domain-invariant prosodic features with HuBERT and wav2vec2.0, training on scenario-based recordings and testing on authentic calls. Domain-invariant features lead without target samples, while HuBERT performs best with five-shot support.
Problem
Authentic criminal recordings are difficult to access and scarce, while the cross-domain effectiveness of SSL models versus domain-invariant prosodic features has not been systematically evaluated across data-availability settings.
Source: §1 · p. 1Method
The study compares domain-invariant prosodic features with HuBERT and wav2vec2.0 on scenario-based training and authentic-call testing across zero-shot and few-shot settings.
Source: §1 · p. 1, §3.4 · p. 3Results
Domain-invariant features achieve 69.5% F1 zero-shot, while HuBERT reaches 94.2% F1 with 5-shot support.
Source: §4 · p. 3Takeaways & Limitations
Domain-invariant features suit cold-start deployment, while SSL methods offer higher performance when target-domain samples and computational resources are available.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Voice phishing detection faces three critical challenges: real criminal recordings are unavailable due to privacy constraints; when available, only a handful of samples exist, insufficient for fine-tuning; and lightweight acoustic-only detection is needed as an alternative to large self-supervised models. We compare domain-invariant prosodic features and self-supervised representations (HuBERT, wav2vec2.0) through cross-domain evaluation-training on scenario-based actor recordings and testing on authentic criminal calls. Domain-invariant prosodic features achieve 69.5% F1 zero-shot and 71.0% with 5-shot learning. HuBERT achieves highest performance (94.2% F1, 5-shot), while wav2vec2.0 exhibits a precision-oriented detection profile (90.2% F1 with 99.4% precision, 5-shot). These findings reveal fundamental trade-offs: domain-invariant features enable zero-shot deployment when no real data exists, while SSL methods achieve higher performance but require real samples and compute.
1. Introduction
Voice phishing detection must generalize to authentic criminal calls despite privacy-constrained access to real recordings and very limited training samples. The paper compares domain-invariant prosodic features with self-supervised models across cross-domain and data-availability settings.
1. Introduction
- Privacy and security concerns constrain raw-audio transmission, making authentic criminal recordings difficult to access and leaving few samples for training.The paper frames cross-domain generalization from scenario-based data to authentic calls as necessary under these constraints.Source: §1 · p. 1
- Whether HuBERT and wav2vec2.0 generalize from controlled actor recordings to authentic criminal calls, and how they compare with domain-invariant prosody, remains systematically untested across data-availability settings.The comparison also considers performance and computational requirements.Source: §1 · p. 1
- HuBERT achieves 94.2% F1 with 5-shot learning, while domain-invariant prosodic features achieve 69.5% F1 zero-shot without authentic criminal samples.The paper presents these as distinct performance and data-availability trade-offs.Source: §1 · p. 1
- The study compares prosodic features and SSL models on authentic criminal recordings and examines accuracy, computational cost, and sample efficiency to inform deployment choices.The stated guidance favors domain-invariant features for resource-constrained or cold-start settings and SSL when target samples and computational resources are available.Source: §1 · p. 1
2. Related Work
Prior work motivates acoustic-only detection as a resource-conscious alternative, but leaves cross-domain voice-phishing performance of SSL models and prosodic features insufficiently established. The paper positions its comparison against work in deception detection and domain adaptation.
2. Related Work
- Text-and-audio systems and transcript-based language-model approaches have been explored for voice phishing, but require substantial resources for text extraction and model deployment.This paper instead focuses on acoustic-only detection and its deployability and generalization.Source: §2 · p. 1
- SSL models perform strongly on several speech tasks, but their generalization from scenario-based actor recordings to authentic criminal calls in voice-phishing detection remains unexplored.Prior cross-domain SSL robustness cited here concerns ASR, not voice phishing.Source: §2 · p. 1
- Prior prosodic deception studies identify useful acoustic cues but validate them within single controlled domains, while authentic criminal training recordings are often unavailable.The cited work includes F0, energy, and duration patterns, alongside cues associated with deception and perceived trustworthiness.Source: §2 · pp. 1–2
- Existing adaptation approaches use target-domain data for fine-tuning or adversarial training, whereas this study selects prosodic features with Cohen’s d < 0.5 for domain stability.Feature selection is separated from classifier training, and the approach is evaluated with 0, 1, or 5 target-domain samples.Source: §2 · p. 2
3. Methods
The methods construct a Korean simulated-to-authentic voice-phishing evaluation, compare domain-invariant prosodic and frozen SSL representations, and assess logistic-regression transfer with varying authentic-sample support.
3.1. Dataset Construction
- The Korean corpus spans VP and non-VP classes in scenario-based and authentic domains, uses 8 kHz mono audio, and applies speaker-independent splits.It includes 406 scripted actor VP utterances and 456 authentic criminal VP utterances, with a separate holdout for few-shot support.Source: §3.1 · p. 2
3.2. Feature Extraction
- The prosodic pipeline ranks 88 eGeMAPS functionals by Random Forest importance, then retains features with Cohen’s d < 0.5 across scenario-based and authentic VP domains.Four features remain: logRelF0-H1-A3, mfcc1V, mfcc4, and F2bandwidth; Figure 1 plots importance against Cohen’s d and annotates the selected features.Source: §3.2 · p. 2, Figure 1 · p. 2
- HuBERT-Base and wav2vec2.0-Base representations are mean-pooled into 768-dimensional utterance embeddings, with encoder weights frozen and no domain-specific fine-tuning.Both models have 12 transformer layers and approximately 94M parameters.Source: §3.2 · p. 3
3.3. Classification Setup
- Logistic Regression with L2 regularization is used across feature types, with feature standardization based only on training-data statistics.The classifier was chosen for few-shot stability, computational efficiency, and a calibrated linear decision boundary on normalized features.Source: §3.3 · p. 3
- Few-shot conditions append 0, 1, or 5 authentic VP holdout samples to the scenario-based training set while keeping the test set fixed at 812 utterances.The holdout and test sets are strictly disjoint.Source: §3.3 · p. 3, §3.4 · p. 3
3.4. Evaluation Protocol
- Evaluation tests transfer from scenario-based training to authentic real-world data and reports F1, VP Recall, and Precision.Here, zero-shot means classifier transfer without authentic VP samples in classifier training; the preliminary feature-stability analysis is separate from classifier training.Source: §3.4 · p. 3
4. Results and Discussion
Domain-invariant prosodic features lead without authentic samples, while HuBERT achieves the strongest 5-shot F1; the unfiltered 88-feature set collapses at low-shot settings. The methods also differ in precision–recall balance, and feature ablations show distinct detection and false-alarm trade-offs.
4. Results and Discussion
- The unfiltered 88-feature set scored 3.8% and 8.8% F1 at zero- and one-shot, then 85.2% at five-shot, surpassing 4f.Figure 2 identifies this five-shot change as Crossover 2.Source: §4 · p. 3, Figure 2 · p. 3
- At 5-shot, HuBERT reaches 99.3% recall and 89.6% precision, while wav2vec2.0 reaches 82.5% recall and 99.4% precision.The paper associates HuBERT's profile with detection completeness and wav2vec2.0's with fewer false alarms.Source: §4 · p. 3
- 69.5% F1 gives 4f the zero-shot lead (HuBERT 58.3%, wav2vec2.0 36.2%), but 5-shot F1 reaches 94.2% for HuBERT, 90.2% for wav2vec2.0, and 71.0% for 4f.Figure 2 marks the transition where HuBERT surpasses 4f as target-domain samples increase.Source: Figure 2 · p. 3, §4 · p. 3
- mfcc1V has the highest F1 and sensitivity across shot conditions, while 4f improves specificity by 3.2 pp at 1-shot and 2.7 pp at 5-shot.The ablation compares mfcc1V with the selected four-feature set; Table 3 distinguishes combined-set specificity from OOD-only specificity.Source: §4 · p. 4, Table 3 · p. 4
- On the authentic Non-VP subset, 4f specificity exceeds mfcc1V at zero-, one-, and five-shot, unlike the ambiguous zero-shot result on the mixed test set.The OOD-only subset contains 203 authentic telephone consultations.Source: §4 · p. 4
- logRelF0-H1-A3 has d=0.497 but only 18.8% individual F1, showing that high domain-discriminativeness does not imply detection utility.The four-feature set was fixed before experiments using the d < 0.5 criterion, rather than reselected from the ablation.Source: §4 · p. 4
5. Conclusion
Cross-domain results favor filtered prosodic features at zero-shot and SSL models with 5-shot adaptation, while deployment suitability depends on compute and hardware constraints.
5. Conclusion
- 69.5% F1 for domain-invariant prosodic features outperforms both SSL models at zero-shot, while 5-shot HuBERT reaches 94.2% F1 and wav2vec2.0 achieves 90.2% F1.HuBERT has near-perfect recall; wav2vec2.0 has near-perfect precision.Source: §5 · p. 4
- 3.8% F1 for the unfiltered 88-feature set makes Cohen’s d-based filtering a prerequisite for cold-start deployment.Source: §5 · p. 4
- Ablation identifies mfcc1V as the primary detection driver, showing domain stability alone does not imply discriminative utility.Source: §5 · p. 4
- Whether this regime structure extends beyond Korean remains an open question for cross-lingual validation.Source: §5 · p. 4
- Domain-invariant prosodic features need no pretrained model and run on CPU, whereas ∼94M-parameter SSL models require GPU but deliver higher accuracy with minimal labeled data.The choice depends on hardware constraints.Source: §5 · p. 4
7. Generative AI Use Disclosure
Generative AI tools assisted with code development and manuscript language editing, but not the research or scientific conclusions.
7. Generative AI Use Disclosure
- Claude was used to assist with code development and manuscript language editing.Source: §7 · p. 5
- AI tools were not used for research design, hypothesis generation, experimental methodology, data analysis, result interpretation, or scientific conclusions.Source: §7 · p. 5
- Authors reviewed, tested, and validated AI-generated code, and wrote and verified all scientific content.AI-assisted manuscript revisions were limited to language polishing and clarity improvements.Source: §7 · p. 5