PaperlayerView exact revision on arXiv

Source-linked AI summary

VietPrism: A large-scale Vietnamese speech and deepfake corpus with diverse dialects and code-switching

Minh Hoang, Thai Le

arXiv:2609.30005v1cs.CLcs.SD

TL;DR

The gap makes it difficult to connect utterances from the same person for speaker-aware training, voice cloning, speaker verification, and emerging spoof-detection research. VIETPRISM combines a large multi-domain Vietnamese speech corpus with dialect and speaker metadata, natural code-switching, and speaker- and transcript-matched spoof speech. Detector performance depends on generator, dialect, and speaker similarity; DFA-1B EER rises from 16.3% to 33.6% across the reported similarity ranges.

  • Problem

    The gap makes it difficult to connect utterances from the same person for speaker-aware training, voice cloning, speaker verification, and emerging spoof-detection research.

    Source: §1 · p. 1

  • Method

    VIETPRISM combines a large multi-domain Vietnamese speech corpus with dialect and speaker metadata, natural code-switching, and speaker- and transcript-matched spoof speech.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Detector performance depends on generator, dialect, and speaker similarity; DFA-1B EER rises from 16.3% to 33.6% across the reported similarity ranges.

    Source: §5 · p. 4, §3 · p. 4

  • Takeaways & Limitations

    VIETPRISM supports evaluation of Vietnamese speech and deepfake detection across diverse linguistic and speaker conditions.

    Source: §4 · p. 4, §5 · p. 4

Abstract

from arXiv · show

Vietnamese speech research is constrained by resources that isolate automatic speech recognition from speaker, dialect, code-switching, and deepfake analysis. We introduce VietPrism, an open, multi-domain corpus that brings these dimensions together at scale: 993.4 hours and 403,941 bona fide utterances from 1,262 verified speakers across 8,388 real-world videos. To our knowledge, it is the first large-scale Vietnamese corpus to jointly provide transcripts, consistent speaker identities, five dialect groups, and naturally occurring Vietnamese--English code-switching, which constitutes nearly half of the corpus by duration. We further create over 3.1K hours of spoof speech with four open-source and commercial synthesis systems. Every spoof is conditioned on a verified speaker reference and paired with a transcript- and speaker-matched bona fide utterance, enabling unique controlled evaluation with reduced lexical and identity confounds. Zero-shot evaluation of five pretrained multilingual detectors reveals striking brittleness: EER greatly varies across detector--generator pairings, while recent multilingual detector DFA-1B degrades from 16.3% to 33.6% as speaker similarity increases. Dialect-stratified results expose further model-dependent disparities. By unifying natural linguistic diversity with controlled spoof generation, VietPrism provides a challenging foundation for Vietnamese speech modeling and trustworthy audio-deepfake detection.

1. INTRODUCTION

Existing Vietnamese speech resources often separate transcripts from speaker identity, while natural Vietnamese–English code-switching and dialect coverage remain limited. VIETPRISM is introduced as an open, multi-domain corpus that combines these dimensions with speaker-linked annotations to support multiple speech tasks.

1. INTRODUCTION

  • Existing resources often omit consistent speaker identifiers or transcripts, making it difficult to connect a person's utterances for speaker-aware tasks.The paper identifies speaker-aware training, voice cloning, speaker verification, and spoof detection as tasks affected by this gap.Source: §1 · p. 1
  • Natural Vietnamese–English code-switching, dialect information, transcripts, and consistent speaker identities are not jointly available in an open, large-scale, multi-domain resource.The cited examples include a 10-hour sociolinguistic corpus and a medical-domain dataset without speaker identities.Source: §1 · p. 2
  • Vietnamese Common Voice statistics report 22.96 hours and 417 contributors, but omit dialect annotations, topic distribution, and an end-to-end quality audit.The paper cautions that effective speaker and domain diversity cannot be inferred from duration alone.Source: §1 · p. 2
  • VIETPRISM contributes 993.4 hours and 403,941 utterances from 1,262 speakers across 8,388 real-world videos.It is presented as an open corpus designed to support multiple speech tasks rather than a single benchmark.Source: §1 · p. 2
  • VIETPRISM jointly provides transcripts, dialect annotations, and consistent speaker identities alongside natural Vietnamese–English speech.Its speaker-linked annotations and natural 8.9-second utterances are described as supporting ASR, speaker verification, dialect and code-switching modeling, voice cloning, and controlled audio-deepfake detection.Source: §1 · p. 2

2. VIETPRISM DATASET

VIETPRISM combines large-scale Vietnamese speech collection with transcript, speaker, and dialect annotation, then adds speaker- and transcript-matched spoof speech. Its statistics highlight coverage across dialects and a uniquely large, controlled spoof subset.

2.1. Dataset Curation Pipeline

  • Collection uses curated public videos, audio enhancement and alignment, transcript quality control, and manual verification of speaker identities before spoof synthesis.Channels are capped at three hours; disagreements exceeding two seconds are manually corrected.Source: §2.1 · pp. 2–3

2.2. Statistical Summary and Uniqueness

  • Table 2 summarizes distributions by dialect, topic, language, and gender; code-switched segments average 2.83 English words, or 8.12% of words.Percentages are duration-based except for gender, which is speaker-based.Source: Table 2 · p. 3
  • The 1,262 verified speakers comprise 568 North, 547 South, 78 Central, 30 Southwest, and 39 Overseas speakers.The North and South groups cover the two largest varieties, while the other groups support evaluation beyond dominant dialects.Source: §2.2 · p. 3
  • Its spoof subset is about nine times larger by utterances and eighteen times larger by duration than the next-largest released Vietnamese subset, with matched pairs from four synthesizers.Each spoof is paired with a bona fide segment matched for transcript and speaker.Source: §2.2 · p. 3, §2.1 · p. 3
  • The corpus provides substantially more speech in North, South, and Central dialects than Vietnam-Celeb, while also including Southwest and Overseas Vietnamese speech.VIETPRISM reports 502.5, 467.3, and 20.0 hours in the three major dialects, respectively; the comparison categories are not directly comparable.Source: §2.2 · p. 3

3. EXPERIMENTS

Zero-shot detector performance varies by synthesizer, speaker similarity, and dialect; code-switching also makes MiniMax spoof speech harder to detect in tested conditions.

Evaluation of Multilingual Deepfake Detectors

  • DFA-1B EER is 10.95–13.14% on three synthesizers but 34.33% on VoxCPM2, while ADF-XLS-R-2B ranges from chance performance on MiniMax to perfect accuracy on OmniVoice and VoxCPM2.ADF-XLS-R-2B has the best mean EER and accuracy, but its results are generator-specific rather than uniform.Source: §3 · pp. 3–4

Similarity-Stratified Evaluation

  • DFA-1B EER rises from 16.3% to 33.6% as speaker similarity increases from [0.5, 0.6) to [0.9, 1.0], while ADF-family detectors are more stable.Fig. 2 reports zero-shot EER by speaker similarity, averaged across synthesizers.Source: §3 · p. 4, Figure 2 · p. 4

Dialect-Stratified Evaluation

  • DFA-1B EER is 28.7% on Southwest and 26.9% on Overseas, versus 16.6% on South and 18.3% on North; dialect effects vary by model.Central performs comparably to larger dialect subsets for several models, while some ADF models remain stable or improve on rarer dialects.Source: §3 · p. 4, Table 5 · p. 4

Performance with Code-Switching

  • On code-switching utterances, MiniMax spoof EER increases by up to 7.1 percentage points on DFA-500M across dominant regional dialects such as South and Southwest.The paper reports this pattern for the tested commercial synthesizer MiniMax.Source: §3 · p. 4, Figure 3 · p. 4

4. OTHER POTENTIAL USE CASES

VietPrism’s annotations support speech and speaker tasks beyond deepfake detection, including modeling, verification, diarization, cloning evaluation, and fairness studies.

4. OTHER POTENTIAL USE CASES

  • VietPrism’s dialect, code-switching, and speaker-matched bona fide–spoof annotations support ASR, language modeling, speaker verification, diarization, accent modeling, voice-cloning evaluation, synthesis attribution, and fairness studies.The matched annotations can potentially help isolate and reduce linguistic sensitivity in detectors trained on utterances without distribution differences.Source: §4 · p. 4

5. CONCLUSION

VietPrism combines a large annotated bona fide corpus with speaker-matched spoof speech, and detector performance varies across generators, dialects, and speaker similarity.

5. CONCLUSION

  • VietPrism contains 993.4 hours and 403,941 bona fide segments from 1,262 speakers, plus 2,765.74 hours of speaker-matched spoof speech from four synthesis systems.Results show detector performance depends on generator, dialect, and speaker similarity, motivating evaluation across diverse conditions.Source: §5 · p. 4
Loading 2609.30005v1…