PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploiting Acoustic and Content-Oriented Speaker Verification Attacks Against Multilingual Voice Anonymization

Ridwan Arefeen, Ze Li, Rong Tong, Ming Li, Xiaoxiao Miao

arXiv:2610.08107v1cs.SDcs.AI

TL;DR

Attacker ASV systems for voice anonymization have been studied mainly in English, leaving multilingual behavior underexplored. This paper evaluates acoustic- and content-oriented attackers and constructs multilingual voice-converted training data. It finds that attacker performance depends on speech utility: acoustic-oriented models generally perform better, but the gap narrows when linguistic information is preserved, and multilingual training partly reduces the cross-lingual gap.

  • Problem

    Attacker ASV for anonymized speech has been studied primarily in English, leaving its behavior in multilingual settings, including cross-lingual generalization, underexplored.

    Source: §I · p. 1

  • Method

    The study compares acoustic-oriented WavLM-ECAPA and content-oriented w2v-BERT-MFA attackers on multilingual anonymized speech and fine-tunes them with a corpus generated by seven voice-conversion systems.

    Source: §I · pp. 1–2, D. Attacker Architectures · p. 3

  • Results

    Acoustic-oriented attackers generally perform better, but content-oriented performance approaches theirs when linguistic information is preserved; multilingual training partly narrows the cross-lingual gap.

    Source: §I · p. 2, 2) Results on Semi-informed Attackers · p. 6

  • Takeaways & Limitations

    Attacker evaluation should consider both privacy and utility rather than rely on a single attacker strategy.

    Source: §I · p. 2

Abstract

from arXiv · show

Attacker ASV systems for voice anonymization have been studied primarily in English, leaving their behavior in multilingual settings largely unexplored. Conventional ASV has shown that both acoustic and contextual information are important for multilingual speaker verification. Inspired by this, we investigate whether the same holds for attacker ASV on anonymized speech. We evaluate both acoustic- and content-oriented attackers on multilingual anonymized speech and construct a multilingual voice-converted dataset to improve cross-lingual generalization. Our results show that attacker effectiveness depends on the linguistic utility of the anonymized speech. Overall, acoustic-oriented attackers achieve better performance. However, when linguistic information is well preserved, the performance gap between content- and acoustic-oriented attackers narrows compared with conditions involving stronger speech distortion. The multilingual voice-converted dataset further improves performance and partially reduces the cross-lingual gap. These findings highlight the need for more comprehensive attacker modeling and evaluation protocols that consider both privacy and utility, rather than relying on a attacker strategy\footnote{Full code and pretrained models and MultiVC Dataset link are available at: https://github.com/monkeyDarefeen/DAST

I. INTRODUCTION

Attacker ASV for anonymized speech has been assessed mainly in English, leaving multilingual behavior—especially across structurally distant languages—underexplored. The study compares acoustic- and content-oriented attackers and finds that effectiveness tracks speech utility, while multilingual voice-converted training partly narrows the cross-lingual gap.

I. INTRODUCTION

  • Most attacker ASV systems were developed and assessed on English; the study adds Japanese and Chinese to probe generalization beyond Indo-European languages.VPC 2026 evaluates English, German, French, and Spanish with a shared attacker, while the authors identify structurally distant languages as an open question.Source: §I · p. 1
  • The study compares WavLM–ECAPA acoustic-oriented attackers [5] with w2v-BERT-MFA content-oriented attackers under multilingual anonymization.The paper labels the models by whether their ASV architecture contains an explicit BERT content component.Source: §I · p. 1
  • Both attackers re-identify speakers reliably in European languages but degrade sharply in East Asian languages, with the decline tracking rising ASR WER.The authors describe this as a coupling between linguistic intelligibility and identity disclosure.Source: §I · p. 2
  • Acoustic-oriented attackers are generally more robust, while multilingual voice-converted training partly closes the cross-lingual gap.The authors construct a corpus from seven voice-conversion systems and note that a residual gap remains tied to intelligibility.Source: §I · p. 2
  • The authors call for attacker modeling and evaluation that consider both privacy and utility rather than relying on a single attacker strategy.Their motivation is that anonymization should protect identity without destroying linguistic intelligibility or other utility cues,.Source: §I · pp. 1–2

II. ATTACKER ASV MODELING AND ACOUSTIC-LINGUISTIC VULNERABILITY ANALYSIS

The evaluation measures re-identification and speech utility across six languages, two anonymization baselines, and two attacker models. It also uses multilingual voice-converted data to examine whether data scarcity contributes to cross-lingual performance differences.

II. ATTACKER ASV MODELING AND ACOUSTIC-LINGUISTIC VULNERABILITY ANALYSIS

  • The pipeline anonymizes multilingual speech with BM1 or BM2, measures attacker EER, and diagnoses retained intelligibility with ASR WER/CER.The analysis relates cross-lingual EER gaps to utility degradation and uses MultiVC data to probe the data-scarcity contribution.Source: §II · p. 2

A. Training and Evaluation Data

  • The evaluation covers English, German, French, Spanish, Chinese, and Japanese using VPC 2026 Track 2 training, development, and test sets.Only test-set results are reported in the paper.Source: A. Training and Evaluation Data · p. 2, §II · p. 2
  • Japanese and Mandarin Chinese receive auxiliary training and evaluation datasets drawn from Common Voice and JVS, and from AISHELL-3, respectively.The Japanese sets include 1,545 training speakers and 48 evaluation speakers; AISHELL-3 provides 340 training speakers and 30 evaluation speakers.Source: A. Training and Evaluation Data · p. 2

B. Anonymization Baselines

The study uses BM1 and BM2 as multilingual anonymization baselines and evaluates lazy-informed and semi-informed attackers. BM3 is not evaluated separately because its architecture resembles BM2 and omits specified prosody and pitch/energy operations.

B. Anonymization Baselines

  • BM1 replaces speaker embeddings with a pseudo-speaker and synthesizes speech from pitch and content representations; BM2 uses Whisper transcriptions, perturbed prosody features, and multilingual TTS.BM1 uses HuBERT representations and HiFi-GAN, while BM2 uses an artificial speaker embedding and IMS Toucan synthesis.Source: B. Anonymization Baselines · p. 2

C. Threat Models

  • Both threat models assume access to the anonymization system, multiple anonymized enrollment utterances, and language-matched public speech data.Enrollment utterances are anonymized to match trial speech.Source: C. Threat Models · p. 2
  • BM3 is not investigated separately because it follows BM2’s architecture but lacks prosody extraction and F0/energy modification.The authors report that VPC 2026 also includes BM3 in Track 2.Source: C. Threat Models · p. 2
  • Lazy-informed attackers are not adapted to the evaluation anonymization data, whereas semi-informed attackers are adapted using speech generated by that same system.The semi-informed setting reflects upper-bound attack strength; lazy-informed EER assesses generalization across anonymization approaches.Source: C. Threat Models · pp. 2–3

D. Attacker Architectures

The paper contrasts acoustic- and content-oriented attacker architectures, then relates multilingual re-identification performance to anonymized-speech utility. Across languages, identity recovery and ASR utility vary together, with substantial utility degradation for Chinese and Japanese under BM2.

D. Attacker Architectures

  • WavLM-ECAPA processes waveform and Mel-filterbank features through parallel ECAPA-TDNN encoders, while w2v-BERT-MFA aggregates intermediate representations from a Conformer model.The latter uses localized layer adapters and a Multi-scale Feature Aggregation module.Source: D. Attacker Architectures · p. 3

E. Diagnostic Vulnerability Analysis

  • Both attackers’ re-identification degrades sharply for Chinese and Japanese, alongside language-specific ASR utility differences.Figure 2 compares average lazy-informed EER across six languages and overlays ASR WER/CER; the authors characterize the relationship as coupling, not demonstrated causation.Source: E. Diagnostic Vulnerability Analysis · p. 3, Figure 2 · p. 3
  • Under BM2, Japanese reaches a CER of 45.1% and Chinese a CER of 70.3%; BM2 has higher error metrics than BM1 across all languages.For French, BM2 WER is 44.9% versus 13.2% for BM1; the authors associate BM2’s greater distortion with explicit prosodic and F0 extraction.Source: E. Diagnostic Vulnerability Analysis · p. 3

III. MULTILINGUAL VOICE-CONVERTED DATASET GENERATION

This section introduces a multilingual voice-converted corpus to investigate whether data scarcity limits cross-lingual performance, and presents its generation pipeline.

  • The corpus is constructed to evaluate whether reducing data scarcity can improve cross-lingual performance.Source: §III · p. 3
  • Table I reports ASR utility metrics for BM1 and BM2, using CER for Japanese and Chinese and WER for other languages.Source: Table I · p. 4
  • Fig. 3 presents the overall pipeline for generating the voice-converted dataset.Source: Figure 3 · p. 4, §III · p. 4

A. Source Corpus

The section describes the multilingual source and target corpora and a speaker-stratified partition of the target corpus into balanced folds.

A. Source Corpus

  • The source pool comprises utterances across the targeted languages and serves as content references for the voice-conversion pipeline.Source: A. Source Corpus · p. 4

B. Target Corpus

C. Split Strategy

  • Ten balanced folds contain exactly 34,022 utterances each and include all 4,474 speakers, with surplus utterances duplicated across folds.Utterances are assigned to folds cyclically within each speaker; duplication distributes counts not divisible by ten.Source: C. Split Strategy · p. 4

D. Voice Conversion Systems

Seven voice-conversion systems are assigned target splits to create converted samples, using a pairing strategy that covers the multilingual source pool.

D. Voice Conversion Systems

  • Seven voice-conversion architectures receive distinct target splits, producing 714,462 converted samples in total while reserving splits 8–10 as controls.Source: D. Voice Conversion Systems · p. 4

E. Pair Generation

  • Each system pairs 34,022 target style references with three source utterances, yielding 102,066 conversion utterances through two-phase source allocation.The allocation first uses all 70,113 source utterances once, then fills the remaining slots by random sampling with replacement; average source use ranges from 1.35× to 1.46×, capped at six uses.Source: E. Pair Generation · pp. 4–5

IV. EXPERIMENT

The experiments evaluate lazy-informed and semi-informed attackers with two ASV architectures, using monolingual or multilingual training and target-system fine-tuning.

IV. EXPERIMENT

A. Training Setup for the Attacker Models

B. Experimental Results

Multilingual training lowers attacker EER in both anonymization settings, but its gains are smaller under BM2. WavLM-ECAPA outperforms w2v-BERT-MFA across both settings.

1) Results on Lazy-informed Attackers:

  • BM1 average EER falls from 25.86% to 17.66% for w2v-BERT-MFA and from 15.77% to 8.99% for WavLM-ECAPA with multilingual training.These are the reported average EERs for ID1–ID4.Source: 1) Results on Lazy-informed Attackers · p. 5
  • BM2 average EER falls from 39.61% to 38.97% for w2v-BERT-MFA and from 29.76% to 24.83% for WavLM-ECAPA with multilingual training.These are the reported average EERs for ID5–ID8.Source: 1) Results on Lazy-informed Attackers · p. 5
  • Multilingual-training gains are substantially smaller on BM2 than BM1, which the authors associate with stronger prosodic distortion and partial semantic degradation from its ASR–TTS generation.The proposed explanation is that this distortion makes voice-conversion training data less effective at preserving or recovering speaker-discriminative cues.Source: 1) Results on Lazy-informed Attackers · pp. 5–6
  • WavLM-ECAPA consistently outperforms w2v-BERT-MFA across BM1 and BM2, indicating greater robustness for acoustic-oriented representations under anonymization.The authors relate the content-oriented model's lower performance to its greater reliance on semantic consistency.Source: 1) Results on Lazy-informed Attackers · p. 6

2) Results on Semi-informed Attackers:

In semi-informed evaluation, the relative effectiveness of attacker types depends on anonymized-speech utility. Acoustic-oriented attackers perform better overall, while content-oriented performance approaches theirs when linguistic information is preserved.

2) Results on Semi-informed Attackers:

V. CONCLUSION

The conclusion links re-identification vulnerability to speech utility and reports that MultiVC mitigates data scarcity while only partially narrowing the cross-lingual EER gap. It calls for attacker modeling and evaluation that weigh privacy and utility.

V. CONCLUSION

  • The study relates cross-lingual EER gaps to utility degradation and reports content-oriented models as competitive when semantic structure is preserved, but less robust under high-WER distortion.Source: §V · p. 6
  • Fine-tuning on MultiVC mitigates data scarcity and partially narrows the cross-lingual EER gap, while an intelligibility-bound gap remains.Source: §V · p. 6
  • The authors call for multifaceted attacker modeling and evaluation protocols that weigh privacy preservation against utility degradation.Source: §V · p. 6

VII. GENERATIVE AI USE DISCLOSURE

  • The authors report that LLMs were used only as auxiliary tools for manuscript language polishing; research and final writing decisions were conducted and verified by the authors.Source: §VII · p. 6
Loading 2610.08107v1…