PaperlayerView exact revision on arXiv

Source-linked AI summary

From Speech Tokens to Stolen Voices: Attacker-Chosen-Text Speaker Impersonation via Representation Inversion

Ye Lu, Shen Wang, Runze Liu, Zhitao Ou, Haonan Huang, Yihan Yan, Fanghui Sun, Zhaoyang Zhang

arXiv:2610.34688v1cs.SDcs.AIcs.CRcs.CVcs.MM

TL;DR

Frontend representations may retain speaker identity even when raw audio stays on the user’s device, leaving open whether an attacker can use them to generate new speech in the victim’s voice with chosen text. STIA conditions speech generation on leaked representations and attacker-chosen text; across three frontends it outperforms SpInv+TTS on speaker similarity and target-text WER, with 0.78 CAM++ similarity and 3.80% WER for a 3-second Moshi leak.

  • Problem

    Can an attacker use identity information retained in leaked frontend representations to generate new speech that preserves the speaker’s voice and conveys chosen text, without access to the original audio?

    Source: §1 · p. 1

  • Method

    STIA converts discrete, continuous, or mixed leaked representations into conditioning sequences and uses same-speaker, cross-utterance public speech training to generate speech with attacker-chosen text.

    Source: §1 · p. 2

  • Results

    Across three frontends, STIA achieves higher mean speaker similarity and lower mean target-text WER than SpInv+TTS; with a 3-second Moshi leak, CAM++ cosine similarity is 0.78 and WER is 3.80%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Under the evaluated conditions, fragmented frontend representations can support speech generation that retains a victim’s vocal characteristics and conveys new, attacker-chosen content, creating voice-impersonation and private-speaker-information risks.

    Source: §1 · p. 2

    Generalization across datasets or languages and transfer across frontends without retraining remain untested; generation also depends on the frozen Higgs codec’s representational and reconstruction capabilities.

    Source: §6.3 · p. 12

Abstract

from arXiv · show

Speech interaction systems have gradually evolved from traditional cascaded systems that use text as intermediate information to end-to-end speech language models that directly process speech representations. In some applications, the speech frontend runs on the user's device and sends speech representations to a remote server for inference, while the original audio remains local. However, these representations may still contain speaker identity information. If an attacker obtains them, it may use this information to imitate the user's voice, threatening their voice privacy. To investigate this risk, we propose \stia, a Speech Token Inversion Attack that generates speech with attacker-specified text in a victim's voice using a single short leaked representation, without access to the original audio or its transcript. Specifically, \stia converts discrete, continuous, or mixed speech representations into conditioning sequences and trains a conditional flow-matching speech generation model using pairs of different utterances from the same speaker in public speech datasets. The model combines speaker characteristics contained in the leaked representation with the specified text to generate speech in the victim's voice. Experiments show that, with a Moshi representation leak corresponding to 3 seconds of speech, \stia achieves an average CAM++ speaker cosine similarity of 0.78 and a target-text word error rate of 3.80\%. These results show that speech frontend representations reveal reusable speaker identity information and highlight the need to consider privacy risks when designing speech frontend representation methods.

1. Introduction

STIA investigates whether leaked speech frontend representations can support arbitrary-text speech generation in a victim’s voice, even when the waveform stays local. It combines a short representation fragment with chosen text and outperforms a speaker-embedding-plus-TTS baseline across three evaluated frontends.

1. Introduction

  • Prior work found speaker identity information in frontend representations, leaving open whether it could support speech generation that preserves identity and conveys unrelated chosen text.In split deployment, raw audio can remain on the device while representations are sent to a remote server.Source: §1 · p. 1
  • STIA combines a leaked representation fragment with independently chosen text to generate speech without the victim’s waveform, source transcript, or victim-specific training.It learns reusable vocal cues from different utterances by the same speaker in public auxiliary speech, using conditional flow matching and codec-aware decoding supervision.Source: §1 · pp. 1–2
  • With a 3-second Moshi leak, STIA achieves mean CAM++ cosine similarity of 0.78 and mean target-text WER of 3.80%.Across three evaluated frontends, it also has higher mean speaker similarity under CAM++ and ECAPA and lower mean WER than SpInv+TTS, with similar mean UTMOS naturalness scores.Source: §1 · p. 2
  • The paper presents STIA as the first study of arbitrary-text speaker impersonation from leaked frontend representations of end-to-end speech language models.The attack accepts discrete, continuous, and mixed representations.Source: §1 · p. 2

2. Related Work

Related work traces speech representations and model inversion, then connects speaker recognition and anonymization to zero-shot voice cloning. STIA’s focus is using leaked frontend representations directly for cross-content speech generation.

2.1. Speech Representations and Speech–Language Interfaces

  • Speech–language interfaces use discrete or continuous representations that may retain prosody and speaker cues; related systems connect these representations to generative language models [1],,.The section frames exposure at module boundaries, caches, or logs as a security question about whether fragments retain reusable speaker information.Source: §2.1 · p. 3

2.2. Model Inversion and Privacy of Intermediate Representations

  • Prior inversion work targets training inputs, sensitive features, or recoverable embeddings; STIA instead generates speech directly from leaked representations and target text,, [4].The SpInv-based SpeechT5 baseline recovers speaker embeddings, an objective that may limit retained timbral and prosodic detail when paired with TTS.Source: §2.2 · p. 3

2.3. Speaker Recognition and Voice Anonymization

  • Speaker-recognition methods evolved from GMM–UBM models and i-vectors to x-vectors and newer verification architectures, while anonymization targets identity or linkability,,,,,,.The paper tests whether representations exposed after McAdams anonymization still condition cross-content synthesis with a fixed STIA model.Source: §2.3 · p. 3

2.4. Zero-Shot Voice Cloning and Audio Deepfakes

  • Zero-shot voice cloning synthesizes new text from short reference recordings, while flow-matching speech generators typically derive voice conditions from reference audio,,,,, [32].STIA studies direct use of leaked representations without the victim’s waveform or source transcript; ASVspoof addresses the complementary task of detecting synthetic, converted, or replayed speech.Source: §2.4 · p. 3

3. Threat Model and Motivation

The paper studies whether a short frontend-representation leak in a split deployment can support impersonation, despite the victim’s waveform and transcript remaining unavailable. The attack seeks to preserve vocal identity, convey attacker-chosen text, and retain natural-speech quality.

3.1.1. Target model.

  • The split deployment keeps source waveforms on-device while making frontend representations available to the remote service.Representations may retain prosody, speaker characteristics, and channel conditions; the attacker intercepts only streams from a finite segment, not the full-utterance representation.Source: §3.1.1 · pp. 3–4

3.1.2. Adversary knowledge and capabilities.

  • The adversary reads one victim’s fragmented representation and can query, but not modify, the black-box frontend; it trains offline on public auxiliary speech.Victim waveforms, transcripts, identity labels, and victim-specific training data are unavailable; attacker-chosen text is independent of source content.Source: §3.1.2 · p. 4, §3.1.3 · p. 4

3.1.3. Adversary objective.

  • The attack objective favors higher identity similarity and naturalness alongside lower content error, allowing tradeoffs rather than combining goals into one weighted score.The scores measure speaker similarity, target-text error, and predicted naturalness.Source: §3.1.3 · p. 4

3.2. Motivation

  • Prior work establishes that frontend representations can retain identity information, but leaves open whether a short leak enables new speech with chosen text and the victim’s voice.Reference-audio cloning and embedding recovery separately establish speaker conditioning and retained identity cues, not direct generation from leaked representations.Source: §3.2 · pp. 4–5

4. Speech Token Inversion Attack

STIA learns to turn fragmented frontend representations and attacker-chosen text into speech conditions, using same-speaker, cross-utterance training. At attack time, it samples speech latents and decodes them into an output waveform.

4.1. Overview

  • STIA combines a leak-derived speaker condition with a target-text condition to generate speech in a continuous codec latent space.A shared Transformer supports denoise and decode modes, while a stop head predicts the output boundary.Source: §4.1 · p. 5

4.2. STIA Training

  • Training pairs a source representation fragment with target speech and text from a different utterance by the same speaker.The leak encoder maps discrete, continuous, or mixed frontend outputs to a common-width condition; the text encoder and projection supply target-content conditioning. Training combines flow-matching, codec-aware, and stopping losses.Source: §4.2 · pp. 5–6
  • Decode mode adds target-code supervision to connect predicted continuous endpoints with the frozen codec’s quantization structure.The shared network also receives stopping supervision in both modes; training samples denoise mode with probability 0.8 and decode mode with probability 0.2.Source: §4.2 · pp. 6–7

4.3. STIA Attacking

  • At attack time, STIA uses Euler ODE sampling to generate latents conditioned on the leaked representation and chosen text, then decodes them into speech.After a final conditional decode determines valid length, frozen RVQ selects codes and the Higgs waveform decoder synthesizes the output.Source: §4.3 · p. 8

5. Experiments

The experiments evaluate STIA against a two-stage embedding-recovery-and-synthesis baseline across three speech-model frontends. The supplied passages describe the evaluation setup and metrics, but not comparative outcomes.

5.1. Experimental Setup

  • The evaluation covers Moshi, Kimi-Audio, and Qwen3-Omni input-side frontends, with attack models trained and validated separately by language.The selected representations instantiate discrete, mixed, and continuous frontend types; validation speakers are disjoint from training speakers.Source: §5.1 · p. 8
  • SpInv+TTS is the baseline: SpInv recovers SpeechT5-compatible speaker embeddings, which SpeechT5 uses with attacker-chosen text to synthesize speech.Both methods use the same leakage segments, texts, victim trials, and evaluators.Source: §5.1 · pp. 8–9
  • Evaluation measures speaker identity with CAM++ and ECAPA cosine similarity, content accuracy with WER or CER, and predicted naturalness with UTMOS.Similarity and naturalness are higher-is-better; error rates are lower-is-better.Source: §5.1 · p. 9
  • Each configuration generates four samples for each of 1,000 recordings using 50-step Euler sampling and CFG scale 2.Scores are aggregated separately by language; Table 2 reports mean ± standard deviation and Tables 3–8 report means.Source: §5.1 · p. 9

5.2. Comparison with SpInv+TTS

Across three speech frontends, STIA outperforms SpInv+TTS on speaker similarity and target-text accuracy while maintaining similar predicted naturalness.

5.2. Comparison with SpInv+TTS

  • Both attacks are evaluated using the same 3-second victim representation fragment, chosen English text, and genuine enrollment utterance.Source: §5.2 · p. 9
  • Across three frontends, STIA yields higher similarity under both speaker encoders and lower mean WER than SpInv+TTS, with similar mean UTMOS.The comparison uses identical 3-second representation leaks and attacker-chosen English prompts on unseen Emilia speakers.Source: §5.2 · pp. 9–10, Table 2 · p. 10
  • Kimi-Audio produces the highest identity similarity, with mean WER of 3.84%, close to Moshi’s 3.80%.UTMOS means remain similar between the two attacks; the principal gains are identity similarity and text accuracy.Source: §5.2 · p. 10

5.3. Language Experiments

Language-specific evaluations retain speaker similarity in English and Chinese; auxiliary-data and leakage-duration ablations show improvements in identity similarity and text accuracy under their tested conditions.

5.3. Language Experiments

  • STIA retains speaker similarity in separate English and Chinese evaluations, but population differences prevent attributing score differences to language alone.Each language uses its own training, speakers, and frozen prompts; English WER and Chinese CER have different counting units.Source: §5.3 · p. 10

5.4. Auxiliary-Data Ablation

  • At a fixed 150,000 training steps, more auxiliary Emilia-English data improves identity scores and lowers mean WER, while UTMOS remains largely unchanged.Identity gains are smaller from the middle to largest data budget than from the smallest to middle budget.Source: Table 4 · p. 10, §5.4 · p. 10

5.5. Leakage-Duration Ablation

  • Across 2-, 3-, 5-, and 8-second leaks, both identity means increase with duration, while mean WER falls from 2 to 3 seconds and then stabilizes.The 3-second setting captures most of the text-accuracy benefit in this comparison; longer leaks primarily add identity similarity.Source: Table 5 · p. 10, §5.5 · p. 10

5.6. Model-Scale Ablation

Within tested scales, larger STIA models improve measured identity similarity, text accuracy, and predicted naturalness. McAdams anonymization attenuates a fixed attacker, while conditioning controls distinguish leakage and target-text effects.

5.6. Model-Scale Ablation

  • Among approximately 0.35B, 0.50B, and 0.60B models, XL has the best mean on all four metrics within the tested range.Training data, leakage fragments, frontend, trial lists, and other settings are fixed across the scale ablation.Source: §5.6 · p. 10, Table 6 · p. 11

5.7. Speaker Anonymization

  • McAdams anonymization lowers mean similarity under both speaker encoders and UTMOS, with a small WER increase, but does not establish identity removal.The fixed attacker is not retrained or tuned, and adaptive attackers remain unevaluated.Source: §5.7 · p. 11

5.8. Conditioning and Text Controls

  • Mismatched-text scoring yields 99.17% WER against an unpaired reference, not a measure of adherence to the generation text.The generated speech is conditioned on yA but scored against a different, unpaired yB.Source: §5.8 · p. 11
  • Removing leakage reduces CAM similarity from 0.78 to 0.25, whereas retaining leakage without text yields 0.57.The text-only no-leak condition may be out of distribution because training jointly drops text and leakage, so degradation alone cannot establish leakage necessity.Source: §5.8 · p. 11

5.9. Training Settings and Cost

The main Moshi configuration uses a 3-second leak and an approximately 0.60B-parameter XL model, trained for 150,000 steps.

5.9. Training Settings and Cost

  • The main Moshi configuration uses a 3-second leak and the XL model with approximately 0.60B parameters.Source: §5.9 · p. 12
  • All STIA training configurations use 150,000 steps.Source: §5.9 · p. 12
  • The main Moshi-targeting model took 110 hours to train in BF16 on two 40-GB NVIDIA A100 GPUs.The reported time excludes preprocessing cached mT5 text representations and Higgs speech latents.Source: §5.9 · p. 12

6. Discussion

Across three representative frontends, STIA outperforms the SpInv+TTS baseline on speaker similarity and target-text accuracy, while predicted naturalness remains similar. The discussion covers representation protections, complementary defenses, and limitations of the evaluation.

6.1. Security Implications

  • STIA has higher mean speaker similarity and lower mean target-text WER than SpInv+TTS, while mean UTMOS scores remain similar.Privacy evaluation should consider vocal identity, text accuracy, and naturalness together.Source: §6.1 · p. 12

6.2. Defensive Directions

  • Limiting representation access is a direct defense, complemented by altering identity cues and application-level safeguards.Encryption protects representations in transit, while endpoint controls reduce exposure; legitimate recipients can still reuse them. McAdams anonymization lowered speaker similarity and UTMOS for the fixed attacker, but does not establish identity removal; its attacker was trained only on unanonymized speech. Additional authentication factors or transaction-specific confirmation can help limit forged-speech consequences.Source: §6.2 · p. 12

6.3. Limitations

  • Evaluation covers three English frontends and a separate Chinese Moshi test, leaving cross-dataset, cross-language, and transfer generalization untested.Generation also depends on the frozen Higgs codec, and automatic metrics may not match listener judgments; authentication success and operational impersonation effectiveness were not measured.Source: §6.3 · p. 12

7. Ethics Considerations

The study uses public data for controlled, offline privacy measurement without recruiting participants or attacking external services. It reports aggregate results and emphasizes that dataset availability does not imply consent to voice cloning.

7. Ethics Considerations

  • Experiments use public Emilia data in controlled settings, without recruiting participants, collecting private recordings, or attacking external services.The dataset is distributed under CC BY-NC 4.0.Source: §7 · p. 12
  • Public availability does not imply speaker consent to voice cloning, which could enable unauthorized impersonation or fraud.The paper reports aggregate metrics rather than identifiable voice demonstrations.Source: §7 · p. 12
  • Future releases should respect dataset licenses and restrict identifiable audio and reusable attack models.The study aims to expose risks and inform more privacy-preserving speech interfaces.Source: §7 · pp. 12–13

8. Conclusion

The conclusion presents STIA as a method for generating arbitrary text in a victim’s voice from leaked frontend representation fragments, then identifies evaluation and defense priorities for future work.

8. Conclusion

  • STIA combines cross-utterance training with conditional flow matching to generate arbitrary text from leaked frontend fragments without the victim’s waveform or source transcript.Experiments on three representative frontends report higher mean speaker similarity and lower mean target-text WER than SpInv+TTS.Source: §8 · p. 13
  • Future evaluation should cover additional datasets, languages, and realistic deployment conditions.The conclusion also calls for listener studies and tests against operational authentication systems.Source: §8 · p. 13
  • Future work should assess stronger anonymization and frontend protections against adaptive attackers while measuring impacts on legitimate speech tasks.Source: §8 · p. 13

Related papers

Loading 2610.34688v1…