Source-linked AI summary
Traceable TTS: Toward Watermark-Free TTS with Strong Traceability
Yuxiang Zhao, Yunchong Xiao, Yushen Chen, Zhikang Niu, Shuai Wang, Kai Yu, Xie Chen
TL;DR
Highly realistic TTS creates security risks, while explicit watermarking can affect speech quality and introduce spoofing concerns. This paper jointly trains a TTS model and discriminator for watermark-free model traceability, reporting strong generalization and preserved audio quality. The approach remains constrained to differentiable TTS models.
Problem
Realistic synthetic speech requires traceability without compromising audio quality or security, but existing explicit watermarking methods can degrade quality, enable forged watermarks, and lack vocoder robustness.
Method
The paper jointly trains a specific TTS model and discriminator so the discriminator recognizes speech generated by its paired model without explicit n-bit watermarking.
Results
The method significantly improves discriminator generalization to out-of-domain data while maintaining audio quality and robustness against common audio editing attacks.
Takeaways & Limitations
Watermark-free joint training enables active TTS speech traceability without adding extra watermark information to the audio.
Takeaways & Limitations
The approach requires differentiability from output waveform back to input text, excluding widely used discrete token-based models such as VALL-E.
Abstract
from arXiv · showhide
Recent advances in Text-To-Speech (TTS) technology have enabled synthetic speech to mimic human voices with remarkable realism, raising significant security concerns. This underscores the need for traceable TTS models-systems capable of tracing their synthesized speech without compromising quality or security. However, existing methods predominantly rely on explicit watermarking on speech or on vocoder, which degrades speech quality and is vulnerable to spoofing. To address these limitations, we propose a novel framework for model attribution. Instead of embedding watermarks, we train the TTS model and discriminator using a joint training method that significantly improves traceability generalization while preserving-and even slightly improving-audio quality. This is the first work toward watermark-free TTS with strong traceability. To promote progress in related fields, we will release the code upon acceptance of the paper.
1 Introduction
Realistic TTS creates security and privacy risks, while existing detection and watermarking approaches have important limitations. The paper proposes watermark-free traceability through joint TTS–discriminator training.
- Realistic synthetic speech can enable impersonation, fraud, fabricated evidence, and misinformation.
- Deepfake detection classifies speech as real or synthetic but does not identify the generating TTS model.
- Explicit audio watermarks can degrade quality, expose information for counterfeit watermark creation, and reduce invisibility when added after generation.
- Model-watermark methods improve robustness and flexibility, but still rely on explicit n-bit information and may be limited to codec-based TTS.
- The proposed joint training integrates the discriminator with the TTS model to identify speech generated by that model without explicit watermarking.
2 Related work
Related work spans generation tracing in text, images, and audio, but TTS attribution remains challenged by vocoder dependence and poor generalization to unseen models or domains. The paper addresses these issues with a paired discriminator for specific TTS models.
- Prior AIGC research studies tracing and attribution across text, image, and audio generation.
- Text-generation detection uses statistical, zero-shot, style-based, resampling, and model-paired approaches, offering ideas for synthetic-audio tracing.
- Image-generation attribution research shows that generated images can contain source-specific fingerprints useful for identifying their originating models.
- 2. Related work: Audio tracing studies examine vocoder and acoustic-model fingerprints, with prior work finding vocoder fingerprints may obscure acoustic-model fingerprints.
- 2.3 Audio-Based Tracing Methods: TTS traceability must remain robust to interchangeable vocoders and generalize beyond TTS models represented in the training data.
- 2.3 Audio-Based Tracing Methods: The paper pairs a discriminator with a specific TTS model to enable active traceability on unseen out-of-domain datasets.
3 Methods
The method jointly trains a TTS generator and discriminator with aligned objectives so generated speech develops traceable, discriminator-recognizable features while retaining quality. It uses an end-to-end F5-TTS generator, a wav2vec 2.0–LCNN discriminator, and a three-stage training mechanism.
- 3.1 Overall Framework: The framework replaces GAN-style adversarial objectives with aligned generator–discriminator optimization for traceable speech features.The TTS generator and discriminator are jointly optimized, with auxiliary quality losses included in the framework.
- 3.2 Model: F5-TTS serves as the generator because discriminator BCE loss must backpropagate through the complete end-to-end TTS model.The selection also considers audio generation quality, model stability, and reproducibility.
- 3.2 Model: The discriminator combines wav2vec 2.0 feature extraction with an LCNN classifier for audio discrimination.This architecture was selected based on prior performance in audio feature extraction and classification.
- 3.3 Training Mechanism: Training has three stages: F5-TTS finetuning, synthetic-dataset generation by inference, and discriminator training.The vocoder is frozen during the first stage, while generated and ground-truth audio are used in the final stage.
- 3.3 Training Mechanism: The joint objective combines TTS loss and discriminator BCE loss, with λ_TTS=1 and λ_BCE=1 in this work.The BCE objective encourages generated audio to be easier for the discriminator to detect as belonging to the paired TTS model.
- 3.3 Training Mechanism: Untuned F5-TTS audio receives label 0 alongside ground-truth audio so discriminator training does not remain an easy synthetic-versus-real distinction.This design is intended to guide model-specific feature generation and improve generalization to extraterritorial data.
- 3.3 Training Mechanism: Joint training enables tracing the origin of speech produced by the TTS model.The discriminator learns features associated with the jointly trained generator.
4 Experiment Setup
The experiments use LibriTTS and compare the proposed setup with an independently trained discriminator baseline. Testing uses balanced and stratified sampling to reduce class-imbalance bias and improve fairness and reliability.
- Dataset: LibriTTS provides 586 hours of 24kHz audio from 2456 speakers for training and testing.The authors finetune F5-TTS on train-clean-100 and reconstruct dev-clean for discriminator training.
- Dataset: The open-source F5-TTS checkpoint was trained on Emilia, containing about 95k hours of filtered Chinese and English data.The checkpoint uses 24kHz audio and originally contains six listed languages before filtering.
- Baseline: The baseline independently trains a wav2vec 2.0–9-layer LCNN discriminator with BCE loss and Adam optimization.Generated dev-clean audio is labeled 1, ground-truth dev-clean audio is labeled 0, and test-clean is processed correspondingly.
- Training Configuration: The baseline discriminator converges after 30 epochs, while the joint-training procedure runs for 10 loops.Each joint-training loop contains three phases, and the discriminator is trained for 20 epochs within the implementation described.
- Testing: Testing downsamples the larger class to match the smaller class and uses stratified sampling across subsets.This procedure is designed to prevent majority-class prediction and preserve the broader data distribution.
- Testing: The sampling strategy addresses imbalance-related evaluation bias and supports fairer, more reliable classification results.The authors state that it provides a more accurate basis for assessing classification ability and model optimization.
5 Experiment Result
The experiment evaluates the discriminator using AUC, EER, and ACC, with Table 1 configuring in-domain and generalization tests.
- The discriminator is evaluated using AUC, EER, and ACC.AUC is based on the ROC curve, while ACC measures overall classification accuracy.
- AUC uses the ROC curve, whose axes are False Positive Rate and True Positive Rate.
- Table 1 configures both in-domain and generalization tests.The table distinguishes the test settings used to assess performance.
E2 TTS
The evaluation compares in-domain detection, out-of-domain generalization, robustness to audio edits, and speech quality after joint training. Joint training improves generalization while preserving or slightly improving measured speech quality, though additive noise and pitch changes remain challenging.
- Evaluation setup: The evaluation uses LibriTTS test-clean ground truth and synthesized audio from timbre cloning, with finetuned F5-TTS used for the joint-training test.The baseline uses original F5-TTS data, while the proposed model uses finetuned F5-TTS data for a fair comparison.
- In-domain result: In-domain detection of F5-TTS-generated speech versus real speech is easy, and both baseline and joint-training methods achieve very good results.
- Generalization Ability: Joint training significantly improves discriminator generalization on out-of-domain speech from F5-TTS, CosyVoice, CosyVoice 2, and E2-TTS.The reported evaluation uses binary classification results and ROC comparisons across these generated-speech sources.
- Generalization Ability: The jointly trained discriminator achieves consistently higher accuracy across diverse unseen-data test datasets.The paper attributes this improvement to integrating complementary features and optimizing learning across domains.
- Robustness Result: 11% is the approximate performance drop under additive noise and pitch changes, while compression, reverb, and time stretching usually change accuracy by less than 5%.The robustness evaluation covers resampling, speed changes, MUSAN noise, reverb, pitch, volume, MP3, and WAV processing.
- Speech Quality: WER decreases from 2.202% to 2.033%, speaker similarity rises from 0.659 to 0.661, and UTMOS rises from 3.926 to 3.958 after joint training.These results indicate that joint training did not damage generated-audio quality on LibriSpeech-PC test-clean.
6 Conclusion
The work introduces watermark-free TTS traceability through joint training of a TTS model and discriminator, with strong generalization to unseen data and robustness against common audio editing attacks. The approach requires the TTS model to be differentiable from output waveform back to input text, limiting applicability to some discrete token-based TTS models.
- Jointly training the TTS model and discriminator enables traceability without adding explicit watermark information to synthesized audio.
- The method demonstrates strong generalization to unseen data and robustness against common audio editing attacks.
- The framework requires differentiability from the output waveform back to the input text, so discrete token-based models such as VALL-E may not apply.