Tracing and Relearning Detection Evidence in Text-to-Speech Systems

1. INTRODUCTION

• Benchmark EER cannot attribute separation to a TTS stage because benchmarks vary synthesis systems, source corpora, and channel conditions, while silence cues and real-mel vocoding can also separate speech.The paper motivates controls that hold source corpus and channel condition fixed.Source: §1 · p. 1, Figure 1 · p. 1
• With BigVGAN fixed, the larger remaining change in detector separation is attributable to acoustic generation, and fine-tuning the acoustic model raises EER against fixed detectors.The evaluation detectors are not part of the acoustic-model update objective.Source: §1 · p. 1
• Adapting a detector to tuned outputs improves detection of both those outputs and unseen base F5-TTS outputs, so higher EER against fixed detectors need not mean speech is indistinguishable from bona fide speech.The adapted detector was not trained on base-model outputs.Source: §1 · p. 1
• The contributions include quality- and content-comparable acoustic-model updates that raise EER across fixed detectors, and detector adaptation that transfers to base outputs while legacy EER varies by adaptation corpus.The fixed-vocoder comparison traces evidence to acoustic generation under BigVGAN, with the share left for that stage suggested to depend on vocoder.Source: §1 · p. 1

2.1. Paired resynthesis and controls

• R, V, T, FT, and TG define paired conditions: resampling, real-mel reconstruction, base synthesis, fine-tuned synthesis, and gain-matched base synthesis.The vocoder, text processing, prompts, and sampling steps remain fixed between T and FT; TG tests the share of their gap that level alone can produce.Source: §2.1 · p. 2

2.2. Acoustic-model fine-tuning

• The update fine-tunes all 337.1 million DiT parameters while freezing BigVGAN and other TTS components, using a discriminator trained on real-mel reconstructions and generated-mel outputs.The evaluation detectors are excluded from the objective; the adversarial and feature-matching losses are weighted one and two, respectively.Source: §2.2 · p. 2

2.3. Detector adaptation

• Detector adaptation trains released XLS-R SLS with FT labeled synthetic and paired with its prompt-source bona fide utterance, while excluding T examples to measure transfer to base F5-TTS.Evaluation includes held-out conditions and legacy ASVspoof 2021 DF/LA and In-the-Wild benchmarks.Source: §2.3 · p. 2

3.1. Datasets

• LibriSpeech supplies acoustic-model fine-tuning, detector adaptation, and evaluation, with adaptation and evaluation speakers separated.Fine-tuning uses train-clean-100; evaluation uses 800 test-clean utterances from 40 other speakers.Source: §3.1 · pp. 2–3
• VCTK supplies detector adaptation and evaluation, with evaluation speakers held out from the ASVspoof training and development partitions; VoxPopuli is evaluation-only.VCTK bona fide speech is within the detectors’ training domain, and the VCTK bona fide and FT levels differ by 0.09 dB.Source: §3.1 · p. 3

3.2. Training, detectors, and metrics

• The reported acoustic-model checkpoint is the last of ten saved checkpoints whose UTMOS, speaker similarity, and WER are no worse than the base model’s; detector EER is not a selection criterion.Adaptation runs use three seeds and each adapted detector is evaluated on both corpora.Source: §3.2 · p. 3
• Three fixed ASVspoof 2019-trained detectors are evaluated using EER, alongside UTMOS, speaker similarity, and WER for speech quality and content.Movement of EER toward 50% indicates poorer separation under the fixed score orientation.Source: §3.2 · p. 3

4.1. Separation emerges after acoustic generation under BigVGAN

• With BigVGAN, the V–T EER decrease was roughly 3 times the R–V step on LibriSpeech and 8 to 9 times on VCTK.The contrast held across detectors with different front ends, but separation varied by detector and corpus and V–T also includes prosody and acoustic-realization differences.Source: §4.1 · p. 3, Table 1 · p. 3
• For SLS, Vocos real-mel reconstructions were separated more strongly than BigVGAN reconstructions on both corpora.This indicates that reconstruction-stage separation depends on the vocoder.Source: §4.1 · p. 3, Table 2 · p. 3

4.2. Fine-tuning raises EER beyond RMS gain matching

• FT increased EER over T in all six detector–corpus pairs, by 0.64 to 18.92 percentage points, while tuned outputs remained detectable in some conditions.The largest increase was for AASIST-L on LibriSpeech, where FT left the detector near chance.Source: §4.2 · p. 3, Table 1 · p. 3
• On LibriSpeech, RMS gain matching accounted for 52% of the T–FT gap for SLS and 30% for Mamba; on VCTK, TG barely changed while FT increased both EERs.TG isolates the part of the T–FT increase attributable to output level alone.Source: §4.2 · p. 3

4.3. Adaptation learns tuned outputs

• LibriSpeech adaptation lowered FT EER from 19.42% to 1.71% on LibriSpeech and 4.09% to 0.20% on VCTK; VCTK adaptation lowered it to 7.46% and 0.10%, respectively.T also reached comparable EER in both adaptation runs, although only FT was labeled synthetic.Source: §4.3 · p. 4, Table 4 · p. 4
• After adaptation, R stayed near chance and V remained far above T and FT, although VCTK adaptation lowered V EER on both corpora.The adapted detector did not learn to separate resampled audio, but partly separated BigVGAN output from a real mel.Source: §4.3 · p. 4
• VCTK adaptation retained performance closer to the released checkpoint on ASVspoof 2021 DF, while LibriSpeech adaptation had higher EER on all three legacy benchmarks.The authors attribute the difference to adaptation corpus, but the experiment cannot separate bona fide-domain effects from those of the paired synthetic outputs.Source: §4.3 · p. 4
• The study covers one acoustic model, one vocoder, and one detector architecture; reported standard deviations reflect generation and adaptation seeds, not speaker variation.These scope limits bound how broadly the findings can be generalized.Source: §4.3 · p. 4

• In the F5-TTS–BigVGAN pipeline, acoustic generation produces a larger change in detector separation than real-mel reconstruction.Fine-tuning makes outputs harder for fixed detectors to identify at comparable quality and content measures; adapting XLS-R SLS improves detection of both fine-tuned outputs and unseen base-model outputs.Source: §5 · p. 4