Source-linked AI summary
Perceptible or Not? Diagnosing Passive Fingerprints for Speech Deepfake Attribution
Yupei Li, Qiyang Sun, Emmanouil Benetos, Berrak Sisman, Björn Schuller
TL;DR
The paper addresses limited evidence about the persistence, reproducibility, and content independence of speech-generator fingerprints, and the lack of a perceptibility-based distinction. It proposes PIPDP, which combines multi-evidence verification with perceptually transparent perturbations and prompt-driven emotion changes; experiments indicate that imperceptible fingerprints provide more reliable attribution cues.
Problem
Evidence remains limited about passive fingerprints’ persistence, reproducibility, and content independence, and prior work does not distinguish perceptible from imperceptible fingerprints.
Method
PIPDP separately analyzes perceptible and imperceptible fingerprints using verification probes, perceptually controlled perturbations, and prompt-driven emotion ablation.
Results
Imperceptible fingerprints provide persistent attribution cues, while perceptually transparent perturbations reduce attribution accuracy by up to 48.2%.
Takeaways & Limitations
Imperceptible fingerprints are more persistent and reliable attribution signals than perceptible fingerprints.
Takeaways & Limitations
Cross-domain speech-emotion recognition remains highly challenging, so relatively low SER accuracy does not necessarily indicate poor emotion generation.
Abstract
from arXiv · showhide
Passive fingerprints (intrinsic traces naturally left by generators) have been shown to enable attribution in speech deepfake detection, yet their persistence, reproducibility, and content-independence remain unverified. Moreover, no prior work distinguishes perceptible from imperceptible fingerprints, although the two have very different implications for attribution reliability. Perceptible fingerprints, such as emotional expression, are shaped by perceptual quality objectives and may change across model updates, whereas imperceptible fingerprints are not explicitly optimised by current training objectives and are rarely considered in existing dataset design or training strategies, as they have limited influence on downstream applications. We therefore propose a Perceptible-Imperceptible Passive-fingerprint Diagnostic Protocol (PIPDP) to define and separately analyze these two fingerprint types. PIPDP comprises three complementary analyses: multi-evidence fingerprint verification through residual-energy, reproducibility, and saliency analyses, perceptually transparent perturbations preserving audio quality, and prompt-driven emotion change that modifies perceptible fingerprints without model retraining. Experiments across ten speech generators and three attribution detectors show that imperceptible fingerprints provide persistent attribution cues. Perceptually transparent perturbations reduce attribution accuracy by up to 48.2\% on HiggsAudioV3, whereas emotion-driven changes leave attribution largely unchanged, with only about a 1.0\% accuracy variation across emotions on CosyVoice2 using w2v-bert-MLP. These results suggest that imperceptible fingerprints are more reliable for trustworthy attribution.
1. INTRODUCTION
The paper distinguishes perceptible from imperceptible passive fingerprints and addresses gaps in evidence about their persistence, reproducibility, and content independence. It proposes PIPDP and reports that imperceptible fingerprints provide more persistent and reliable attribution signals.
- Passive fingerprints are unintended generator traces, unlike active watermarks deliberately embedded for traceability.
- Perceptible fingerprints are human-audible, whereas imperceptible fingerprints are not consciously perceived but still preserve source-related information.
- Imperceptible fingerprints are rarely optimized explicitly because they do not affect perceived audio quality, leaving limited supervised datasets and systematic analyses.
- Prior studies mainly test robustness to external degradations, leaving persistence, reproducibility, and content independence largely unexplored.
- PIPDP is a lightweight, model-agnostic protocol using training-free input ablations to diagnose fingerprint reliability across ten speech generators and three cross-architecture detectors.
2. METHOD: PERCEPTIBLE-IMPERCEPTIBLE PASSIVE-FINGERPRINT DIAGNOSTIC PROTOCOL
PIPDP separates perceptible and imperceptible passive fingerprints and evaluates their reliability through three complementary probes within a common attribution pipeline.
- PIPDP characterizes passive fingerprints along the axis of human perceptibility.
- Probe 1 tests whether attribution signals correspond to persistent and reproducible generator-specific traces.
- Probe 2 uses perceptually controlled audio perturbations to examine whether reliable attribution fingerprints are imperceptible.
- Probe 3 modifies only perceptible attributes through prompting while keeping generator parameters fixed to test their stability.
2.1. Attribution pipeline
The attribution pipeline covers ten speech generators and uses a closed-set 10-way evaluation with controlled training, in-domain testing, and OOD testing data.
- The study selects ten generators spanning vocoders, classical synthesis, LLM-based models, and diffusion models.
- The dataset contains 25,000 synthesized utterances from 2,500 utterances sampled from ten speakers and rendered by all ten generators.
- The data split uses 1,800, 450, and 250 utterances for training, in-domain testing, and OOD testing, respectively.
- Attribution is evaluated in a closed-set 10-way setting, with an additional OOD test set built from ten ESD speakers and 100 English scripts per speaker.
2.2. Probe 1: Existence and reproducibility
Probe 1 triangulates fingerprint existence, persistence, and reproducibility using attribution accuracy, residual-energy patterns, Grad-CAM saliency, and human inspection.
- Classifier accuracy alone may reflect classifier ability, so Probe 1 combines four complementary signals to verify generator fingerprints.
- Residual-energy analysis compares generated speech with real references using mel-power spectrograms and frequency-wise residual energy.
- The analysis evaluates cross-content consistency, cross-generator distinguishability, and reproducibility for stochastic generators.
2.3. Probe 2: Audio perturbation
Probe 2 tests whether attribution depends on imperceptible signal components by applying perceptually transparent perturbations that selectively alter phase, high frequencies, spectral detail, waveform precision, or narrow frequency bands.
- Perturbation design: Perceptually transparent perturbations and accuracy or probability drops quantify the reliability of imperceptible fingerprints.DNSMOS, STOI, and WER assess whether perturbations remain perceptually transparent.
- Phase perturbation: Phase perturbation adds small independent uniform noise to the STFT phase while preserving magnitude, with σ set to 0.2 and 0.4.The method relies on the human auditory system's relative insensitivity to phase.
- High-frequency dithering: High-frequency dithering injects complex Gaussian noise above 6 kHz, with energy proportional to local high-frequency energy and r set to 20 and 10.The modified region lies just beyond the primary speech-intelligibility range.
- Spectral smoothing: Frequency-axis magnitude smoothing preserves phase while suppressing fine spectral variations and retaining coarse spectral structure.This targets fingerprints potentially caused by resolution mismatches between internal representations and STFT analysis.
- Bit-depth dither: Bit-depth dither requantizes waveforms to 8- or 10-bit precision, decorrelating quantization errors and removing sub-LSB details without tonal artifacts.The operation uses triangular probability density function dither.
- Notch filtering: A second-order IIR notch filter selectively attenuates a randomly centered narrow frequency band to test reliance on localized spectral information.The bandwidth is controlled by the quality factor Q, set to 40 or 20.
2.4. Probe 3: Emotion ablation
Probe 3 tests perceptible-fingerprint stability by changing only prompted emotion while holding generator parameters and other generation conditions fixed.
- Probe rationale: Modifying perceptible attributes through prompting tests whether emotion-linked fingerprints remain stable without retraining the generator.A non-trivial attribution drop would indicate lower stability and reliability.
- Experimental setup: For CosyVoice2 and HiggsAudioV3, happy, sad, and angry versions are generated against neutral while fixing text, speaker reference, model weights, and decoder seed.Only the emotion descriptor is minimally changed in the prompts.
3. EXPERIMENTS AND RESULTS
Experiments evaluate attribution backbones and diagnose passive fingerprints through saliency, residual-energy, reproducibility, perceptually transparent perturbations, and emotion changes. The results indicate that imperceptible fingerprint components are persistent and attribution-relevant, whereas perceptible emotion changes have limited impact on attribution.
- Attribution setup: Attribution features were distinguishable across backbones; RawNet2 performed worst, so w2v-bert-MLP became the primary attributor.The comparison reports Accuracy / Macro-F1 and attributes RawNet2’s lower performance likely to its smaller model capacity.
- Existence and reproducibility: Grad-CAM maps showed overlapping salient regions across utterances from the same generator despite speaker and content changes, but noticeable differences across generators.The visualization retained the top 15% highest-saliency regions and included HiFiGAN and F5-TTS examples.
- Existence and reproducibility: Each of the ten generators traced a visually separable spectral profile in the residual-energy analysis.The profiles were compared on one held-out utterance using ΔE(t), where positive and negative values indicate more or less energy than real speech in a band.
- Existence and reproducibility: 2.00%/1.78% and 6.00%/3.65% accuracy/probability decreases were the only measurable reproducibility drops, for Qwen3-TTS and XTTS-v2 respectively.Deterministic vocoders showed zero degradation, while other stochastic generators remained highly stable across ten regenerations with different random seeds.
- Audio perturbation: 48.2% and 29.9% accuracy drops occurred under moderate BD (b=10) for w2v-bert-MLP and WavLM-AASIST despite largely imperceptible perturbations.The perturbation criterion used ΔDNSMOS below 0.8 and STOI above 0.95; degradation trends across both attribution models implicated generator-specific fingerprints.
- Emotion ablation: Emotion changes reduced attribution by less than 4.00% for HiggsAudioV3 and around 1.00% for CosyVoice2, while intended emotions remained recognizable.Human majority-vote recognition reached 100% for angry, 95% for neutral, 80% for happy, and 78% for sad; ΔE(t) patterns remained consistent across emotions.
4. CONCLUSION
PIPDF separates perceptible and imperceptible fingerprints in speech deepfake detection and uses controlled interventions with attribution analysis to assess their reliability. It finds that imperceptible fingerprints are generally more persistent.
- PIPDF separates perceptible and imperceptible fingerprints and combines controlled interventions with attribution analysis to assess fingerprint reliability.