PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Data Drift Monitoring for Deployed Deepfake Speech Detectors

Lukáš Prokeš, Vojtěch Staněk, Anton Firc, Kamil Malinka

arXiv:2610.16826v1cs.SDcs.AIcs.CRcs.CVcs.LG

TL;DR

Deployed deepfake speech detectors face evolving inputs, while labels are unavailable to determine whether conditions remain familiar. This paper adds an unsupervised monitor that compares detector embeddings with bona fide and spoof references. The monitor reduces deepfake false acceptance and also detects bona fide drift, with alerts trading off against automatic coverage.

  • Problem

    Deployment inputs can differ from training attacks and recording conditions, while labels are unavailable during operation to establish whether the detector remains in familiar conditions.

    Source: §I · p. 1

  • Method

    An unsupervised monitor compares a fixed detector’s internal embeddings with bona fide and spoof training references to identify trusted, ambiguous, or novel inputs.

    Source: §I · p. 1

  • Results

    The monitor substantially reduces silent deepfake acceptance, while bona fide drift also produces alerts and the alert rate trades off against automatic coverage.

    Source: §V · pp. 5–6

  • Takeaways & Limitations

    Deepfake speech detection should assess whether inputs remain within familiar training conditions, not only classify them as genuine or fake.

    Source: §V · p. 5

    The monitor flags inputs for review but does not determine detector updates, unknown-data labeling, or retraining, and its behavior depends on embedding geometry and representative training references.

    Source: §V · p. 6

Abstract

from arXiv · show

Deepfake speech detectors are typically evaluated as binary classifiers on fixed benchmarks, but deployed systems face evolving attacks, codecs, speakers, and recording conditions, with labels unavailable during operation. This makes it difficult to know when a detector operates in familiar conditions and can be relied on. We propose an unsupervised drift monitor that operates on the internal embeddings of a deployed detector. The monitor compares each input against bona fide and spoof references using regularized Mahalanobis distances. These distances define trusted, ambiguous, and novel regions of the detector embedding space, allowing unfamiliar inputs to be routed for review. We evaluate the monitor on four modern detectors on three datasets. The monitor raises alerts for dataset mismatches and substantially reduces the security-critical case of accepting deepfakes at the cost of reduced automatic coverage. The results also show that drift affects both classes: bona fide recordings can move away from the training reference, increasing alert rates even in in-domain settings. Our findings suggest that deployed deepfake speech detectors should be treated not only as a binary classification, but also as a reliability problem.

I. INTRODUCTION

Deployed detectors face evolving inputs and unavailable labels, so benchmark performance alone does not establish whether a decision can be trusted. The paper proposes an unsupervised, two-reference embedding monitor and evaluates its ability to flag drift and reduce false acceptance.

I. INTRODUCTION

  • Evolving attacks, codecs, speakers, channels, and recording conditions can shift deployment data, while labels needed for standard evaluation are unavailable during operation.A shifted detector may still make confident but wrong decisions, making silent failure a practical concern.Source: §I · p. 1
  • The proposed monitor compares detector embeddings with bona fide and known-spoof references without retraining or labels for incoming recordings.It distinguishes familiar, ambiguous, and novel inputs using the two references.Source: §I · pp. 1–2
  • The two-reference view distinguishes security-relevant drift: attacks moving toward bona fide speech may be more dangerous, while genuine-speech drift can increase false alarms.The paper identifies channel, codec, language, and speaker differences as possible sources of bona fide drift.Source: §I · p. 1
  • Across four detectors and three datasets, the monitor alerts on dataset mismatch, reduces deepfake false acceptance, and exposes bona fide drift as a source of uncertainty.The evaluation uses AS19 and AS5 evaluation subsets and In-the-Wild.Source: §I · pp. 1–2
  • The findings frame deepfake detection as a reliability problem as well as binary classification, with uncertainty indicated when inputs fall outside training conditions.The stated evaluation also reports reduced automatic coverage as a trade-off for review.Source: §I · p. 2

II. BACKGROUND

Benchmark evaluation uses fixed datasets, but deployed detectors encounter open-set inputs that may differ from known attacks and recording conditions. This motivates label-free monitoring of whether an input resembles the detector’s training distributions.

II. BACKGROUND

  • Detectors trained on finite known attacks may encounter newer synthesis methods, codecs, or processing pipelines after deployment, so high benchmark accuracy alone does not guarantee reliable operation.Benchmark datasets provide controlled protocols and common evaluation data, but deployment is open-set.Source: §II · p. 2
  • Under distribution shift, neural detectors can remain confident while wrong, and confidence-based rejection may therefore be insufficient.The passage motivates monitoring whether inputs resemble the bona fide and spoof conditions used for training.Source: §II · p. 2
  • The deployment pipeline extracts an utterance embedding and compares it with both bona fide and spoof references, routing borderline or novel inputs to review.Inputs in known regions are trusted for automatic processing.Source: Figure 1 · p. 2
  • The proposed objective checks consistency with known bona fide and spoof training distributions, rather than predicting whether the detector’s current decision is correct.The paper contrasts this with dataset-level drift monitoring and learned sample-level reliability estimation.Source: §II · p. 2

III. UNSUPERVISED DRIFT MONITORING

The monitor is added to a fixed deepfake speech detector and uses its internal utterance embedding without changing detector parameters or using evaluation labels.

III. UNSUPERVISED DRIFT MONITORING

  • The monitoring setup keeps the trained detector’s parameters fixed and does not use labels from evaluation data.Source: §III · p. 2
  • The monitor operates on the fixed detector’s internal representation rather than requiring detector retraining or evaluation labels.Source: §III · p. 2
  • For an input recording, the detector’s utterance-level embedding is extracted before the final classification head.The embedding is denoted z and has dimension 1024.Source: §III · p. 2

A. Reference profiles and covariance regularization

The monitor builds bona fide and spoof reference profiles from training embeddings, regularizes their covariance estimates, and measures incoming embeddings against both profiles.

A. Reference profiles and covariance regularization

  • Each detector’s training split supplies separate bona fide and spoof reference profiles containing embeddings from the corresponding recordings.Source: A. Reference profiles and covariance… · p. 2
  • Shrinkage covariance regularization addresses unstable ordinary covariance estimates when embedding variance is concentrated in few directions and near-zero in others.The final monitor uses α = 0.9, selected for stable preliminary results.Source: A. Reference profiles and covariance… · pp. 2–3
  • For each incoming embedding, regularized Mahalanobis distances measure its separation from the bona fide and known-spoof references.The two distances indicate resemblance to training regions rather than directly predicting the binary class.Source: A. Reference profiles and covariance… · p. 3

B. Two-reference categorization

Two reference distances and training-calibrated thresholds divide the embedding space into familiar, ambiguous, and novel regions. The categorization estimates whether the detector operates in a familiar part of its embedding space, without replacing its binary classifier.

B. Two-reference categorization

  • The monitor marks an input close to reference Rc when its distance is at or below τc, with each τc set to the 95th percentile of training self-distances.Source: B. Two-reference categorization · p. 3
  • Inputs close only to the bona fide or spoof reference are known bona fide or known spoof; inputs close to both are borderline, and inputs far from both are novel.Novel inputs can be further labeled bona fide-like or spoof-like according to which reference is closer.Source: B. Two-reference categorization · p. 3
  • Borderline inputs are simultaneously close to both references, so their automatic detector decisions should not be trusted.Source: B. Two-reference categorization · p. 3
  • In the illustrated MHFA space, AS19 evaluation spoofs move toward bona fide, while AS5 bona fide samples cross τb into spoof regions.Source: Figure 2 · p. 3

C. Operating policy and evaluation

The monitor gates automatic processing according to whether an input is familiar or ambiguous. It routes novel and borderline samples for review and automatically processes known-region samples using the detector prediction.

C. Operating policy and evaluation

D. Experiment setup

The evaluation tests four pretrained detectors across two ASVspoof benchmarks and In-the-Wild, with in-domain status determined by the detector’s training dataset. It reports detector performance and operational monitor outcomes under a fixed classification threshold.

D. Experiment setup

  • Four systems use XLS-R-300M with either an MHFA or AASIST backend, each trained on AS19 or AS5.Source: D. Experiment setup · p. 3
  • Evaluation covers AS19 eval, AS5 eval, and In-the-Wild; only the detector’s training benchmark is in-domain, and mismatched datasets show degraded detector performance.In-the-Wild represents a more realistic open-set setting; the table reports EER (%) across evaluation datasets.Source: D. Experiment setup · pp. 3–4, Table I · p. 4
  • The detector threshold is fixed at 0.5, and post-monitoring deepfake FAR counts deepfakes classified as bona fide and wrongly trusted.Changing the classification threshold affects detector errors and operational outcomes, but not monitor categories or alert rates.Source: D. Experiment setup · p. 4

IV. DRIFT MONITOR RESULTS

The monitor often raises out-of-domain alerts, especially on In-the-Wild, and exposes drift in bona fide recordings as well as spoof attacks. Its conservative calibration also produces alerts in some in-domain settings, trading automatic coverage for review.

IV. DRIFT MONITOR RESULTS

  • In-domain alerts can occur because benchmark membership does not ensure that evaluation recordings match the training reference distribution.ASVspoof training and evaluation splits are disjoint across speakers and synthesizers.Source: §IV · p. 4
  • Out-of-domain alerts are often elevated, especially on In-the-Wild, as unfamiliar embeddings are routed away from automatic processing.Table II reports percentages of correctly trusted, alerted, and wrongly trusted inputs.Source: Table II · p. 4, §IV · p. 4
  • AS5 alert rates remain high even for AS5-trained detectors, with alerts arising partly from bona fide drift that may relate to codec and condition variability.The authors regard AS5 evaluation as a reasonable generalization test even for AS5-trained detectors.Source: §IV · p. 4
  • The 95th-percentile self-distance calibration is deliberately conservative, but its threshold can be adjusted for a more lenient trade-off between trusted and alerted inputs.The authors describe this cautious setting as suitable for forensic triage, where unreliable decisions are more costly than additional review.Source: §IV · p. 4

A. Preventing critical deepfake acceptance

The monitor reduces silent deepfake acceptance by flagging detector errors that fall outside trusted regions, trading automatic coverage for review. For AS19-trained detectors, bona fide rejection remains a separate operational concern.

A. Preventing critical deepfake acceptance

  • For AS19-trained detectors, deepfake FAR is already much lower, while more errors involve bona fide recordings rejected as deepfake.These errors may increase false alarms or unnecessary manual review, rather than silently accepting a deepfake.Source: A. Preventing critical deepfake… · pp. 4–5
  • Below 1% across all settings, uncaught deepfake FAR remains after monitoring, with relative reductions exceeding 98% across all AS5 detectors.Before monitoring, AS5-trained AASIST accepted 59.43% of AS19 deepfakes as bona fide, while AS5-trained MHFA accepted 61.72% of ITW deepfakes.Source: A. Preventing critical deepfake… · p. 4, Table III · p. 5
  • The monitor does not improve detector accuracy; it converts many otherwise silent errors into alerts by reducing automatic coverage.Uncertain samples can be routed to review or rejected, lowering the probability of accepting deepfakes as bona fide.Source: A. Preventing critical deepfake… · p. 5

B. Effect of covariance regularization

Covariance shrinkage stabilizes Mahalanobis distances and reshapes monitor categories, with backend-dependent effects on alerts and uncaught deepfake FAR. It reduces artificial alerts but does not remove genuine dataset novelty.

B. Effect of covariance regularization

  • Shrinkage reduces apparent over-alerting for AASIST, especially when trained on AS19, while high alert rates persist for AS5-trained detectors.The remaining alerts indicate genuine dataset novelty rather than only a covariance artifact.Source: B. Effect of covariance regularization · p. 5, Table IV · p. 5
  • Regularization often increases MHFA alert rates while reducing uncaught deepfake FAR in most settings; its effect is backend-dependent.Empirical covariance can be poorly conditioned for highly anisotropic embeddings, allowing low-variance directions to destabilize distances and trigger excessive alerts.Source: B. Effect of covariance regularization · p. 5
  • For AS5 spoof samples, automatic assignment to the known-spoof region rises from 62% to 92% after regularization.For AS19-trained AASIST, this increased handling comes with a small increase in bona fide rejection.Source: B. Effect of covariance regularization · p. 5, Table IV · p. 5

V. DISCUSSION AND CONCLUSION

The monitor adds an unsupervised reliability layer to fixed deepfake speech detectors, flagging inputs outside familiar training conditions. It reduces silent deepfake acceptance but trades automatic coverage for alerts, and the study is limited to offline evaluation.

V. DISCUSSION AND CONCLUSION

  • The monitor substantially reduces silent deepfake acceptance by alerting many recordings that the detector would falsely accept as bona fide.This reduces uncaught deepfake FAR by reducing automatic coverage, not by making the detector itself more accurate.Source: §V · p. 5
  • Fig. 3 shows normalized category assignments for a specific evaluation subset by dataset and class, whereas Table IV reports aggregate alert and FAR rates per detector.Source: Figure 3 · p. 6
  • Drift affects both bona fide and deepfake recordings, with codec, channel, or recording-condition changes contributing to bona fide drift and high alerts even in some in-domain settings.The discussion also identifies shrinkage as stabilizing Mahalanobis estimates across backends at the cost of geometric specificity.Source: §V · p. 6
  • The monitor identifies samples not to trust automatically but does not determine detector updates, unknown-data labels, or retraining; experiments are limited to offline benchmark evaluation.The authors call for future study of online monitoring with human-review workflows.Source: §V · p. 6
Loading 2610.16826v1…