Warning: Humans Cannot Reliably Detect Speech Deepfakes

• Existing research centers on machine-learning detection for voice authentication, while studies of human detection capabilities are sparse.Source: Introduction · p. 2
• Speech deepfakes threaten uses beyond biometric authentication, including misleading people through voice impersonation.Source: Introduction · p. 2
• The experiment measured how well people distinguish bona fide from synthesized speech and tested whether language or familiarization affected detection.Source: Introduction · p. 3
• Listeners had limited speech-deepfake detection capabilities, performance was similar between English and Mandarin, and familiarization improved performance only slightly.Source: Introduction · p. 3

Deepfake media

• Deepfakes are generative-AI outputs that resemble humans and their actions, and speech deepfakes imitate a genuine person’s voice.Source: Deepfake media · pp. 3–4

Synthesizing speech

• A common speech-synthesis framework collects and preprocesses speaker recordings, trains a generative model, and may use text-to-speech modules with transcriptions.Source: Synthesizing speech · pp. 4–5

Related work on human deepfake detection capabilities

• For images, multiple studies find humans perform not much better than chance, and video evaluation shows no improvement; intervention effectiveness is debatable, while neither tested intervention improved performance.Source: Related work on human deepfake… · pp. 5–6
• Human-detection studies are fewer for speech than visual deepfakes; prior speech studies vary in sample size, task setting, language, and participant confidence.Source: Related work on human deepfake… · p. 6

• The paper frames its investigation around a set of research questions.Source: Materials and methods · p. 8

Bona fide stimuli

• The English and Mandarin experiments used bona fide speech from LJSpeech and CSMSC, respectively, with one female speaker reading generic sentences to limit external cues.Source: Bona fide stimuli · p. 8

Deepfake stimuli

• Deepfakes were generated with pretrained VITS models from 50 validation sentences per language, matched to the bona fide sentences for 100 clips total.Source: Deepfake stimuli · pp. 8–9

Procedure

• Participants were randomly assigned to unary or binary tasks, rated confidence, gave freeform justifications, could replay clips, and received no correctness feedback.Source: Procedure · p. 9

Unary

• In the unary task, participants judged 20 randomly selected clips as real or fake without being told the real/fake proportion.Source: Unary · p. 10

Binary

• In the binary task, participants identified the fake in 20 pairs containing the same sentence spoken by a human and by VITS.Source: Binary · p. 10

Familiarization treatment

• The familiarization group heard five separate deepfake examples before the task, while controls completed a filler task about synthetic speech.Source: Familiarization treatment · p. 11

Participants

• The study recruited 529 participants through Prolific and obtained informed written consent after notifying participants about the study purpose.Source: Participants · p. 11, Ethics statement · p. 12

Benchmarking against automated deepfake detectors

• Two LFCC-LCNN neural networks served as automated-detector benchmarks, with in-domain models trained on dataset training splits and VITS-generated samples.Source: Benchmarking against automated… · p. 12

Out-of-domain evaluation

• The English-language evaluation used the pre-trained ASVspoof network, while the Mandarin evaluation used a Mandarin-language dataset.The passage does not specify further details about the Mandarin detector.Source: §2 · p. 12
• Automated networks often lose performance on examples unlike their training data, even within the same language, where differences such as speaker identity or environment may be subtle to listeners.The study introduces an out-of-domain comparison because participants are unlikely to recognize the LJSpeech and CSMSC speaker identities.Source: §2 · p. 12

Overall performance

• In the unary task, overall accuracy was 70.35%, with 73% correct deepfake identification and 67.78% correct bona fide identification; in the binary scenario, participants recognized deepfake audio in 85.59% of trials.The authors speculate that awareness of deepfakes from the task briefing partly contributed to misclassified bona fide samples.Source: Overall performance · pp. 13–14

Measuring the effects of interventions

• Confidence-scaled accuracy weights each correct or incorrect decision by reported confidence, and linear regression assessed intervention and clip-duration effects.A confidence rating of 7 maps to 88%; a correct decision scores 0.88 and an incorrect one 0.12.Source: Measuring the effects of interventions · pp. 14–15

Reference audio helps with deepfake detection

• The binary scenario significantly improved performance (p < 0.001), suggesting reference audio helps listeners uncover quirks in synthesized speech.Source: Reference audio helps with deepfake… · p. 16

Training humans to detect deepfakes only helps slightly

• Familiarization increased detection accuracy by 3.84% on average; reported accuracy improvements also included 3.76% (p = 0.017) and 3.85% (p = 0.032), without specified setting assignments.In the unary setting, accuracy was 52.31% at the mean clip length of 5.76 seconds, ceteris paribus.Source: Training humans to detect deepfakes… · p. 16

It is equally challenging to detect deepfakes in Mandarin and English

• Mandarin-speaking participants outperformed English-speaking participants by 1.79%, a nonsignificant difference (p = 0.202); performance was comparable across treatment groups.Source: It is equally challenging to detect… · p. 16

Shorter speech deepfakes are not easier to identify

• Each additional second of clip length improved accuracy by only 0.80%, with no relationship evident in the scatter plot; this conflicts with Watson et al..Source: Shorter speech deepfakes are not… · p. 16, Figure 4 · p. 17

Listening and time do not aid detection

• Replay count showed no relationship with accuracy, and task completion time did not improve performance (ρ = 0.10, p = 0.018).Source: Listening to the clips more frequently… · p. 17, Figure 5 · p. 17, Spending more time on the task also… · p. 18

Participants do not get better throughout the task without explicit feedback

• Correct responses remained relatively stable across question numbers, indicating participants did not improve during the task without explicit feedback.The authors tested this pattern against a uniform distribution.Source: Participants do not get better… · p. 18, Figure 7 · p. 19

Comparing human performance to automated detectors

• Human AUROC was 73.83% without familiarization and 75.54% with it, while humans misclassified clips about a quarter of the time.The analysis uses unary trials, where both humans and detectors evaluate one clip at a time, and aggregates English and Mandarin results.Source: Comparing human performance to… · pp. 19–20, Figure 8 · p. 20
• In-domain detectors achieved 100% AUROC in both languages, whereas out-of-domain detectors had 25.31% AUROC; humans were described as more robust to unknown factors such as speaker identity.Source: Comparing human performance to… · p. 20
• Crowd responses were on par with the in-domain detector, but familiarization offered no crowd-level benefit: AUROCs were 95.51% without and 94.04% with familiarization.Crowd responses were calculated by averaging participants’ accuracy scores per clip.Source: Comparing human performance to… · p. 20

Freeform text analysis

• Across English and Mandarin, participants cited naturalness, robotic sound, pauses, intonation, pronunciation, and speed, while the attributes mentioned did not substantially differ between correct and incorrect judgments.English participants referenced breathing more; Mandarin participants more often mentioned cadence, pacing between words, and fluency, which the authors suggest may relate to timing differences between the languages.Source: Freeform text analysis · pp. 22–23

Limitations

• The detector-comparison setup may not reflect realistic encounters with speech deepfakes.Source: Limitations · p. 23
• The task presented deepfakes and bona fides equally often, unlike their real-world proportions, and participants may have been unusually attentive after disclosure.The authors note that AI-generated content, including nefarious deepfake use, was still rare at the time.Source: Limitations · p. 23
• The stimuli minimized context, omitting factors such as speaker identity, emotional status, conversation size, and political affiliations that may matter in false-news settings.Source: Limitations · p. 23
• Both language conditions used utterances from one female speaker, although speaker identity may affect perception and warrants further study.Source: Limitations · p. 23
• The deepfake stimuli used an older synthesis approach and may not represent performance against current, more realistic systems.The authors note that the approach was chosen for limited computational resources.Source: Limitations · p. 23

Discussion

• Similarities between English and Mandarin listeners support a consistent threat across languages, while naturalness is used to identify deepfakes regardless of language.The authors expect improved speech naturalness to make detection harder.Source: Discussion · p. 24
• Comparing a clip with reference audio helps in the binary scenario when the speaker’s identity is known, but that information is not always available.The paper describes the creation-detection contest as an arms race.Source: Discussion · p. 24
• Showing examples of deepfake audio has a limited effect on detection, and spending more time evaluating clips does not seem to help.The authors conclude that attempting to improve human detection capabilities is unrealistic.Source: Discussion · p. 24
• Listeners detect speech deepfakes inconsistently, even in a controlled setting where the task is easier and the synthesizers are not state of the art.The discussion says detection is not high even under these conditions.Source: Discussion · p. 24
• Automated detectors can pick up subtleties humans miss on in-domain data, but they are brittle when test audio’s environmental conditions change.The authors call for improving detectors and describe crowdsourcing as a reasonable interim mitigation; extending fact-checking tools to audio is one proposed approach.Source: Discussion · pp. 24–25

Supporting information

• Figures S1–S4 show confidence-adjusted accuracy scores per clip for English, across unary and binary tasks with and without familiarization.Source: Figure S1 · p. 25, Figure S2 · p. 25, Figure S3 · p. 25
• Figures S5–S8 show confidence-adjusted accuracy scores per clip for Mandarin, across unary and binary tasks with and without familiarization.Source: Figure S5 · p. 26, Figure S6 · p. 26, Figure S7 · p. 27