PaperlayerView exact revision on arXiv

Source-linked AI summary

"What I See is What I Hear": Deepfake Detection Across Diverse Hearing Abilities

Magdalena Pasternak, Malvika Jadhav, Palavi V. Bhole, Aviva Smith, Elaina Trapatsos, Vincent Bindschaedler, Roshan Peiris, Ersin Uzun, Patrick Traynor, Matthew Wright, Kevin R. B. Butler

arXiv:2609.28659v1cs.CR

TL;DR

Audiovisual deepfakes create an authenticity challenge for viewers whose access to auditory and visual evidence differs, while research has largely overlooked DHH populations. In an in-person mixed-methods study, we compare hearing profiles and manipulation types, finding lower overall accuracy for DHH participants and distinct susceptibility patterns across groups.

  • Problem

    Audiovisual deepfakes exploit human perceptual limits, while DHH users may encounter content through access layers that preserve, omit, or add information relevant to judging authenticity.

    Source: §I · p. 1

  • Method

    An in-person mixed-methods study compared deepfake judgments across hearing profiles using talking-head clips manipulated through voice conversion, text-to-speech, face-swap, or lip-sync.

    Source: §I · p. 2, B. Stimuli · p. 5

  • Results

    DHH participants reached 76.4% overall accuracy versus 88.0% for HPs, and susceptibility varied by hearing profile and manipulation method.

    Source: §I · p. 2, §VIII · p. 14

  • Takeaways & Limitations

    Deepfake threats and defenses should be evaluated across access profiles rather than solely against a default user.

    Source: §VIII · p. 14

    Provenance alone cannot establish whether a caption or interpretation faithfully represents its source, and warnings must be accessible across modalities.

    Source: F. Defenses Must Compensate Perceptual… · p. 13

Abstract

from arXiv · show

The proliferation of audiovisual deepfakes has lowered the cost of fraud, impersonation, and misinformation, but their success ultimately depends on human perception. Detection requires integrating auditory and visual cues, yet security and privacy research has largely overlooked d/Deaf and hard-of-hearing (DHH) populations. We address this gap with an in-person, mixed-methods study of 80 participants: 31 hearing persons (HPs), 15 hard-of-hearing (HoH) participants, 17 d/Deaf participants, and 17 cochlear implant (CI) users. Each participant judged the authenticity of 30 clips, where manipulations spanned text-to-speech, voice conversion, lip-sync, or face-swap. DHH participants were less accurate than HPs overall (76.4% vs. 88.0%, p<.001), primarily because they more often classified authentic clips as manipulated (FPR: 29.7% vs. 11.2%). Differences depended strongly on the manipulated channel. For audio-only manipulations, HoH participants matched HPs (90.0% vs. 90.3%), followed by CI users (79.4%) and d/Deaf participants (41.2%). When clips contained an audiovisual manipulation, accuracy clustered between 84% and 87%, although performance still varied by manipulation method. Our work systematically characterizes how deepfakes affect DHH populations, highlighting the asymmetric risks audiovisual manipulations may pose to groups with different hearing abilities and the need for accessible, tailored defenses that support all users.

I. INTRODUCTION

Audiovisual deepfakes create a human-centered security problem, yet prior human-detection research largely assumed a default viewer and overlooked how hearing ability shapes susceptibility. This study compares hearing profiles and manipulation types, finding distinct performance gaps and cue-use patterns.

I. INTRODUCTION

  • Deepfakes can enable fraud and extortion, erode public trust, and polarize communities as generative tools become cheaper and easier to use.Automated detectors face dataset-coverage and generalization limits, while provenance standards depend on broad adoption and fact-checking may arrive after initial exposure.Source: §I · p. 1
  • Prior human deepfake-detection work largely assumed a default viewer, despite hearing loss affecting more than 430 million people worldwide.For DHH users, audiovisual access may combine residual hearing, speech-reading, assistive technology, captions, and interpretation, each of which can preserve, omit, or add information.Source: §I · p. 1
  • DHH participants reached 76.4% overall accuracy versus 88.0% for HPs, with the gap driven primarily by rejecting authentic clips (TNR 70.3% vs. 88.8%).All DHH subgroups performed below HPs; subgroup accuracy ranged from 73.1% to 78.6%.Source: §I · p. 2
  • For audio-only manipulations, HoH participants matched HPs (90.0% vs. 90.3%), while d/Deaf participants reached 41.2%; CI users scored 90.8% on face-swaps and 77.5% on lip-syncs.The reported patterns varied by both hearing profile and manipulation method.Source: §I · p. 2
  • Participants’ cue use reflected sensory access: d/Deaf participants focused on mouths, whereas CI users attended more to facial geometry and shape.The authors argue that defenses and detection guidance should account for access profiles rather than assume viewers can inspect the same cues.Source: §I · p. 2

C. Human Detection

Human judgments depend on perceptual cues, context, and accessible evidence, while DHH users vary in hearing, language, and communication practices. The threat model treats that variation as differences in accessible evidence and evaluates judgments under controlled conditions.

C. Human Detection

  • Human deepfake detection draws on visual and auditory cues, heuristics, and context, while authentic evidence can also be dismissed as fake through the liar’s dividend.Warnings about manipulation may increase judgments that authentic clips are fake without improving deepfake detection.Source: C. Human Detection · pp. 2–3

D. DHH Users

  • DHH users’ available evidence depends on auditory access as well as language background, literacy, communication practices, and accessibility mechanisms.Research on human detection has rarely tested populations with systematically different sensory access.Source: D. DHH Users · p. 3

1) Hearing and Assistive Devices:

  • HoH users may receive degraded audio, d/Deaf users little or none without a device, and CI users reconstructed audio that can alter pitch and prosody.These auditory differences matter alongside lip reading, contextual cues, and other ways of supplementing sound.Source: 1) Hearing and Assistive Devices · p. 3

2) Signed Language and Literacy:

  • DHH users vary in whether they primarily communicate through signed, spoken, or written language, and deaf readers show lower average English reading proficiency than hearing peers.The passage argues that security-relevant text should accommodate a wide range of literacy levels.Source: 2) Signed Language and Literacy · p. 3

3) Mediated Access:

  • Captions selectively preserve and transform information; transcript-only speech detection approaches chance and improves when audio is added.Captions can omit acoustic artifacts that reveal manipulation and compete with visual information for attention.Source: 3) Mediated Access · p. 3

III. THREAT MODEL

  • The study’s access profile represents which auditory and visual information a recipient can access and use, including cue reliability, communication modality, technology, and delivery layer.Profiles are not ranked by ability: reduced access in one channel may increase reliance and scrutiny of another.Source: §III · p. 3
  • The adversary may seek false acceptance of manipulated media or false rejection of authentic media; false acceptance is the primary threat in the model.The model distinguishes perceptual deception from evidentiary absence, where diagnostic cues lie in a channel the target cannot reliably inspect.Source: §III · p. 4
  • The threat model assumes access to source material and generation tools, and that an adversary can infer a target’s population-level access profile and select manipulations accordingly.The study evaluates four manipulation families under audio-only, visual-only, and audiovisual conditions, but does not implement optimized or individually targeted attacks.Source: §III · p. 4

IV. DESIGN AND METHODOLOGY

The study used an in-person, mixed-methods design to compare judgments across hearing profiles and manipulation channels. Its controlled stimuli and shared task provided a baseline for comparing responses to the same audiovisual signal.

IV. DESIGN AND METHODOLOGY

  • The 80-participant study collected 2,400 trial-level observations and 62.5 hours of interview data using real and manipulated clips.Manipulations included audio-only, visual-only, and combined audiovisual conditions.Source: §IV · p. 4

1) Recruitment and Eligibility:

  • Recruitment ran for ten months, and DHH subgroup recruitment ended when coded justifications reached thematic saturation; all participants who began completed the study.Sessions were in person, allowing enrollment criteria including CI use to be verified.Source: 1) Recruitment and Eligibility · p. 4

2) Grouping and Classification:

  • Participants were assigned to HP, HoH, d/Deaf, or CI groups using self-reported identity, hearing loss, and assistive-device use; CI users were grouped as CI regardless of identity.Among participants without CIs, self-reported hearing-loss degree determined d/Deaf or HoH assignment.Source: 2) Grouping and Classification · p. 4

3) Sample Composition:

  • The cohort included 31 HPs and 49 DHH participants: 15 HoH, 17 d/Deaf, and 17 CI users.The authors describe the 49-person DHH cohort as among the largest in-person security-research studies with this population.Source: 3) Sample Composition · pp. 4–5

4) Accessibility Accommodations:

  • ASL interpreters supported consent, instructions, and researcher communication when requested, but did not interpret the stimuli.Captions were not provided, so all participants received the same unmediated audiovisual signal as a controlled comparison baseline.Source: 4) Accessibility Accommodations · p. 5

B. Stimuli

  • The talking-head stimuli used four manipulation families—voice conversion, text-to-speech, face-swap, and lip-sync—to vary which identity or behavior cues were altered.Real videos came from HDTF, and independently varying visual and audio components produced real, visual-only, audio-only, and audiovisual conditions.Source: B. Stimuli · p. 5, 1) Dataset · p. 5, 3) Manipulation Grid · p. 5

2) Standardization:

  • Stimuli were standardized for framing, resolution, frame and audio rates, loudness, encoding, and two clip lengths to reduce unintended presentation variability.Each participant saw a randomized set of 30 clips, made real/fake judgments, rated confidence, described cues, and completed a post-study questionnaire.Source: 2) Standardization · p. 5, Figure 1 · p. 5

4) Generator Selection and Experimental Scope:

The experiment selected locally deployable manipulation generators, filtered stimuli for plausibility, and distributed clip sets across participants. In the overall comparison, HP accuracy exceeded DHH accuracy, with all DHH subgroups below HPs.

4) Generator Selection and Experimental Scope:

  • The study selected one locally deployable open-source implementation per manipulation family, prioritizing plausible pipeline-compatible outputs and reproducibility.Commercial tools can change without notice or impose controls; the authors do not claim their selected implementations are the most prevalent in practice.Source: 4) Generator Selection and… · pp. 5–6

5) Quality and Piloting:

  • The team generated over 1,000 candidate clips, removed obvious artifacts, and retained only clips judged perceptually plausible in a 10-person internal pilot.Reviewers checked for failures including audio glitches, incomplete facial replacement, flicker, and audio-lip desynchronization.Source: 5) Quality and Piloting · p. 6

C. Exposure Design

  • Each participant judged 15 authentic and 15 manipulated clips, including 2 audio-only, 4 visual-only, and 9 combined audiovisual manipulations.The 30-trial cap balanced manipulation coverage with participant burden and sessions of about one hour.Source: C. Exposure Design · p. 6

1) Within-participant Constraints:

  • Within each clip set, source-speaker identities and utterances appeared only once, preventing direct comparison of authentic and manipulated versions.Presentation order was randomized independently for each participant to mitigate ordering effects.Source: 1) Within-participant Constraints · p. 6

2) Matrix Construction:

  • Ten clip sets distributed 300 clips across sessions, with each DHH participant receiving a set previously assigned to an HP for matched comparisons.The analysis included clip-specific random intercepts to account for residual differences in clip difficulty.Source: 2) Matrix Construction · p. 6

D. Study Procedure

  • After each clip, participants classified it as real or fake, rated confidence on a five-point scale, and described their cues; replays were permitted.They could not return to earlier trials, and explanations followed the judgment to avoid overlapping speech with stimulus audio.Source: D. Study Procedure · p. 6

E. Ethics

  • The study had IRB approval and informed consent; compensation increased from $25 to $50 because recruitment was slower than anticipated.Source: E. Ethics · p. 6

A. Overall Detection Performance

  • HPs achieved 88.0% accuracy, compared with 76.4% for DHH participants; all DHH subgroups performed significantly below HPs.The d/Deaf subgroup had the lowest accuracy at 73.1%, and only the CI-versus-d/Deaf comparison was significant within DHH groups.Source: A. Overall Detection Performance · p. 7

1) Error Decomposition:

Detection varied by hearing profile and manipulation channel: the overall DHH–HP gap was driven mainly by false alarms on authentic clips, while performance differed across audio, visual, and audiovisual manipulations.

1) Error Decomposition:

  • 76.4% vs. 88.0% overall accuracy (DHH vs. HP) reflected false alarms on authentic clips, with all DHH subgroups below HPs in TNR.DHH TNR was 70.3% versus 88.8% for HPs; subgroup TNRs were 66.7% for d/Deaf, 71.1% for HoH, and 73.3% for CI users.Source: 1) Error Decomposition · p. 7, Figure 2 · p. 7

2) Confidence Calibration:

  • DHH calibration error was roughly four times HPs’ (ECE=0.100 vs. 0.022), concentrated in d/Deaf participants (ECE=0.139).Confidence was a weaker signal of correctness for DHH participants.Source: 2) Confidence Calibration · p. 7

1) Performance by Manipulation Channel:

  • 41.2% of audio-only fakes were detected by d/Deaf participants, versus 90.3% for HPs; accuracy was similar across groups for visual-only and audiovisual manipulations.For audiovisual manipulations, accuracy ranged from 84.3% to 87.1% across groups; visual-only group differences were not significant.Source: 1) Performance by Manipulation Channel · pp. 7–8, Figure 3 · p. 8

2) Performance by Manipulation Type:

  • Across groups, face-swaps (88.6%) and TTS (87.8%) were detected more accurately than VC (81.5%), while lip-sync accuracy was also 81.5%.Face-swaps were easier than lip-sync; other reported contrasts were not significant.Source: 2) Performance by Manipulation Type · p. 8, Figure 4 · p. 8
  • Audio-method results varied by group: HoH reached 93.3% on TTS, while d/Deaf participants reached 74.5% on VC versus 84.4% for HPs.For visual methods, CI users scored 90.8% on face-swaps and 77.5% on lip-syncs; the within-group gap was significant, while between-group differences were not.Source: 2) Performance by Manipulation Type · p. 8

C. High-Confidence Errors

DHH participants made high-confidence errors more often than HPs, and the study’s interview analysis documented group differences in cue use and reported visual evidence.

C. High-Confidence Errors

  • 17.7% vs. 7.4% of judgments were high-confidence errors for DHH participants and HPs, respectively; the largest false-negative gap occurred for d/Deaf participants on audio-only fakes.On audio-only manipulations, 68.2% of d/Deaf trials were confident false negatives, compared with 3.6% for HPs.Source: C. High-Confidence Errors · pp. 8–9

1) Corpus and Analysis:

  • Researchers used inductive latent thematic analysis and a multi-label cue codebook, organizing labels into 13 themes across audio, visual, and heuristic domains.The corpus contained 7,476 cue occurrences, averaging 2.9 cues per trial.Source: 1) Corpus and Analysis · p. 9

1) Cue Output:

  • DHH participants cited more cues per trial than HPs (3.02 vs. 2.66), with HoH and CI users driving the difference; d/Deaf participants cited fewer (2.38).Motion and cross-modal synchrony were the most-cited themes overall, while d/Deaf participants cited synchrony less often and visual degradation more often.Source: 1) Cue Output · p. 9, 2) Theme Frequency · p. 9, Figure 6 · p. 9

2) Theme Frequency:

Participants described different visual and contextual cues across hearing profiles, and cue accuracy depended on whether a clip was real or manipulated.

2) Theme Frequency:

  • DHH participants relied on background cues nearly three times as often as HPs (17.6% vs. 5.9%) and offered more detailed scene-based rationales.The authors caution that most theme-frequency group comparisons were not statistically significant given sample sizes.Source: 2) Theme Frequency · pp. 9–10

C. Theme Uses and Accuracy

  • Compositing-boundary and temporal-stability cues yielded 97.7% and 97.0% accuracy on manipulated clips, but only 36.1% and 46.6% on real clips.These patterns held for HP and DHH groups.Source: C. Theme Uses and Accuracy · p. 10
  • Heuristics supported real judgments at 92.7% accuracy but performed at 39.9% on manipulated clips; cross-modal synchrony was accurate on both real and manipulated clips.Cross-modal synchrony accuracy was 85.9% on real clips and 87.1% on manipulated clips, and it was frequently used for both.Source: C. Theme Uses and Accuracy · pp. 10–11, Figure 7 · p. 10

D. Detection by Manipulation Channel

  • The largest cross-group accuracy gap occurred for audio-only fakes, where access to the manipulated channel varied most; explanations also reflected qualified or limited auditory access.The paper describes the gap as consistent with differences in which evidence participants could access.Source: 1) Audio · p. 11, D. Detection by Manipulation Channel · p. 11

1) Audio:

  • Audio-realism cues appeared in 4.1% of d/Deaf trials versus 22.2% of HoH trials, and d/Deaf accuracy when using them was 57.6%.Audio-realism reliability was lower for VC than TTS (62.8% vs. 88.6%); cross-modal synchrony was reliable for both and cited more often on TTS trials.Source: 1) Audio · p. 11

2) Visual:

Visual manipulations produced similar overall accuracy across hearing groups, but participants prioritized different visual cues. The discussion connects these patterns to hearing-profile-specific attack surfaces and channel access.

2) Visual:

  • All groups invoked motion at broadly similar rates for visual manipulations, but d/Deaf participants emphasized visual degradation and compositing boundaries, while CI users prioritized whole-face realism.HoH participants concentrated more on eye-related cues, which yielded 68% accuracy.Source: 2) Visual · p. 11

E. Confidence and Effort

  • DHH participants expressed lower confidence and sought additional cues; their higher cue counts and false-positive rates may be related to that effort.The authors phrase this relationship as a possibility, not a demonstrated cause.Source: E. Confidence and Effort · p. 12

VII. DISCUSSION

  • The authors frame audiovisual deepfake risk as depending on hearing profile, manipulated channel, and manipulation type, and present defense requirements as proposals rather than validated interventions.Source: §VII · p. 12

A. Hearing Profile Creates Distinct Attack Surfaces

  • The discussion interprets d/Deaf participants’ mouth-focused cues and CI users’ attention to facial identity as possible explanations for descriptive differences between lip-sync and face-swap detection.The group-by-family interaction was not statistically significant, so these within-group contrasts do not establish different manipulation-family effects across hearing profiles.Source: A. Hearing Profile Creates Distinct… · p. 12

B. Channel Selection and Evidentiary Absence

C. Accessibility Layers Transform Available Evidence

Deepfake detection varies with hearing profile, manipulation method, and the perceptual cues viewers can access. The discussion connects these differences to sharing risks, false alarms, accessible defenses, and study limitations.

F. Defenses Must Compensate Perceptual Limitations

C. Accessibility Layers Transform Available Evidence

  • Accessibility layers may distort or omit evidence, but captioning and interpretation could also surface speaker identity, audio quality, or voice anomalies.The authors suggest annotations such as “[distorted voice]” as a possible complement to system-level verification.Source: C. Accessibility Layers Transform… · p. 13

D. Sharing Across Access

  • Sharing clips across access and language boundaries may expose DHH users to blame for errors or discourage them from sharing and engaging.Recipients may interpret a share as endorsement even when the sharer lacked access to the clip’s audio evidence.Source: D. Sharing Across Access · p. 13

E. Excessive Skepticism is a Security Risk

  • DHH participants rejected authentic clips more often than hearing participants, and longer trials, replays, and explanations suggest deliberate evaluation.The study did not directly observe resulting harms, but the authors warn that false alarms could undermine trust in VRS, VRI, telehealth, and captioned video.Source: E. Excessive Skepticism is a Security… · p. 13

G. Limitations

  • Small local subgroup samples limit within-DHH contrasts, and one generator per manipulation family leaves manipulation and generator effects potentially mixed.The authors treat subgroup effects as exploratory and note that newer models or stimuli could produce different behavior.Source: G. Limitations · p. 14
  • Controlled viewing omitted real-world context and accessible renderings, while post-decision explanations may reflect rationalization rather than contemporaneous reasoning.The study therefore cannot determine how captions or interpretation affect detection; its main between-group comparisons use matched trials to control clip difficulty.Source: G. Limitations · p. 14

VIII. CONCLUSION

  • DHH detection depends on where a manipulation occurs and which perceptual cues it targets, so aggregate accuracy can obscure subgroup-specific attack risks.The authors caution that findings based predominantly on hearing participants do not generalize to DHH populations.Source: §VIII · p. 14

IX. ETHICAL CONSIDERATIONS

The study weighs the defensive value of documenting hearing-profile risks against possible misuse, while describing safeguards, stakeholder impacts, and the researchers’ positionality.

Ethical Considerations and Impact

  • The study received minimal-risk IRB approval, used informed consent and screened clips, and retained de-identified data on restricted institutional servers after deleting original recordings.Participants could withdraw, and the team avoided publishing individually identifying cues or clip-level difficulty rankings.Source: Ethical Considerations and Impact · pp. 14–15
  • The authors identify broader DHH communities as potential beneficiaries and risks, since findings may inform inclusive defenses but could also help attackers select manipulations by hearing profile.They report results at a granularity intended to support defense design without revealing individual targeting profiles.Source: Stakeholders · p. 14, Ethical Considerations and Impact · p. 15
  • In-person recruitment limited participation to adults able to commute to the study site, and compensation increased from $25 to $50 without retroactive adjustment.The authors included commuting costs in the participation fee and describe accessibility options including interpreters, an ASL-proficient researcher, and text input.Source: Ethical Considerations and Impact · p. 15
  • The authors frame publication as a dual-use trade-off: results may aid attack selection, but withholding them would leave defenders without evidence to address existing access-related asymmetries.They argue that mediated access layers require system-level defenses rather than asking users to inspect inaccessible evidence or exercise greater caution.Source: Ethical Considerations and Impact · p. 15
  • The authors identify themselves as hearing researchers without DHH or CI insider status, and distinguish Deaf cultural identity from deaf audiological status in their terminology.They report drawing on prior community collaboration and delivering study materials in participants’ preferred modalities.Source: Ethical Considerations and Impact · p. 16

APPENDIX APPENDIX A STUDY PROTOCOL & INSTRUMENTATION

The appendix reproduces the study protocol, from pre-task questions and accessible instructions through clip judgments and post-task questions.

APPENDIX APPENDIX A STUDY PROTOCOL & INSTRUMENTATION

  • The protocol comprised informed consent, a pre-task questionnaire, the detection task, and a post-task questionnaire.The appendix reproduces each instrument verbatim.Source: Appendix A · p. 18

A. Pre-Task Questionnaire

  • Before the task, participants reported online media use, prior deepfake exposure, estimated prevalence of manipulated media, and concern about synthetic media.The questionnaire asked about daily viewing time, whether participants had encountered a deepfake, their prevalence estimate, and their concern.Source: A. Pre-Task Questionnaire · p. 18

B. Task and Instructions

  • Participants watched 30 clips, could replay them, judged each real or fake, rated confidence, and explained their cues; captions were off for everyone.Instructions defined “deepfake” as an umbrella term for altered or synthetic media, and participants viewed 15 authentic and 15 manipulated clips without being told the ratio.Source: B. Task and Instructions · pp. 18–19

1) Post-task questionnaire:

  • After the final trial, participants repeated the concern item, reported policy and assistive-technology views, and provided demographic information.The post-task instrument also asked about hearing loss and assistive devices, with follow-up questions for participants reporting either.Source: 1) Post-task questionnaire · p. 19

APPENDIX B DEMOGRAPHICS AND DEEPFAKE DETECTION

The appendix describes the sample and examines background associations with detection outcomes using observational analyses.

APPENDIX B DEMOGRAPHICS AND DEEPFAKE DETECTION

  • Background variables were examined against accuracy, error rates, confidence, calibration, sensitivity, and criterion using scale-appropriate association and regression models.The authors characterize these analyses as observational and contextual, not causal.Source: Appendix B · p. 19

A. Demographics

  • Assumed deepfake prevalence was the only background item that differed between cohorts, with DHH participants reporting higher estimates; concern rose descriptively, but no group-level change was significant.The prevalence difference was significant (Mann–Whitney p = .002; Holm-adjusted p = .021).Source: A. Demographics · p. 19, 4) Prior exposure and assumed prevalence · p. 19

1) Age, gender, education:

  • Most participants were 18–24 years old (61.3% HP, 61.2% DHH); DHH participants spanned older age brackets more broadly, and the sample included 40 men, 39 women, and one non-binary HP.Education categories were reported as selected levels, not necessarily completed degrees.Source: 1) Age, gender, education · p. 19

2) Hearing loss, communication, and assistive technology:

  • HoH participants predominantly reported moderate hearing loss, 14 of 17 d/Deaf participants reported profound bilateral loss, and 16 of 17 CI users reported severe-to-profound loss before implantation.All d/Deaf participants used ASL in the session, and every DHH participant selected captions as an assistive technology they use.Source: 2) Hearing loss, communication, and… · p. 19

3) Media consumption.:

  • Most participants reported viewing online media for 3–4 hours daily (16 HP, 24 DHH), with no significant difference between cohorts.The next most common category was more than six hours daily (4 HP, 9 DHH).Source: 3) Media consumption · p. 19

4) Prior exposure and assumed prevalence:

  • Prior deepfake exposure did not differ between cohorts, but DHH participants estimated a higher prevalence of manipulated media (mean 3.31–3.46 across DHH subgroups versus 2.58 HP).The reported higher prevalence did not translate into greater detection accuracy; DHH participants classified 56.1% of trials as fake versus 49.1% for HPs.Source: 4) Prior exposure and assumed prevalence · pp. 19–20, Figure 9 · p. 20

5) Pre- to post-concern shift:

The appendix reports exploratory associations and patterns in concern, response tendencies, clip judgments, replay behavior, and confidence calibration. DHH participants’ response and clip-level patterns varied from those of HPs, while replaying was associated with lower accuracy.

5) Pre- to post-concern shift:

  • Concern rose by 0.22 points among DHH participants and 0.13 among HPs, but neither within-group change nor the difference between cohorts was statistically reliable.Across participants, mean concern increased by 0.19 points; 48 participants’ concern was unchanged, 21 increased, and 11 decreased.Source: 5) Pre- to post-concern shift · p. 20

B. Impact on Detection

  • Within DHH participants, prior deepfake exposure (p = .027) and age (ρ = −0.305, p = .033) were associated with accuracy at the p < .05 threshold; no HP background item reached it.The authors treat the remaining background analysis as exploratory.Source: B. Impact on Detection · p. 20

A. Response-Level Distribution

  • 56.1% of DHH trials were classified as fake versus 49.1% of HP trials; the DHH rate exceeded the balanced 50% distribution (p = .003), and the cohort difference was significant (OR = 1.32, p = .004).Rates were similar across DHH subgroups: HoH 56.4%, d/Deaf 56.5%, and CI users 55.3%.Source: A. Response-Level Distribution · p. 20

B. Clip Judgments

  • Median per-clip accuracy was 81.7% for real clips and 87.5% for manipulated clips; only 4 of the 30 hardest clips overlapped between HP and DHH rankings, versus 17 of the easiest.HPs had at least 80% accuracy while DHH participants were below 60% on 29 clips; the reverse occurred on 7 clips.Source: B. Clip Judgments · p. 20

1) Manipulated Clips:

  • Participants correctly classified 13 of 70 lip-sync clips, while face-swap clips were classified correctly in 31 of 60 cases; voice conversion was the only class with no clip correctly classified by every viewer.Unanimous false acceptance occurred for 15 of 20 lip-sync-with-voice-conversion clips, 5 of 10 voice-conversion-only clips, and 2 of 10 text-to-speech-only clips.Source: 1) Manipulated Clips · p. 20

C. Replay Behavior

  • Accuracy declined from 93.2% on trials viewed once to 64.8% after four or more replays; DHH participants replayed more often on average than HPs.Mean replay counts were 2.92 for HoH, 3.00 for d/Deaf, and 3.05 for CI users, versus 2.60 for HPs; Figure 10 presents clip views and accuracy across hearing profiles.Source: Figure 10 · p. 21, C. Replay Behavior · p. 21
  • For audio-only manipulations, replay counts were 1.31 for HPs, 1.63 for HoH participants, 1.97 for CI users, and 2.56 for d/Deaf participants.The passage describes this replay pattern as consistent with repeated inspection of accessible visual cues when manipulation evidence was confined to audio.Source: C. Replay Behavior · p. 21

D. Confidence Calibration Mapping

  • Figure 11 reports calibration against confidence: HPs shifted from underconfident toward calibrated as confidence rose, while DHH subgroups remained moderately overconfident.The study mapped the five-point confidence scale to probabilities from 0.5 to 1.0 and used Brier score and Expected Calibration Error.Source: Figure 11 · p. 21, D. Confidence Calibration Mapping · p. 21

E. GLMM Measures

Detection patterns differed by hearing group and manipulation channel; the coded explanations also showed strong annotator agreement, substantial cue coverage, and distinct group patterns in cue use. The thematic framework spans visual and cross-modal cues as well as technical, intuitive, and contextual judgments.

E. GLMM Measures

  • χ2(9) = 45.5, p < .001 indicated channel-level patterns differed across groups; d/Deaf accuracy was 41.2% on audio-only, 85.0% on audiovisual, and 86.8% on visual-only clips.HoH participants scored 90.0% on audio-only fakes versus 90.3% for HPs, but 76.7% on visual-only fakes; HP and CI performance was relatively stable across manipulation families.Source: 1) GLMM Across All Video Types · p. 21
  • The group × channel interaction was significant (χ2(6) = 35.08, p < .001); protected simple-effect contrasts compared HPs with each DHH subgroup within each channel.The trial-level GLMM modeled correct response with hearing group × manipulation channel as fixed effects and crossed random intercepts for participant and stimulus.Source: E. GLMM Measures · p. 21

B. Annotator Agreement

  • γAC1 = 0.997 and κ = 0.74 across the full response-cue matrix; mean Dice and Jaccard overlap across 2,429 coded responses were 0.73 and 0.67.Agreement remained high when empty-set true negatives were excluded.Source: B. Annotator Agreement · p. 22

C. Saturation Analysis

  • The cue-saturation test required every subset of size n to cover at least 90% of pooled cues; average coverage reached 95% at n = 31 and worst-case coverage at n = 48.Worst-case group thresholds were roughly two-thirds of each group's size, which the authors interpret as supporting stable within-group cue-frequency contrasts.Source: C. Saturation Analysis · p. 22

D. Cue Output

  • Words per trial were statistically indistinguishable between HPs and DHH participants: 51.0 versus 54.5 (p = .61).The passage reports no significant difference for any DHH subgroup against HPs either.Source: D. Cue Output · p. 22
  • DHH participants discussed 3.02 cues per trial versus 2.66 for HPs (p < .001); HoH cited 3.68, d/Deaf 2.38, and CI users 3.08.d/Deaf participants cited significantly fewer cues than HPs, while HoH and CI users cited significantly more.Source: D. Cue Output · pp. 22–23
  • Across 30 trials, DHH participants cited 53.6 distinct cue codes versus 44.4 for HPs; HoH and CI users drove the gap, while d/Deaf participants matched HPs.The authors suggest this asymmetry may reflect manipulation cues being more salient and lexically diverse than judgments of naturalness, as well as artifact-seeking prompted by the task framing.Source: D. Cue Output · p. 23

E. Theme Structure

  • Nine of 13 themes were visual, while the framework also covered cross-modal synchrony, technical flaws, intuitive judgments, and contextual familiarity.Participants also used poor video quality as a detection cue, which the authors note could be problematic as deepfake quality improves; Table II organizes the reported cues by perceptual domain.Source: Table II · p. 22, A. Developed Themes · p. 22, E. Theme Structure · p. 23

Related papers

This paper cites

Loading 2609.28659v1…