PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep-Fake CAPTCHA: Mitigating Next-Generation Social Engineering Attacks

Guy Frankovits, Lior Yasur, Fred M. Grabovski, Yisroel Mirsky

arXiv:2609.11404v1cs.CRcs.AI

TL;DR

Real-time deepfakes threaten trust in live voice and video communications, while passive detectors struggle with evolving technologies and degraded call signals. DF-CAPTCHA uses human-easy, deepfake-difficult challenges to actively test callers, and the paper reports substantially improved detection across both audio and video. The framework is supported by user studies and experiments, but its scope remains tied to limitations of current real-time deepfake systems.

  • Problem

    Passive deepfake defenses struggle with evolving technologies, evasion through common call processing, novel deepfake types, and varying video quality.

  • Method

    DF-CAPTCHA actively challenges callers with simple human-easy tasks that push real-time deepfake pipelines beyond their stable operating regimes.

  • Results

    DF-CAPTCHA substantially improves deepfake detection performance in both audio and video settings, while StarGANv2-VC fooled 46% of volunteers in one experiment.

  • Takeaways & Limitations

    Active challenge-response verification provides a practical, extensible defense for live communications against real-time deepfake impersonation.

  • Takeaways & Limitations

    The defense exploits limitations of current real-time deepfake pipelines, which generally fail on content outside their training regime.

Abstract

from arXiv · show

This paper presents DF-CAPTCHA, an active defense against real-time deepfake impersonation in voice and video calls. Instead of passively searching for artifacts, DF-CAPTCHA prompts the caller to perform simple challenge-response tasks that are easy for humans but difficult for current real-time deepfake systems to generate convincingly. The framework verifies the response using four criteria: realism, identity consistency, task completion, and response time. We evaluate the approach across both audio and video modalities using user studies and experiments with real-time deepfake models. Results show that people often struggle to distinguish real-time deepfakes from authentic media, while DF-CAPTCHA substantially improves detection performance over passive methods, reaching high accuracy in both modalities. These findings suggest that active challenge-based verification is a practical and robust defense against next-generation social engineering attacks based on real-time deepfakes.

I. INTRODUCTION

Real-time deepfakes make live impersonation attacks increasingly plausible while undermining passive detection methods. DF-CAPTCHA addresses this gap by challenging callers with tasks that expose deepfake limitations.

  • Real-time deepfakes enable convincing face and voice impersonation in live calls, supporting financial fraud, political deception, corporate intrusion, and extortion.
  • Passive detectors face obsolescence as deepfake quality improves and can lose forensic evidence after compression, filtering, or noise addition.
  • DF-CAPTCHA asks callers to perform difficult tasks, flagging responses that are distorted or fail to address the challenge.
  • Live-call degradation and strict latency budgets further weaken heavyweight forensic analysis, motivating defenses that actively engage callers.

II. REAL-TIME DF-CAPTCHA

DF-CAPTCHA converts deepfake detection from passive inspection into active challenge-response verification. The framework extends this defense across audio and video while targeting weaknesses in existing detection pipelines.

  • DF-CAPTCHA actively issues human-easy, deepfake-difficult tasks to force real-time pipelines beyond their operating envelope.The resulting response can expose failures in identity, geometry, temporal coherence, or other observable dimensions.
  • The video extension adds threat analysis, user studies, defensive face challenges, and analysis of structural limitations in face-impersonation pipelines.
  • DF-CAPTCHA substantially improves state-of-the-art detector performance in both audio and video modalities.
  • The approach claims stronger detection guarantees and improved longevity through an extensible challenge set that can adapt as deepfake technology evolves.
  • Existing audio and video detectors rely on staged preprocessing and learned or handcrafted features, but struggle with novel deepfakes and varying media quality.

C. Limitations of Current Defenses

Current defenses passively inspect media, but real-time deepfake systems remain constrained by task scope, representation, optimization, and inference-speed trade-offs.

  • C. Limitations of Current Defenses: Passive detectors analyze content without engaging the caller, leaving them exposed as deepfake quality improves.DF-CAPTCHA instead uses active interaction to probe the deepfake function's limitations.
  • A. RT-DF Limitations: Current RT-DF models generate content only within the narrower tasks represented during training.Models optimized for frontal faces or calm speech generally fail on other poses, sounds, or activities.
  • 1) Technology:: Supporting broader task coverage requires more parameters, richer features, and potentially more complex real-time inference.Higher-resolution STFTs and MFCCs would expand frequency coverage but increase model complexity and training difficulty.
  • 1) Technology:: Multiple objectives for realism, identity, and additional tasks compete during optimization, making convergence harder and potentially degrading some aspects.Current systems commonly use separate realism and identity objectives.

2) Resources:

DF-CAPTCHA turns deepfake detection into randomized challenge-response verification designed around four constraints: realism, identity, task complexity, and real-time execution.

  • 2) Resources:: Broader RT-DF capability remains costly because it requires extensive data, expertise, labeling, and potentially powerful real-time hardware.The authors expect data-collection and training constraints to remain relevant for near-future systems, while noting future technologies may overcome some limitations.
  • 1) Creating a Challenge:: DF-CAPTCHA asks callers to produce content that appears authentic, preserves the target identity, performs a difficult task, and is generated in real time.These constraints are intended to be easy for humans but difficult for current RT-DF systems.
  • 1) Creating a Challenge:: The time constraint keeps the attacker within the real-time setting rather than switching to a more capable offline model.The challenge design explicitly targets live-generation latency.
  • 1) Creating a Challenge:: Two random seeds select a task and then a challenge from that task's challenge set, making requests difficult to anticipate.For example, a task can request humming a randomly selected song.
  • Resources:: Evaluation assumes an RT-DF optimized for regular talking in audio and face video, testing diverse challenges against a specialized system.This setup assesses whether challenges generalize beyond the model's optimized task.
  • 1) Creating a Challenge:: A robust challenge combines a random task and random instance to create difficulty across realism, identity, task complexity, and time.This framework evaluates challenge effectiveness, robustness against bypass or mixing, and performance against naive and advanced attackers.

2) Verifying a Challenge:

DF-CAPTCHA verifies each challenge response independently for realism, identity, task completion, and response time, then accepts it only when all threshold checks pass.

  • 2) Verifying a Challenge:: Realism verification uses anomaly detectors and deepfake classifiers to identify distortions or artifacts in the response.Its output represents the degree of unrealism according to the supplied formulation.
  • 2) Verifying a Challenge:: Identity verification compares a caller sample collected before the challenge with the response using zero-shot recognition.The pre-challenge sample is confirmed by the victim, helping prevent identity switching after the challenge.
  • 2) Verifying a Challenge:: Task verification uses a classifier to check whether the response contains the requested task rather than a different task or no task.This addresses both generation failure and task-substitution evasion.
  • 2) Verifying a Challenge:: Time verification checks whether the first response frame arrives approximately one second after the instructions.The measured delay is included as the time-verification output.
  • 2) Verifying a Challenge:: A response is valid only when the time, realism, identity, and task outputs all remain below their respective thresholds.Exceeding any one threshold makes the response invalid.
  • 2) Verifying a Challenge:: Weighting constraints can adjust false rejects, but the authors warn that this may compromise security and requires balancing usability against protection.Thresholds are tuned separately for the four algorithms.

C. Detection Framework

DF-CAPTCHA screens calls by forwarding them to a challenge system, issuing a randomized task, and verifying the response before connecting the call.

  • C. Detection Framework: At challenge start, the caller states their name, receives instructions and a tone, and then provides a timed response for verification.The name sample is shared with the victim for acknowledgment and identity checking.
  • C. Detection Framework: Multiple challenges may be issued in higher-security settings to reduce the false negative rate.The suggested execution order prioritizes time, realism, task, then identity checks to reduce unnecessary computation.
  • C. Detection Framework: The framework forwards selected calls, generates a challenge, verifies the response, and resumes the call only when all checks pass.Failed checks cause the call to be dropped and evidence to be shown to the user.
  • D. Deployment: DF-CAPTCHA can run on-device or in the cloud and supports autonomous or manual operation across personal and organizational call-screening settings.Examples include call centers and online meeting rooms.
  • D. Deployment: The system is designed for live RT-DF impersonation, not pre-recorded deepfakes, and poorly calibrated challenges may burden legitimate callers.The authors frame this security–convenience trade-off as most acceptable in high-security communications.

V. THREAT ANALYSIS

The threat analysis evaluates real-time deepfakes across audio and video using user studies of current models, testing realism, identity, authenticity, trust, and detection. It covers five voice-cloning models with 41 volunteers and four face-manipulation models with 38 volunteers, using multiple experiments and challenge examples.

  • Study design: 41 audio and 38 video volunteers evaluated five voice-cloning and four face-manipulation RT-DF models in separate user studies.Both studies used university students averaging 25 years old, most outside computer science.
  • Challenge design: The challenge catalogue organizes video and audio tasks by how strongly they constrain adversaries on realism, identity, complexity, and time.Table I presents examples of tasks that can serve as D-CAPTCHA challenges.
  • Audio experiments: The audio study measured realism and identity cloning through quality ratings and comparisons against real voices.Participants rated 90 short clips in EXP1a, then compared real and fake samples against target-identity baselines in EXP1b.
  • Video experiments: The video study tested blind authenticity, informed detection, and perceptual quality across authentic recordings and generated videos.EXP2a measured trust without deepfake warnings, EXP2b measured authenticity judgments after disclosure, and EXP2c compared model quality.

C. Experiment Results

The experiments show that current real-time deepfakes can fool people in both audio and video settings, even when participants actively assess authenticity. Human judgments remain unreliable because participants miss fake media and can overdetect genuine media.

  • Audio results: Roughly 20–50% of volunteers gave RT-DF audio positive quality scores, with StarGANv2-VC achieving the highest quality.The comparison used blind MOS ratings against real audio.
  • Audio results: 46% of volunteers were fooled by StarGANv2-VC audio under conservative conditions in which they were actively expecting deepfakes.The authors caution that this is not a real-world attack-success probability because actual victims may overlook more anomalies under pressure.
  • Audio results: For most cases, volunteers rated the identity capture of the top two audio models well, with normalized scores above −1.Ratings remained mixed overall, with some fake clips judged more authentic and others less authentic.
  • Video results: About half of identities had fake videos perceived as more trustworthy than their genuine versions.Figure 8 reports an approximately zero-centered distribution with nearly 50% positive differences.
  • Video results: 67% recall and approximately 41% false positives show that participants struggled to detect fake videos and misidentified genuine videos when primed.The detection analysis compared MOS distributions for fake and real videos.
  • Video results: Even informed participants could not reliably distinguish authentic videos from real-time deepfakes, making human judgment alone insufficient for defense.The result combines low fake-video detection with high false-positive rates on genuine videos.

D. Takeaways for DF-CAPTCHA

User studies show that real-time deepfakes can appear realistic and identity-consistent across audio and video, making passive perception unreliable. These findings motivate DF-CAPTCHA’s active challenge-response defense across both modalities.

  • 46% of listeners were fooled by StarGANv2-VC despite expecting a deepfake, while video participants often trusted deepfakes as much as or more than real videos.The audio result came from conservative conditions; the video findings also included poor explicit fake detection.
  • Passive human detection is unreliable because deepfakes can receive realistic ratings and resemble the target identity in both audio and video.Audio identity scores were mostly favorable for the top two models, while video participants showed difficulty distinguishing fake from real content.
  • DF-CAPTCHA addresses this threat by issuing human-easy challenges designed to push real-time deepfake pipelines beyond their stable operating regimes.The defense is evaluated in parallel across audio and video to assess whether the active principle generalizes across modalities.
  • Performance is reported with AUC and EER, where higher AUC and lower EER indicate better detection.An AUC of 1.0 represents perfect classification, whereas 0.5 represents random guessing.

A. Experiment Setup

The evaluation combines voice and video corpora with real and deepfake samples, challenge-response data, human studies, and modality-specific verification models. The setup measures realism, identity consistency, and task completion using pretrained or task-trained detectors.

  • The audio corpus includes real speech, StarGANv2-VC voice conversions, and challenge responses across nine human-performed tasks.The audio data include 1,821 RT-DF voice-conversion samples and 3,317 real challenge responses, with deepfakes generated from the challenge data.
  • Train-test identities are kept disjoint, and additional ASVspoof-DF and RITW data train the realism models.The supplied passage specifies the disjoint-identity protocol and the external datasets used for realism-model training.
  • The video corpus contains regular and challenge-response footage from 20 volunteers, totaling 310 minutes and 20.92 GB.The supplied setup identifies the regular-video portion and the overall corpus size.
  • Realism is evaluated with five audio detectors and nine pretrained video detectors drawn from DeepfakeBench.The audio models include SpecRNet, One-Class, GMM-ASVspoof, PC-DARTS, and LOF; the video passage identifies the DeepfakeBench source and pretrained-weight protocol.
  • Task verification classifies whether a response contains the requested task, using per-task GMM classifiers for audio and a fine-tuned ResNet-3D model for video.The audio split is 70–30 and the video split is 75–25.
  • Identity verification compares challenge responses with pre-challenge references using speaker or face embeddings, with audio L2 distance and video cosine similarity.Smaller audio distances and higher video cosine similarities indicate greater identity similarity.

3) Experimental Protocol:

The protocol compares passive and challenge-boosted realism detection, evaluates task and identity verification, and measures the complete DF-CAPTCHA pipeline. Across modalities, challenges generally improve detection by pushing deepfake systems outside their capabilities.

  • The protocol includes realism comparison, task verification, identity verification, and end-to-end evaluation of all components together.Timing is implemented as a binary check for receiving the first response frame within approximately one second and is not separately evaluated.
  • EXP-R: Realism Detection. Active vs. Passive: GMM-ASVspoof achieved audio AUC 0.978 and EER 0.071 on talk & clap, compared with baseline AUC 0.949 and EER 0.122.LOF reached AUC 0.982 on cough from a baseline of 0.678, whereas SpecRNet degraded under several challenge conditions.
  • EXP-R: Realism Detection. Active vs. Passive: Challenges force real-time deepfake pipelines outside their training distributions, amplifying artifacts that are subtle or absent during passive observation.The reported pattern affects most models in both audio and video, supporting active elicitation over passive observation.

C. EXP-C: Task Verification

The experiments evaluate task completion, identity consistency, and the complete DF-CAPTCHA pipeline across audio and video challenges. End-to-end results show substantial gains over passive detection under a fixed false-positive constraint.

  • C. EXP-C: Task Verification: Identity verification varied by task: audio performed best on speaker-preserving challenges, whereas face-occluding video tasks were most difficult.Audio AUCs reached 0.942 for vary volume and 0.926 for vary speed; the supplied video results identify occluding tasks as difficult.
  • E. EXP-E: End-to-End System Performance: 91–100% overall accuracy was achieved across audio tasks, compared with 71% for SpecRNet at FPR = 0.01.Audio TPR ranged from 0.89–1.00, with cough and sing reaching 99–100% accuracy.
  • E. EXP-E: End-to-End System Performance: The full pipeline combines realism, task, and identity components, with thresholds calibrated to FPR = 0.01.The end-to-end evaluation passed 3,317 real responses and 8,758 deepfake responses through the audio pipeline.
  • E. EXP-E: End-to-End System Performance: 20–29 percentage points of audio accuracy improvement over the best passive method were reported, while video improved by 12–25 points.The cross-modal comparison reports a higher audio TPR floor of 0.89 versus 0.73 for video.

VII. DISCUSSION

DF-CAPTCHA improves real-time deepfake detection by actively eliciting behaviors that expose weaknesses in current pipelines. The discussion also identifies scope, usability, and adaptability limitations while supporting extension across audio and video.

  • VII. DISCUSSION: Active challenge-response detection substantially outperforms passive detection and generalizes across audio and video modalities.The conclusion describes the approach as improving detection in both settings and extending the defense from voice cloning to video manipulation.
  • VII. DISCUSSION: The framework targets real-time attacks but does not address pre-recorded deepfake media.This scope boundary is stated alongside the need to replace challenges as generative models improve.
  • VII. DISCUSSION: User friction and false positives remain concerns when challenges are used aggressively or under poor recording conditions.The limitation passage specifically flags low-risk settings and legitimate users performing challenges with degraded recordings.
  • VII. DISCUSSION: Future deployments require an expanding challenge library and evaluation of latency, accessibility, and challenge-selection policies.Future work also proposes improving robustness against adaptive adversaries as deepfake pipelines evolve.
  • B. XAI in Video CAPTCHAs: XAI found stronger fake-related activation and higher fake probabilities in frames containing executed challenges across multiple challenge types.Salient regions typically occurred around challenged facial or motion-related areas.
Loading 2609.11404v1…