PaperlayerView exact revision on arXiv

Source-linked AI summary

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

Felix Juefei-Xu, Run Wang, Yihao Huang, Qing Guo, Lei Ma, Yang Liu

arXiv:2103.00218v3cs.CV

TL;DR

DeepFake generation and detection research has grown rapidly, while the field’s battleground and detection-evasion direction remain insufficiently surveyed. This paper synthesizes more than 318 studies across generation, detection, and evasion, using taxonomies and interaction analyses to characterize the evolving landscape. It identifies quality-dependent detection difficulty and the open challenge of remaining effective against future generation techniques.

  • Problem

    The interactions between DeepFake generators, detectors, and evasion methods remain unclear in recent surveys despite rapid growth in related research.

  • Method

    The paper surveys more than 318 studies, categorizes generation and detection methods, and analyzes their technical evolution and battleground interactions.

  • Results

    The survey presents a taxonomy and detailed battleground analysis, including interactions between DeepFake generation methods and detection methods.

  • Takeaways & Limitations

    The battleground perspective provides analysis of DeepFake research trends, challenges, opportunities, and future directions.

  • Takeaways & Limitations

    Detection performance is highly dependent on DeepFake quality, with high-quality examples remaining difficult for detectors to determine.

Abstract

from arXiv · show

The creation or manipulation of facial appearance through deep generative approaches, known as DeepFake, have achieved significant progress and promoted a wide range of benign and malicious applications, e.g., visual effect assistance in movie and misinformation generation by faking famous persons. The evil side of this new technique poses another popular study, i.e., DeepFake detection aiming to identify the fake faces from the real ones. With the rapid development of the DeepFake-related studies in the community, both sides have formed the relationship of battleground, pushing the improvements of each other and inspiring new directions, e.g., the evasion of DeepFake detection. Nevertheless, the overview of such battleground and the new direction is unclear and neglected by recent surveys due to the rapid increase of related publications, limiting the in-depth understanding of the tendency and future works. To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 318 research papers carefully surveyed. We present the taxonomy of various DeepFake generation methods and the categorization of various DeepFake detection methods, and more importantly, we showcase the battleground between the two parties with detailed interactions between the adversaries (DeepFake generation) and the defenders (DeepFake detection). The battleground allows fresh perspective into the latest landscape of the DeepFake research and can provide valuable analysis towards the research challenges and opportunities as well as research trends and future directions. We also elaborately design interactive diagrams (http://www.xujuefei.com/dfsurvey) to allow researchers to explore their own interests on popular DeepFake generators or detectors.

1 Introduction

DeepFakes have become easier to produce and can support both benign applications and malicious misinformation, prompting policy, platform, and detection responses. This survey organizes generation, detection, and evasion research while analyzing their adversarial battleground and future directions.

  • DeepFake emergence: Deep generative models enable realistic synthetic faces and facial manipulations such as identity swaps.The survey defines DeepFakes as deep-generative creation or manipulation of facial attributes, identity, or expression.
  • Scope and taxonomy: The survey covers four DeepFake categories: entire face synthesis, attribute manipulation, identity swap, and expression swap.Attribute manipulation excludes identity and expression from its definition of facial attributes.
  • Applications and risks: DeepFakes can support benign uses but may also enable misinformation, election interference, and other real-world harms.The paper emphasizes that consequences depend on the content and intent of the generated media.
  • Research growth: The field has expanded rapidly, with around 78% of collected papers appearing in the last two years.The paper attributes this trend to growing research interest in DeepFakes.
  • Survey contribution: The survey presents interactive, detailed analyses intended to clarify the evolving landscape and inform research challenges, opportunities, and horizons.Its diagrams and tables support method lookup, comparison, and exploration of generator–detector relationships.
  • Survey contribution: The paper surveys more than 318 studies across generation, detection, and evasion, emphasizing technical evolution and interactions between adversaries and defenders.Its battleground analysis uses large-scale visualizations to identify trends, major battles, rising approaches, and obsolete nodes.

2 Paper Collection and Review Schema

The survey focuses on facial DeepFakes and reviews generation, detection, and detection-evasion research using a collection of 318 papers assembled from repositories, databases, arXiv, and manual searches. Its collection results show uneven publication distributions across these topics.

  • The survey technically examines facial DeepFake generation, detection, and evasion, while excluding broader social, ethical, and non-facial media-manipulation topics.
  • The authors combine a GitHub repository, keyword searches in Google Scholar, DBLP, and arXiv, manual reviews of recent top-tier publications, and references from detection papers.
  • 318 papers were collected from Google Scholar and arXiv, with venues categorized as top conferences and journals, Others, or arXiv.
  • Two-thirds of generation papers appeared in top conferences and journals, whereas top-tier detection papers formed only a small share of detection publications.
  • Across DeepFake research, top conferences and journals accounted for one-third of papers, and CVPR represented about two-thirds of those top papers.

3 Generation of DeepFakes

The generation survey first covers broad deep image-generation and manipulation techniques, then concentrates on facial appearance-related DeepFake methods. It organizes the latter into four functional categories and traces their technological evolution alongside datasets and challenges.

  • Generation Categories: Facial DeepFake generation is organized into entire face synthesis, attribute manipulation, identity swap, and expression swap according to task requirements.
  • Datasets and Challenges: The generation review also covers real and fake image/video datasets and DeepFake challenges before summarizing the methods.
  • Overview of Deep Image Generation and Manipulation Methods: The survey distinguishes broad techniques such as style transfer, inpainting, rendering, super resolution, fusion, and de-identification from its primary focus on facial appearance manipulation.
  • Generation Categories: The survey details each generation category’s technological evolution using intuitive descriptions and categorizations of optimization methods.

3.2 Entire Face Synthesis

Entire face synthesis generates nonexistent faces from random vectors, with GAN-based methods dominating because they produce more realistic faces than VAEs. Its evolution emphasizes training stability, resolution, and controllable facial attributes, while retaining notable limitations.

  • Definition and Foundations: Entire face synthesis generates a nonexistent fake face from a random vector through a neural network.
  • Definition and Foundations: GANs are the mainstream baseline for entire face synthesis, while VAEs generally produce less realistic, blurrier faces.
  • Technical Evolution: GAN-based synthesis maps a random distribution to the human-face distribution, enabling stable high-resolution generation through improved networks and training procedures.
  • Limitations: Current synthesis methods still face training difficulty, including mode collapse, and realism gaps associated with limited general knowledge of facial structure.
  • Research Focus: Existing methods mainly improve training stability, image resolution, and controllable facial attributes while generating faces that are difficult to distinguish from real images.
  • Technical Evolution: The field evolved from GAN advances addressing unstable training and artifacts toward high-resolution and style-guided generation.

3.3 Attribute Manipulation

Attribute manipulation, or face editing, changes selected facial properties while preserving other regions. Its methods progressed from latent-space encoder-decoder pipelines toward multi-domain, attention-based, text-guided, and more precise editing, but remain constrained by training labels.

  • Limitations: Current methods remain limited by training-dataset labels, making attributes absent from those labels difficult to control.
  • Definition and Scope: Face editing spans simple attributes such as hair color and smiles as well as complex properties including gender and age.
  • Technical Evolution: Later methods pursue multi-domain transformation, accurate attribute control, preservation of unchanged regions, and new representations of modified attributes.
  • Technical Evolution: TediGAN maps images and text into a common embedding space, enabling interactive editing of different facial attributes from textual descriptions.
  • Definition and Scope: Attribute manipulation modifies facial properties of an input face and is commonly implemented with encoder-decoder architectures and GANs.
  • Technical Evolution: The foundational IcGAN pipeline encodes real images into latent space, changes property-specific latent codes, and decodes them into fake faces.

3.4 Identity Swap

Identity swap replaces a source face’s identity with a target identity, using autoencoder- or GAN-based pipelines and progressively improving realism, resolution, fidelity, and heterogeneous-domain handling.

  • Definition and general pipeline: Identity swap replaces the source identity with a target identity through a neural network that generates a fake image.The formal pipeline is xf = φ(xs,ti).
  • Autoencoder-based methods: Autoencoder-based methods use one shared encoder and two independent decoders trained on source and target identities.The encoder extracts shared features, while separate decoders reconstruct the respective identities.
  • GAN-based methods: GAN-based methods are mainstream in identity swap, with CycleGAN enabling unpaired image translation through forward and inverse mappings constrained by cycle consistency.Faceswap-GAN extends the approach with adversarial and perceptual losses.
  • Technical evolution: Identity-swap research has addressed subject-agnostic manipulation, joint pose-expression-identity adjustment, high-fidelity occlusion awareness, and appearance mapping.Examples include FSGAN, FaceShifter, and AOT.
  • Technical evolution: Later methods target limitations in resolution and domain heterogeneity by improving feature representations, attribute control, and refinement across materials such as oil paintings and 3D cartoons.MegaFS uses hierarchical encoding and a face transfer module, while FaceInpainter uses staged inpainting and refinement.
  • Technical evolution: Overall, identity-swap evolution emphasizes separating and fusing identity-related and attribute-related features while improving realism, resolution, and heterogeneous-domain performance.The survey highlights autoencoder methods, GAN-based methods, MegaFS, and FaceInpainter as representative milestones.

3.5 Expression Swap

Expression swap, also called face reenactment, changes a source face’s expression while preserving identity. Its evolution emphasizes controllable expressions, diverse inputs, audio guidance, and temporal consistency for videos.

  • Definition and scope: Expression swap replaces the source expression with a target expression, producing a fake image through a neural network.The formal pipeline is xf = φ(xs,te).
  • Method families: Expression-swap methods include image-based reenactment, audio-driven mouth synthesis, controllable facial animation, and one-shot talking-head generation.Representative methods include Face2Face, A2V, ICface, and Wang et al.’s talking-head approach.
  • Cross-modal synthesis: Audio-driven approaches initially emphasized lip motion, while later methods additionally model head pose, eye blinks, phonetics, context, and identity information.PC-AVS adds a pose source video, and FACIAL-GAN models implicit facial relationships.
  • Detection relationship: The survey treats expression swap as an extension of identity swap because most detectors are not specifically designed for expression-swap detection.The stated reason is its technical similarity to identity swap and limited dedicated detection methods.
  • Technical evolution: Action units provide an identity-independent and flexible representation for controllable expression synthesis, addressing the difficulty of disentangling expression and pose from identity.ICface represents emotions through activations of 17 facial muscles.
  • Video synthesis: Video expression swap is more difficult than image swapping because facial appearance must remain consistent across frames.SVGAN uses appearance contrastive loss and temporal structure loss to impose video-level constraints.

3.6 Real Dataset

Real face datasets support supervised DeepFake-detector training and provide sources for constructing fake datasets. They vary substantially in scale, annotation richness, identity coverage, and visual diversity.

  • Role of real datasets: Real datasets are required for supervised training of DeepFake detectors, while independent real video datasets remain infrequent.Many real face videos used to create fake datasets come from YouTube or actor recordings.
  • Representative datasets: CASIA-WebFace contains 10,575 subjects and 494,414 images, providing a large and diverse Internet-collected face set.Its collection process is semiautomatic.
  • Representative datasets: CelebA contains 200,000 images across 10,000 identities, annotated with 40 facial attributes and five key points.Its abundant attribute labels make it useful for attribute manipulation.
  • Representative datasets: VGGFace contains 2.6 million images of 2,622 identities, while MegaFace covers more than 690K individuals with one million photos.MegaFace evaluates recognition under many distractor identities.
  • Representative datasets: VGGFace2 contains 3.31 million images of 9,131 subjects with variation in pose, age, illumination, ethnicity, and profession.The images were harvested from the Internet.
  • Representative datasets: FFHQ provides 70,000 high-quality 1024 × 1024 face images with broad variation in age, ethnicity, backgrounds, and accessories.Its variation exceeds that of CelebA-HQ in the cited dimensions.

3.7 Fake Dataset

Fake datasets benchmark DeepFake generation and detection, but early collections often lacked diversity, wild conditions, or multiple manipulated people. Later datasets expand scale, perturbations, real-world content, and annotations.

  • Dataset role and limitations: Fake image and video datasets benchmark DeepFake-generation methods and detection performance as synthetic quality and fidelity improve.The survey organizes popular fake datasets by release date.
  • Early datasets: Early datasets include UADFV with 98 videos, DeepFake-TIMIT with 620 videos, and FaceForensics++ with 1,000 manipulated video sequences.FaceForensics++ covers DeepFakes, Face2Face, FaceSwap, and NeuralTextures.
  • Large-scale datasets: DFDC provides 128,514 face-swap videos and over 100,000 clips sourced from 3,426 actors, with distractor and augmenter augmentations.It was constructed for the DeepFake Detection Challenge.
  • Large-scale datasets: DeeperForensics-1.0 contains 60,000 videos and 17.6 million frames, including 50,000 originals and 10,000 manipulated videos.Its actors cover four typical skin tones, with 55 males and 45 females.
  • Dataset role and limitations: Dataset diversity is limited when videos use few actors, scenes, and popular software, whereas wild DeepFakes include varied scenes, multiple people, and combinations of software.Wild content also differs in processing quality and face-region characteristics.
  • Expanded coverage: ForgeryNet contains 2.9 million images and 221,247 videos spanning four forgery tasks, 15 manipulation approaches, and more than 36 mix-perturbations across 5,400 subjects.FFIW10K and OpenForensics specifically address multi-person forgery detection.
  • Dataset usage: FaceForensics++ is the most commonly detected dataset for facial-appearance swapping, while PGGAN is the most commonly detected technique for entire-face synthesis.DFDC has the highest Elo rating score, and FaceForensics and FaceForensics++ have the highest citation counts.

3.8 DeepFake Challenges

The DFDC and DeeperForensics challenges provide large-scale evaluations of DeepFake detectors, revealing the importance of backbone selection and data augmentation.

  • The DFDC and DeeperForensics challenges used datasets containing 100,000 videos and 100,000,000 frames.
  • 2,114 teams participated in DFDC, whose private-test evaluation required processing 10,000 videos within 90 hours.
  • DeeperForensics attracted 115 registered participants, with 25 teams submitting valid entries evaluated on 3,000 private-test videos.
  • Top-3 DeeperForensics solutions achieved promising performance, with two using EfficientNet backbones and all using training augmentation.
  • The challenges identify backbone selection and appropriate data augmentation as key factors for building powerful forgery detectors.

3.9 Summary of DeepFake Generation Methods

DeepFake generation spans multiple manipulation categories and increasingly realistic outputs, while the survey organizes methods, datasets, and battleground-based rankings. It uses Elo rating to compare generation difficulty through detector performance, but notes continuing room for improvement.

  • DeepFake generation covers entire face synthesis, attribute manipulation, identity swap, expression swap, and other methods.
  • Generation methods still have substantial room for improvement in image resolution, manipulable face properties, and video continuity.
  • The survey compiles four tables covering generation methods, real datasets, and fake datasets, including method information and dataset characteristics.
  • Recent generation methods produce facial images that are immensely hard for human perception to distinguish from real ones.
  • The survey reports top-three generation methods for each DeepFake type, including DCGAN, GDWCT, StyleGAN2, StarGAN, AttGAN, and FaceForensics++.
  • Elo rating ranks generation methods by detection difficulty using detector accuracy or AUC, complementing battleground-based rankings.

4 Detection of DeepFakes

DeepFake detection uses spatial, frequency, biological, and other cues to distinguish synthetic from natural media, with practicality requiring generalization, attack robustness, and explainability. Spatial methods remain popular but face robustness limitations, especially under adversarial noise and transformations.

  • The survey organizes DeepFake detectors by spatial, frequency, biological, and other extracted features, evaluating generalization, attack robustness, and explainability.
  • Spatial-domain detection is popular because visible and invisible artifacts can distinguish real and fake faces.
  • Traditional forensic methods provide explainable pixel-level clues but suffer robustness problems after simple image or video transformations.
  • DNN-based detectors use spatial features to improve effectiveness and generalization, but studies report vulnerability to additive-noise adversarial attacks.
  • Some detectors seek transferability across unseen manipulations, including ResNet-50 trained on PGGAN images and OC-FakeDect trained with real-face data.
  • SSTNet combines spatial, steganalysis, and temporal features using XceptionNet and recurrent modeling.

4.2 Frequency based Detection

Frequency-based detection exploits spectral artifacts and frequency-domain features introduced by GAN generation, often improving generalization to unknown synthesis methods. However, robustness to common image degradations remains a practical limitation.

  • Frequency-based detectors exploit GAN spectral artifacts or frequency-domain features of real and fake faces.
  • GAN upsampling can introduce replicated spectral patterns that classifiers use to distinguish synthesized faces.
  • GAN fingerprints can identify synthesized images and their sources, but blur and JPEG compression can destroy them.
  • Frequency-domain classifiers can achieve promising results and, in some studies, remain robust to blurring, cropping, and future unseen GANs.
  • Frequency-based methods generally offer higher generalization ability than spatial approaches by exploiting DeepFake frequency artifacts.
  • These methods remain insufficiently robust to degradations such as compression and reconstruction, limiting deployment in real-world settings.

4.3 Biological Signal based Detection

Biological-signal detection exploits physiological cues that remain difficult for current DeepFake methods to replicate, complementing visual, audio, and other detection approaches. The survey identifies robustness, future realism, and explainability as continuing challenges.

  • Other detection cues: Audio-visual inconsistency and mismatches between mouth dynamics and spoken phonemes provide additional clues for detecting manipulated videos.A Siamese network models visual-audio similarity and perceived-emotion cues, while lip-sync methods examine speech–mouth-shape inconsistencies.
  • Biological signals: Heart-rate-based detectors use Neural-ODE prediction, motion-magnified spatial-temporal representations, or remote photoplethysmography to expose fake videos.DeepRhythm highlights heart-rhythm signals with MMSTR and a dual-spatial-temporal attentional network, while DeepFakesON-Phys uses rPPG to observe subtle skin-color changes.
  • Biological signals: Biological signals provide detection cues because they are broken or unrealistic in synthesized videos and are not easily replicated by current DeepFake techniques.The survey discusses eye blinking, facial landmarks, fixed face sizes, heart rhythms, blood flow, and related physiological indicators.
  • Survey findings: Existing detectors commonly use CNN backbones and ACC or AUC, but many do not test unseen DeepFakes, transformations, or explainable evidence.The survey also reports that still images are easier to detect than videos and that public video datasets dominate evaluation because fake-image datasets are scarce.
  • Challenges and outlook: Future detectors should withstand degradations, capture subtle real–fake differences, and investigate longer-lasting clues as synthesis quality improves.The survey warns that fake textures may be corrupted or removed and that spatial and frequency artifacts may become difficult to detect.

5 Battleground

DeepFake generators and detectors form a battleground whose mapped interactions reveal which methods and datasets are most widely evaluated. The survey uses linked diagrams and comparative analyses to characterize this landscape, while noting uneven modality coverage and difficulty identifying seminal detectors.

  • Battleground mapping: The battleground links 83 DeepFake generation methods with 117 detection methods to show which generators and detectors attract the most evaluations.In the Sankey representation, node out-degree indicates popularity; FaceForensics++ is evaluated by many detection methods.
  • Popular generators and datasets: FaceForensics++, Celeb-DF, DFDC, PGGAN, and StarGAN are among the prominent generation methods or datasets identified through battleground topology.Figure 12 reports the top-ranked generation methods or datasets overall and for 2020 alone.
  • Modality coverage: Only 10 of 117 surveyed detectors were evaluated on both image and video modalities, while most experiments targeted DeepFake images.The survey attributes this imbalance partly to scarce or more challenging video datasets.
  • Baseline comparisons: XceptionNet is the most popular baseline, and more than one-third of studies compare against it.Other frequently used baselines include Afchar et al., Nguyen et al., ResNet, and Yang et al.; XceptionNet, ResNet, and VGG are the leading CNN baselines.
  • Research visibility: Citation popularity does not cleanly identify seminal detection papers because the field is young and its current research is multi-threaded.The survey describes this ambiguity as double-edged: milestone works may need time to emerge, while their absence may indicate distributed progress.

6 Evasion of DeepFake Detection

DeepFake-detection evasion modifies real or fake faces so state-of-the-art detectors misclassify them. The surveyed approaches use adversarial attacks, frequency-domain trace removal, or general image reconstruction and filtering.

  • Overview: Evasion methods map real or fake faces to new images that cannot be correctly classified by state-of-the-art DeepFake detectors.The survey groups these methods into three broad types and illustrates an exemplar pipeline in Figure 20.
  • Adversarial attacks: Adversarial perturbations can significantly reduce detector accuracy under both white-box and black-box attacks.The surveyed attacks include fast gradient sign and C&W methods; adversarial noise inevitably reduces face-image quality.
  • Adversarial attacks: Universal adversarial perturbations improve transferability when detector preprocessing and face-detection differences make direct attacks difficult on DFDC.Neekhara et al. study these challenges on the more difficult DeepFake Detection Challenge dataset.
  • Frequency-domain methods: Frequency-domain methods target spectral artifacts caused by generative up-sampling and narrow the gap between synthetic and natural frequency distributions.Spectral regularization and focal frequency loss dynamically emphasize frequency components that are difficult to synthesize.
  • Reconstruction and filtering: Shallow reconstruction, advanced filtering, and generative models treat evasion as image generation intended to mislead detectors.Huang et al. demonstrate detector evasion through sparse-coding and dictionary-based shallow reconstruction.

7 Horizon

The survey identifies major weaknesses in current DeepFake datasets and detectors, then outlines future directions for more realistic, evasive, robust, and forward-looking systems. It also envisions evolving datasets that support both standardized benchmarks and rapidly updated task-specific evaluation.

  • Research landscape: More than 91 generation and dataset papers and more than 117 detection papers reveal challenges for creating realistic DeepFakes and defending against them.The survey links improved generation methods with continued advances in detection.
  • Generation and datasets: Current fake datasets emphasize scale and content diversity but inadequately represent video-quality variation, diverse generation methods, and organic video-level degradations.DeeperForensics-1.0 adds several perturbations, but these are artificially applied at the image level rather than arising organically in video.
  • Detection challenges: Existing detectors often rely on simple baselines, are evaluated mainly on easy datasets, and remain vulnerable to unknown DeepFakes, degradations, occlusions, and high-quality forgeries.Low-quality datasets with visible artifacts are easy to classify, whereas high-quality datasets such as Celeb-DF and DFDC are harder for detectors.
  • Detection challenges: More than 90% detection accuracy on common DeepFakes falls below 70% for the winning DFDC submission, exposing a gap between academic and challenging real-world evaluation.The survey notes that existing challenge platforms and datasets do not yet provide a sufficient evaluation setting.
  • Evasion: Future evasion research should replace unrealistic additive perturbations with natural degradations and remove fake traces while preserving realistic image and video appearance.Suggested attack factors include blur, lighting, weather, exposure, and shadows.
  • Epilogue and the Next Chapter: The survey anticipates a two-part dataset ecosystem: large standardized benchmarks for general-purpose models and smaller, frequently updated datasets for emerging generators and long-tail scenarios.New datasets can support specialized evaluation and finetuning, while older datasets retain value through scale and diversity.

8 Conclusion

The survey analyzes DeepFake generation, detection, and detection evasion across more than 318 papers. It organizes generation and detection methods while emphasizing the evolving battleground between adversaries and defenders.

  • 8 Conclusion: More than 318 research papers are surveyed across DeepFake generation, detection, and detection evasion.The paper provides a comprehensive overview and detailed analysis of all three areas.
  • 8 Conclusion: The paper presents taxonomies of generation and detection methods together with analyses of their technical evolution.Its synthesis covers the changing methods on both sides of the battleground.
  • 8 Conclusion: The survey highlights the battleground between DeepFake generators and detectors as a perspective on research challenges, opportunities, trends, and future directions.The analysis emphasizes interactions between adversaries and defenders.
Loading 2103.00218v3…