Source-linked AI summary
Deep Learning for Deepfakes Creation and Detection: A Survey
Thanh Thi Nguyen, Quoc Viet Hung Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, Thien Huynh-The, Saeid Nahavandi, Thanh Tam Nguyen, Quoc-Viet Pham, Cuong M. Nguyen
TL;DR
Photorealistic deepfakes can undermine trust in digital visual evidence, while detection methods face limited generalization across forgeries and datasets. This survey synthesizes deepfake creation and detection methods, reviews challenges and research directions, and provides an overview to support more effective detection research.
Problem
Increasingly photorealistic deepfakes challenge the authentication of digital visual evidence, creating a need for reliable detection and contextual assessment.
Method
The paper surveys deepfake creation and detection algorithms and synthesizes challenges, research trends, and future directions.
Results
The survey provides a timely, comprehensive overview of deepfake techniques and state-of-the-art detection methods, alongside reported challenges and research directions.
Takeaways & Limitations
The synthesis is intended to support development of effective methods for tackling increasingly accessible and rapidly disseminated deepfakes.
Takeaways & Limitations
Most detection models are evaluated in same-forgery, in-dataset settings, leaving cross-forgery and cross-dataset generalization insufficiently addressed.
Abstract
from arXiv · showhide
Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is deepfake. Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity of digital visual media is therefore indispensable. This paper presents a survey of algorithms used to create deepfakes and, more importantly, methods proposed to detect deepfakes in the literature to date. We present extensive discussions on challenges, research trends and directions related to deepfake technologies. By reviewing the background of deepfakes and state-of-the-art deepfake detection methods, this study provides a comprehensive overview of deepfake techniques and facilitates the development of new and more robust methods to deal with the increasingly challenging deepfakes.
1. Introduction
Deepfakes are AI-synthesized visual content encompassing faceswap, lip-sync, and puppet-master techniques, commonly created with deep learning models such as autoencoders and GANs. Because they enable both beneficial creative applications and malicious manipulation, the paper surveys deepfake creation, detection methods, challenges, and research directions.
- Deepfake Definitions: Deepfakes broadly denote AI-synthesized content, including faceswap, lip-sync, and puppet-master categories.Faceswap superimposes a target person’s face onto a source person’s video.
- Deepfake Creation: Autoencoders and generative adversarial networks commonly examine facial expressions and movements to synthesize analogous facial images.Deepfake creation normally requires large amounts of image and video data.
- Benefits and Risks: Deepfakes can intensify political or religious tensions, influence elections, disrupt financial markets, and mislead military analysts with fabricated satellite imagery.A cited example is a nonexistent bridge generated across a river to mislead troops in battle.
- Benefits and Risks: Deepfakes can support visual effects, digital avatars, voice restoration, film updating, photography, video games, virtual reality, and entertainment.Examples include realistic dubbing of foreign films and creating voices for people who have lost theirs.
- Detection Motivation: The difficulty of establishing digital truth has increased as deepfakes become widely accessible, prompting numerous deep learning-based detection methods and a conflict between malicious and positive uses.DARPA’s MediFor scheme and the Deepfake Detection Challenge were launched to accelerate detection and prevention research.
- Survey Scope: The survey organizes detection research into fake image and face video detection, distinguishing handcrafted versus deep features and temporal versus within-frame visual artifacts.It also discusses challenges, research trends, and directions in deepfake detection.
2. Deepfake Creation
Deepfake creation primarily uses deep-learning architectures that reconstruct or synthesize faces through encoder–decoder models and GANs. These methods support feature transfer, adversarially improved realism, and controllable image synthesis through style representations.
- Encoder–decoder deepfakes: Deep autoencoders underpin many deepfake applications because they represent complex, high-dimensional data and are accessible to users with varied computer skills.The applications’ ease of use spans professional to novice users.
- Encoder–decoder deepfakes: Two encoder–decoder pairs can share an encoder while using different decoders to transfer facial features from one face and reconstruct another.The process connects face A’s feature set to decoder B, producing face B with face A’s mouth shape; examples include DeepFaceLab, DFaker, and DeepFake tf.
- GAN-based deepfakes: Faceswap-GAN improves encoder–decoder deepfakes by adding adversarial and VGGFace perceptual losses.The perceptual loss makes eye movements more realistic and consistent with input faces while smoothing segmentation-mask artifacts, leading to higher-quality output videos.
- GAN-based deepfakes: GAN-based creation trains a generator to produce images resembling real data while a discriminator learns to distinguish generated images from real ones.After sufficient training, the generator produces highly similar images and the discriminator becomes highly capable of distinguishing fake from real images.
- StyleGAN: StyleGAN enables controllable face synthesis by mapping latent code z through an intermediate space and modifying styles at different image scales.It can mix two latent codes so different styles control different portions of the generated image.
3. Deepfake Detection
Deepfake detection is surveyed as a binary classification task addressing increasingly realistic image and video manipulations. Methods are organized into fake-image detection and fake-video detection, with video approaches examining either frame-level artifacts or temporal features across frames.
- Detection Framework: Deepfake detection typically classifies authentic versus tampered videos and requires large databases of real and fake videos for training.The limited availability of fake videos remains a challenge for benchmarking detection methods.
- Detection Framework: The survey groups methods into fake-image detection and fake-video detection, with video methods using frame-level visual artifacts or temporal features across frames.Temporal-feature methods commonly use recurrent deep-learning classifiers.
- Detection Framework: Detection has progressed from handcrafted artifact features to deep learning models that automatically extract salient and discriminative features.This shift responds to deepfakes’ increasing risks to privacy, society security, and democracy.
- Fake Image Detection: Image detectors include preprocessing, statistical testing, compact handcrafted features, Siamese CFFN representations, and self-consistency of local source features.The CFFN extracts discriminative fake-versus-real features using pairwise information, while local source features test whether image regions share consistent origins.
- Fake Video Detection: Image-based methods do not generally transfer directly to videos because compression strongly degrades frame data and videos contain varied temporal characteristics.This motivates detectors that explicitly model motion and temporal information.
- Fake Video Detection: Optical-flow features capture unnatural motion patterns in inserted lips, eyes, or faces and are fed into a CNN to discriminate synthetic from pristine videos.Optical flow is calculated between temporally distinct frames to describe object movement, which is expected to differ between synthetic and naturally generated frames.
4. Discussions and Future Research Directions
Deepfakes are becoming easier to create, more photorealistic, and potentially harmful even when distributed only to targeted audiences. Future research should improve generalized detection, establish updated benchmarks, integrate screening into platforms, and incorporate social context and forensic expertise.
- Threats and Impact: Targeted deepfake distribution can influence important decision-makers without reaching a massive audience, creating risks for privacy, democracy, and national security.Malicious creators may deliver deepfakes directly to target audiences, including politicians, as part of sabotage strategies.
- Detection Challenges: Detection methods must improve as deepfake quality increases, supported by growing, updated benchmark datasets rather than fragmented evaluation datasets.Current detection methods remain at an early stage and are often evaluated using fragmented datasets.
- Detection Challenges: Cross-forgery and cross-dataset generalization remains essential because same-forgery and in-dataset evaluations do not ensure performance on unknown real-world manipulations.A model trained on one specific forgery should work against another unknown forgery, while existing methods often exploit generation-pipeline weaknesses that may be unavailable.
- Platform Responses: Detection should be integrated into social-media distribution platforms through screening, filtering, rapid removal requirements, and watermarking tools.Platform-based mechanisms could help address the widespread impact of deepfakes, while legal requirements could require technology companies to remove them quickly.
- Social Context: Assessing publishers’ intent and social context is crucial because detection software may lag behind increasingly photorealistic deepfake creation.Relevant context includes who distributed the deepfake and what they said about it.
- Forensic Authentication: Machine-learning-based media modification challenges the ability of digital-forensics experts to authenticate photographs and videos used as evidence in investigations and courts.Experts’ opinions may no longer be sufficient by themselves to authenticate altered digital contents.
5. Conclusions
Deepfakes are eroding trust in media and can harm targeted individuals, amplify disinformation and hate speech, and intensify political tension, violence, or war. Their growing accessibility and rapid spread through social media underscore the survey’s timely overview of creation and detection methods, challenges, trends, and future directions.
- Deepfakes erode trust in media because seeing content no longer guarantees belief in its authenticity.
- Deepfakes can distress targeted individuals, heighten disinformation and hate speech, and stimulate political tension, public violence, or war.
- Increasingly approachable creation technologies and rapid social-media distribution make deepfakes especially critical today, motivating this survey’s overview of methods, challenges, trends, and future directions.