Source-linked AI summary
KoDF: A Large-scale Korean DeepFake Detection Dataset
Patrick Kwon, Jaeseong You, Gyuhyeon Nam, Sungwoo Park, Gyeongsu Chae
TL;DR
Deepfake detection needs datasets that better represent varied real-world manipulations and subject distributions. The paper constructs KoDF and evaluates it against existing datasets, finding that combining datasets improves robustness to out-of-domain data. KoDF is therefore presented as complementary to, rather than sufficient instead of, existing resources.
Problem
Existing deepfake datasets may not approximate the true in-the-wild distribution, limiting evidence about real-world generalization.
Method
The paper constructs KoDF with controlled Korean subject distributions, varied synthesis methods, quality assurance, high-resolution clips, and adversarial examples.
Results
Combining KoDF with other datasets makes detection models much more robust against varied out-of-domain data than using individual datasets alone.
Takeaways & Limitations
Deepfake detection benefits from compositing multiple datasets because no standalone dataset published so far achieves sufficient generality.
Takeaways & Limitations
3GDFD is excluded because its real and fake clips have time differences and lack metadata needed to correct them.
Abstract
from arXiv · showhide
A variety of effective face-swap and face-reenactment methods have been publicized in recent years, democratizing the face synthesis technology to a great extent. Videos generated as such have come to be called deepfakes with a negative connotation, for various social problems they have caused. Facing the emerging threat of deepfakes, we have built the Korean DeepFake Detection Dataset (KoDF), a large-scale collection of synthesized and real videos focused on Korean subjects. In this paper, we provide a detailed description of methods used to construct the dataset, experimentally show the discrepancy between the distributions of KoDF and existing deepfake detection datasets, and underline the importance of using multiple datasets for real-world generalization. KoDF is publicly available at https://moneybrain-research.github.io/kodf in its entirety (i.e. real clips, synthesized clips, clips with adversarial attack, and metadata).
1. Introduction
Deepfakes have become easier to create and can support harmful activities, motivating detection research and new datasets. KoDF addresses dataset limitations through large scale, Korean representation, distribution control, and complementary use with existing datasets.
- Deepfakes became easier to produce through deep-learning methods that manipulate facial expressions or swap faces.
- Deepfakes can be used to defame individuals, spread disinformation, or commit fraud despite benign applications.
- KoDF contains 175,776 fake clips and 62,166 real clips from 403 subjects, generated using six synthesis models.
- KoDF focuses mostly on Korean participants and controls distributions across age, sex, and content.
- The authors argue that no established dataset alone approximates the true deepfake distribution, while combining datasets improves detection generality.
2. Related Works
Early deepfake datasets were limited in scale, quality, diversity, or provenance, while later datasets expanded coverage through larger collections and varied synthesis or augmentation methods. Nonetheless, important limitations remained in dataset diversity and consent.
- Early datasets contained no more than 50 identities and fewer than 1,000 real and fake videos, often with unclear provenance or consent.
- FF++ introduced 1,000 real YouTube videos and 4,000 fake videos synthesized by four methods, but its size, diversity, and subject permissions remained insufficient.
- DFDC contained over 960 subjects and more than 120,000 videos, using varied environments and eight synthesis methods.
- Celeb-DF used 590 real and 5,639 fake videos centered on 59 celebrities and one improved face-swap method.
- DF-1.0 added diversity through seven perturbation methods, producing 50,000 real and 10,000 fake clips, while using FF++ target videos without explicit consent.
3.1. Contributions
KoDF is constructed as a large, controlled, quality-assured dataset covering Korean subjects, varied synthesis methods, realistic transformations, and adversarial attacks. Its design emphasizes demographic complementarity and compatibility with future augmentation.
- KoDF contains 62,166 real clips and 175,776 deepfake clips, exceeding DFDC in total duration and clip counts.
- Controlled Subject Distribution: KoDF controls the distribution of 403 participants by age, sex, and recording location.
- Human inspection excludes obvious synthesis failures, retaining clips whose realism makes them difficult for people to classify.
- All real clips come from paid participants who consented to face manipulation and public database use.
- Forward-Looking: KoDF uses full-HD, unaugmented clips to preserve room for researchers’ posterior augmentation of real-world transformations.
- Forward-Looking: The dataset includes extensive face-reenactment outputs because reenactment manipulates actions or expressions using external video or audio.
- Korean subjects complement datasets with underrepresented East and Southeast Asian participants, supporting more generalized detection models when combined.
- KoDF adds adversarial examples to encourage detectors robust against attacks designed to confuse classification models.
3.2. Real Data
KoDF’s real videos were specially recorded and controlled for content, emotion, environment, and participant diversity, then manually inspected for technical defects.
- KoDF records source videos specifically for the dataset, enabling control over recording environment, emotive content, and speech corpus.
- 353 of 403 subjects contributed 150 clips each, including idle, scripted, and scenario recordings with varied filming conditions.Variations included camera angle, focal length, location, background, props, and lighting.
- Each clip belongs to positive, negative, or neutral emotion categories, supporting varied responses while helping mostly amateur participants continue speaking.
- Script clips use ten sentences from a screened Korean-language corpus, organized by sentence type and assigned a common emotion category.Statement-to-question/exclamation occurrence is adjusted to 8:2.
- Scenario clips use proofread questions categorized by three annotators, while participants may also create questions matching the assigned emotion category.
- Fifty subjects completed a separate studio task involving professional recording conditions, repeated runs, and long scripted sessions.
- Real clips were manually checked for synchronization, noise, speech, lighting, and face-position defects, with problematic clips reshot.
3.3. Synthesized Data
KoDF synthesizes facial manipulations with six diverse models, preprocesses and postprocesses facial regions, and retains clips after human quality and realism screening.
- KoDF uses six synthesis models spanning face swapping, video-driven reenactment, and audio-driven reenactment.FaceSwap, DeepFaceLab, and FSGAN perform face swapping; FOMM is video-driven; ATFHP and Wav2Lip are audio-driven.
- Face swapping replaces the target identity with the source identity, whereas face reenactment changes the source person’s pose or expression while preserving other content.
- Facial regions are cropped, aligned, and resized to 512×512 pixels using 2DFAN landmarks before synthesis.Face-swapping training uses 4,000 to 5,000 selected frames per subject, while reenactment models need little or no additional training.
- The selected methods are not equally represented because manual validation removes unusable outputs, especially from methods sensitive to lighting and noise.
- FaceSwap uses a shared encoder with separate source and target decoders to learn non-identity features and identity-specific traits.
- DeepFaceLab adds an intervening network and mixed structural-plus-mean-squared-error loss to improve mapping and fidelity.
- FSGAN reenacts and segments faces, then inpaints and blends the completed face with the target; its adopted swap scheme fine-tunes each source-target pair for 800 iterations.
- FOMM separates appearance and motion, warps the source according to driving-video keypoints, and inpaints resulting artifacts.
3.4. Adversarial Attack
KoDF creates adversarial examples with a fast gradient sign procedure by training a preliminary detector and adding scaled gradient-sign noise to selected clips.
- KoDF uses the fast gradient sign method to simulate malicious attempts to evade deepfake detection.
- The pipeline trains a preliminary detector, derives noise frames that confuse it, and mixes those noises with corresponding input clips.
- The preliminary model is an EfficientNet-B4 backbone with two fully connected layers trained for 10 epochs on approximately 40,000 frames per category.
- Every one in ten clips is randomly selected, decomposed into frames, and processed iteratively through the trained preliminary model.
- Noise is formed from the sign of input gradients, scaled, and added to the original frames before reconstructing videos with the original audio.
- The noise scaling factor ϵ takes values 0.01, 0.05, and 0.1, while the preliminary model is retrained three times to vary noise intensity and character.
4. Detection Evaluation
The evaluation tests whether individual deepfake datasets generalize to unseen domains and finds that combining datasets produces more robust detection. The results attribute this pattern to overfitting to synthesis-specific artifacts and support using multiple datasets.
- Experimental setup: The study trains the DFDC winning model on equal-sized combinations of FF++, DFDC, and KoDF, then evaluates it on unseen test sets.The test sets include KoDF adversarial samples, GDFD, and DF-1.0.
- Generalization results: Each dataset alone is insufficient to approximate in-the-wild deepfake distributions, limiting adaptation across detection datasets.The authors report that standalone training produces models incapable of adapting to different deepfake datasets.
- Generalization results: Training on dataset combinations makes the detection model much more robust against out-of-domain data.The reported improvement is especially notable when all three datasets are used instead of only FF++ and DFDC.
- Interpretation: Deepfake detection is strongly prone to overfitting because models focus on artifacts that vary with synthesis methodology.The authors contrast these artifacts with diverse signals learned in regular image classification.
- Interpretation: No standalone published deepfake dataset provides sufficient generality, making adjoined multiple datasets a practical solution.The authors recommend incorporating diverse deepfake methods and a wide range of real videos.
5. Conclusions
KoDF is presented as a large-scale Korean-focused dataset for developing and evaluating deepfake detection methods. The authors expect it to be most effective when complementing existing and future datasets, and demonstrate benefits from compositing datasets for in-the-wild detection.
- Dataset and purpose: KoDF is a new large-scale dataset intended to help researchers develop and evaluate deepfake detection methods.It focuses on Korean subjects and expands the range and management of real and synthesized clips.
- Dataset and purpose: The authors expect KoDF to work more effectively by mutually complementing existing and future deepfake detection databases.They specifically identify FF++ and DFDC as milestone datasets for this complementarity.
- Main conclusion: Experiments demonstrate the benefit of compositing datasets for in-the-wild deepfake detection.The conclusion frames KoDF as a stepping stone for future deepfake detection studies.