PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepFakes Evolution: Analysis of Facial Regions and Fake Detection Performance

Ruben Tolosana, Sergio Romero-Tapiador, Julian Fierrez, Ruben Vera-Rodriguez

arXiv:2004.07532v2cs.CVcs.MM

TL;DR

As DeepFakes became more realistic across database generations, the study compares full-face and region-based detection using state-of-the-art detectors. Detection errors remain substantially higher on second-generation databases than on first-generation ones.

  • Problem

    The increased realism of second-generation DeepFakes raises the question of how well machines can detect them.

    Source: §I · p. 1

  • Method

    The study compares first- and second-generation databases using full-face and specific-region inputs and two state-of-the-art fake detectors.

    Source: §I · p. 2

  • Results

    Second-generation databases yield EER values around 20-30%, versus 1-3% for first-generation databases; nose, mouth, and face-edge results range from 24% to 44% EER.

    Source: §VI · p. 7, B. Facial Regions Analysis · p. 7

  • Takeaways & Limitations

    The analysis offers insights for robust detectors that fuse facial regions under different conditions and for focusing future DeepFakes on artifacts in specific regions.

    Source: §VI · p. 7

Abstract

from arXiv · show

Media forensics has attracted a lot of attention in the last years in part due to the increasing concerns around DeepFakes. Since the initial DeepFake databases from the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an exhaustive analysis of both 1st and 2nd DeepFake generations in terms of facial regions and fake detection performance. Two different methods are considered in our experimental framework: i) the traditional one followed in the literature and based on selecting the entire face as input to the fake detection system, and ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Among all the findings resulting from our experiments, we highlight the poor fake detection results achieved even by the strongest state-of-the-art fake detectors in the latest DeepFake databases of the 2nd generation, with Equal Error Rate results ranging from 15% to 30%. These results remark the necessity of further research to develop more sophisticated fake detectors.

I. INTRODUCTION

DeepFakes have become more realistic across database generations, raising questions about automated detection. The study compares full-face and facial-region detection across generations and detectors.

I. INTRODUCTION

  • Second-generation databases improve realism and include more varied acquisition conditions, including lighting, distance, and pose variations.Source: §I · p. 1
  • Earlier detectors achieved strong results on first-generation databases, prompting the question of performance on newer Identity Swap databases.Source: §I · p. 1
  • The study compares full-face and facial-region inputs across first- and second-generation DeepFake databases using state-of-the-art detectors.Source: §I · p. 2, Figure 1 · p. 2
  • The study examines detector performance across generations and the discriminative power of facial regions across generations and detectors.Source: §I · p. 2
  • The analysis is intended to inform robust detector proposals using region fusion under different scenarios and attention to regional artifacts in future DeepFakes.Source: §I · p. 2

II. PROPOSED EVALUATION FRAMEWORK

The evaluation framework separates facial-region segmentation from fake detection systems.

II. PROPOSED EVALUATION FRAMEWORK

  • The framework evaluates facial regions and fake-detection performance through two modules.Source: §II · p. 2
  • Facial-region segmentation is described as the framework's first module.Source: §II · p. 2
  • Fake-detection systems form the second module of the framework.Source: §II · p. 2

A. Facial Region Segmentation

The framework compares the entire face with four segmented facial regions, using OpenFace2 landmarks to define the regions.

A. Facial Region Segmentation

B. Fake Detection Systems

The study evaluates Xception and Capsule Network detectors, training a separate detector for each database and facial region.

B. Fake Detection Systems

III. DATABASES

The study evaluates four public databases spanning two generations of DeepFake videos. UADFV provides an early-generation example with one identity represented across all fake videos.

III. DATABASES

  • Four public databases are examined: UADFV and FaceForensics++ from the 1st generation, and Celeb-DF and DFDC from the 2nd.Table I summarizes their main features.Source: §III · p. 3

A. UADFV

  • UADFV contains 49 real YouTube videos and 49 fakes made with FakeApp by swapping in Nicolas Cage’s face.Videos average 11.14 seconds at a typical resolution of 294×500 pixels.Source: A. UADFV · p. 3
  • All UADFV fake videos use the same identity, Nicolas Cage, while each video represents one individual.Source: A. UADFV · p. 3

B. FaceForensics++

The section describes three public databases spanning distinct face-swapping approaches and generation settings. Their video sources and synthesis conditions differ across datasets.

B. FaceForensics++

  • FaceForensics++ contains 1,000 real YouTube videos and fake videos generated with computer graphics and deep learning; this study analyzes the computer-graphics FaceSwap fakes.FaceSwap uses face alignment, Gauss Newton optimization, and image blending.Source: B. FaceForensics++ · p. 3

C. Celeb-DF

  • Celeb-DF pairs 408 real YouTube videos from 59 celebrities with 795 fakes, aiming for better visual quality than UADFV.The real videos vary in face size, orientation, lighting, and background; fakes swap faces between pairs of subjects.Source: C. Celeb-DF · pp. 3–4

D. DFDC

  • DFDC Preview includes 1,131 real videos from 66 paid actors and 4,119 fakes generated with two unknown approaches.The fakes swap subjects with similar facial attributes, and the videos include varied acquisition scenarios, lighting, camera distances, and poses.Source: D. DFDC · p. 4

IV. EXPERIMENTAL PROTOCOL

The experimental protocol separates identities between development and evaluation to assess performance on unseen people. The results compare whole-face detection with facial-region analysis and state-of-the-art performance.

IV. EXPERIMENTAL PROTOCOL

  • Each database is divided into development and evaluation sets, using approximately 80% and 20% of identities, respectively.For Celeb-DF, the sets use 40 and 19 identities, respectively.Source: §IV · p. 4
  • Separating identities, rather than only videos, is intended to support fair evaluation and estimate generalization to unseen identities.For UADFV, all videos of Donald Trump are held for final evaluation.Source: §IV · p. 4

V. EXPERIMENTAL RESULTS

  • The experiments evaluate detectors using the entire face, analyze the discriminative power of individual facial regions, and compare results with the state of the art.Source: §V · p. 4

A. Entire Face Analysis

Whole-face detection performs well on first-generation datasets but degrades substantially on second-generation databases. The evaluation also shows why development and evaluation identities must be separated.

A. Entire Face Analysis

  • 23.05% EER for Xception and 22.84% EER for Capsule Network were achieved on average across 2nd-generation databases; Celeb-DF EERs were 28.55% and 24.29%, respectively.Both detectors showed an average absolute worsening of around 20% EER compared with 1st-generation databases.Source: A. Entire Face Analysis · p. 4
  • 1st-generation videos yielded AUC values close to 100%, while FaceForensics++ had around 3% EER and was more challenging than UADFV.Source: A. Entire Face Analysis · p. 4
  • Using the same identities in development and evaluation produced results up to five times better than using different identities, but the paper attributes this to learning identity features.The authors warn that such results can mislead because those features are not the key features for distinguishing real from fake videos.Source: A. Entire Face Analysis · p. 4

B. Facial Regions Analysis

Across the evaluated databases, Eyes generally provide the strongest regional detection results, while Rest tends to perform worst. Second-generation fakes show greater realism in some facial regions and remain substantially harder to detect than first-generation fakes.

B. Facial Regions Analysis

  • A separate detector is trained for each facial region and database, and Grad-CAM visualizations show image areas important to detector decisions.The paper reports similar Grad-CAM results for Xception and Capsule Network.Source: B. Facial Regions Analysis · p. 4, Figure 3 · p. 6
  • Second-generation fakes show improved realism in Nose, Mouth, and Rest, yet detectors still reach 24%–44% EER; Eyes remain the most useful region.The analysis reports this range across databases and identifies Eyes as the strongest region overall.Source: B. Facial Regions Analysis · pp. 4, 5, 7, Table II · p. 5
  • On UADFV, Eyes-only Capsule Network EER is reported as 2.00% versus 0.28% using the entire Face; Xception using Rest achieves 7.90% EER.The passage links Xception's Rest result to color contrast and visible mask boundaries.Source: B. Facial Regions Analysis · p. 5
  • On FaceForensics++, Mouth is best at 13.77% and 9.66% EER, while Rest is worst at 22.37% and 21.58% for Xception and Capsule Network, respectively.The passage associates Mouth results with blurred teeth and lip inconsistencies, and suggests improved color contrast and mask boundaries may explain Rest's weaker performance.Source: B. Facial Regions Analysis · p. 5
  • For UADFV, Capsule Networks improve absolute EER over Xception by 1.92%, 9.58%, and 9.30% for Eyes, Nose, and Mouth, respectively.Source: B. Facial Regions Analysis · p. 5
  • On Celeb-DF, Eyes achieve around 30% EER, while other facial regions yield around 40%; the Eyes result is over 13 times higher than UADFV’s original EERs.Source: B. Facial Regions Analysis · p. 5
  • On DFDC, Eyes yield EERs of 23.82% and 25.06%, improving on Celeb-DF Eyes by 5.58% and 5.52%, respectively.Source: B. Facial Regions Analysis · p. 7

C. Comparison with the State of the Art

The study compares Xception and Capsule Network with prior fake detectors across four databases. Both proposed detectors achieve state-of-the-art results, though the comparison is not always protocol-matched.

C. Comparison with the State of the Art

  • The AUC comparison covers methods using head-pose variation, face-warping artifacts, mesoscopic features, image and temporal features, and deep-learning features with attention.Source: C. Comparison with the State of the Art · p. 7
  • Both Xception and Capsule Network achieve state-of-the-art results across the evaluated databases; Xception leads on Celeb-DF and DFDC, while Capsule Network leads on FaceForensics++.Both detectors also obtain good results on UADFV.Source: C. Comparison with the State of the Art · p. 7
  • Because the compared results do not always use the same datasets and protocols, the reported rankings should be interpreted with care.Source: C. Comparison with the State of the Art · p. 7

VI. CONCLUSIONS

The conclusions compare whole-face and region-specific detection across first- and second-generation databases. Second-generation fakes remain harder to detect, while the regional analysis points to scenario-dependent detector design and specific facial areas for future fake improvements.

VI. CONCLUSIONS

  • The evaluation compares detectors given either the entire face or specific regions, including the eyes and nose, across databases from both generations.Source: §VI · p. 7
  • Second-generation databases yield around 20%–30% EER, versus 1%–3% for first-generation databases; regional results reach 24%–44% EER.Source: §VI · p. 7
  • The authors identify improved second-generation realism in the nose, mouth, and face edge, alongside poor detection performance in those regions.Source: §VI · p. 7
  • The analysis offers research directions: fuse facial regions according to lighting, pose, and camera distance, and target artifacts in specific regions when improving future DeepFakes.Source: §VI · p. 7
Loading 2004.07532v2…