Fighting Deepfake by Exposing the Convolutional Traces on Images

• GAN-generated Deepfakes are realistic, increasingly easy for non-experts to create, and can raise safety concerns.The introduction describes mobile apps such as FACEAPP as enabling realistic face transformations with a few taps.Source: §I · pp. 1–2
• CNN-based detectors can be context-specific, lack robustness and generalization, and rely on image semantics rather than a unique fingerprint.The paper identifies these as limits of recent detection approaches.Source: §I · p. 2
• The paper uses Expectation-Maximization to extract GAN-related Convolutional Traces and Random Forest classification for fakeness detection.The approach is also tested for robustness to image attacks and generalization across ten GAN architectures.Source: §I · p. 2
• The approach achieves the best classification results in almost all reported state-of-the-art comparisons while running on a common laptop CPU.The paper contrasts this computational demand with CNN-based methods that require substantial hardware and time.Source: §I · p. 2

• A GAN consists of a generative model G and discriminative model D, which are trained simultaneously in a min-max two-player game.G captures the data distribution, while D estimates whether samples come from training data rather than G.Source: §II · p. 2
• G maps random input z to output x, while D estimates whether x comes from the training-data distribution pdata.The figure is described as a schematic of the GAN learning framework.Source: §II · pp. 2–3, Figure 2 · p. 3
• Training aims for G to make D unable to distinguish generated samples from training samples, so the generated distribution pg follows pdata.The section notes that GANs can use different generative-model implementations and lists image translation, super-resolution, completion, and text-to-image generation.Source: §II · p. 3

• STARGAN performs image-to-image translation across multiple domains, while ATTGAN uses latent-space constraints to modify attributes.Examples include changing hair color or facial expression.Source: A. Deepfake Generation Techniques for… · p. 3
• GDWCT uses group-wise deep whitening and coloring to improve image translation and style-transfer efficiency and quality.Source: A. Deepfake Generation Techniques for… · p. 3
• STYLEGAN produced images for the “this person does not exist” website, and STYLEGAN2 improved its generator to fix some imperfections and create more realistic images.The described changes include redesigned normalization, multi-resolution, and regularization methods.Source: A. Deepfake Generation Techniques for… · p. 3

• Fourier-domain analysis uses frequency anomalies for detection; Guarnera et al. [4] reported preliminary insights, while Zhang et al. used a simple strategy with good performance.Wang et al. [18] proposed FakeSpotter, monitoring the behavior of a dedicated CNN to detect generated faces.Source: B. Deepfake detection methods · p. 3
• Wang et al. trained ResNet-50 on ProGAN images and reported generalization to other architectures and robustness to JPEG compression, blurring, and scaling.Source: B. Deepfake detection methods · p. 3
• DAD-HCNN uses three hierarchical levels to distinguish real from altered images, retouched from GAN-generated images, and then identify the GAN architecture.The passage reports high accuracy for detecting retouching and GAN-generated images.Source: B. Deepfake detection methods · p. 3
• CNN methods can achieve good results but tend to discriminate image semantics over GAN-specific traces and are computationally intensive and difficult to control.The paper identifies these as motivations for analytical fingerprint-based detection.Source: B. Deepfake detection methods · p. 3
• The paper evaluates ten generation architectures, with 2,000 images generated for each, to test generalization and robustness.The architectures include CYCLEGAN, STARGAN, ATTGAN, GDWCT, STYLEGAN, STYLEGAN2, PROGAN, FACEFORENSICS++, IMLE, and SPADE.Source: B. Deepfake detection methods · p. 4

III. EXTRACTING CONVOLUTIONAL TRACES

• The EM-derived CT represents local pixel correlations associated with GAN transpose-convolution layers, providing a feature vector that encodes image-generation structure.The method models natural pixels and other pixels as separate distributions and estimates the correlation kernel using weighted least squares.Source: §III · p. 4
• EM alternates between estimating each pixel’s probability of belonging to the natural model and updating model parameters, including the kernel.The natural model is Gaussian with zero mean and unknown variance; the alternative model is uniform.Source: §III · p. 4
• The kernel feature vector excludes k0,0 and is concatenated across the three RGB channels to form the image’s CT.For a 3×3 kernel, the resulting vector has 24 elements.Source: §III · p. 6
• CT extraction requires no training and is not a deep-learning architecture, so it does not encode high-level image semantics.The finalized pipeline applies EM to each RGB channel, concatenates the features, and uses Random Forest for real-versus-Deepfake classification.Source: §III · p. 6, Figure 4 · p. 6

IV. CLASSIFICATION OF DEEPFAKES

• For real-versus-GAN pairings, maximum accuracy was 92.99% for ATTGAN and 93.59% for CYCLEGAN, both using KNN with a 3×3 kernel.ATTGAN used KNN with K = 5 or 7; CYCLEGAN used K = 3.Source: §IV · pp. 7–8
• Maximum pairwise accuracy was 97.31% for FACEFORENSICS++, 91.58% for GDWCT, 97.76% for IMLE, and 95.85% for PROGAN.Each result used KNN with a 3×3 kernel; the K values were 3, 9, 3, and 5, respectively.Source: §IV · p. 8
• Maximum pairwise accuracy was 96.72% for SPADE, 90.55% for STARGAN, and 99.48% for STYLEGAN.SPADE used KNN with K = 3 and a 3×3 kernel; STARGAN used linear SVM and a 7×7 kernel; STYLEGAN used KNN with K = 3 and a 5×5 kernel.Source: §IV · p. 8
• A 3×3 kernel achieved the best average results across the classification tests, although the best kernel varied by task.The reported maximum for STYLEGAN2 was obtained with a 5×5 kernel.Source: §IV · p. 8
• CTs distinguished STYLEGAN from STYLEGAN2 with 99.31% maximum accuracy, despite the architectures being similar.The authors note that even a slight generator modification leaves different traces; on the separate real-versus-STYLEGAN2 task, accuracy reached 99.64%.Source: §IV · p. 8
• 98% accuracy was obtained with Random Forest for real-versus-deepfake classification across all 10 architectures.The task used a 70% training split and five-fold cross-validation.Source: §IV · p. 8, Table III · p. 8
• The experiments include four non-face generators, and the authors report that CTs also detect deepfakes that do not depict faces.Those architectures are CYCLEGAN, PROGAN, IMLE, and SPADE.Source: §IV · pp. 6, 8
• FACEAPP testing correctly classified 437 of 471 generated images as deepfakes, using the previously trained Random Forest without further training.The authors attribute some errors possibly to low light or minimal changes to source images.Source: §IV · pp. 8–9

V. ROBUSTNESS EXPERIMENTS

• The experiment applies attacks to CELEBA images and deepfakes from ATTGAN, GDWCT, STARGAN, STYLEGAN, and STYLEGAN2.Classifications compare attacked real images with attacked deepfakes for each architecture.Source: §V · p. 9
• Random rectangles remove image details, potentially obscuring pixel correlations used by CTs.The authors expect this to be a concern when rectangles cover the few elements changed by ATTGAN or STARGAN.Source: §V · p. 9
• Gaussian blur, rotation, and scaling are tested because their noise or interpolation may alter pixel correlations and CT information.The tested rotations are 45°, 90°, and 180°; scaling is +50% or −50%.Source: §V · p. 9

5) JPEG compression with quality factor equal to 50:

• JPEG quality-factor-50 compression is included as a real-world-like test because the authors compare it with compression used by social networks and messaging apps.The section notes that compression removes high-frequency information that may matter for CT discrimination.Source: 5) JPEG compression with quality… · pp. 9, 11
• CT extraction remains robust to almost all considered attacks in the real-versus-deepfake classification tests.The reported table uses the final classifier on an augmented dataset, with 70% for training and five-fold cross-validation.Source: 5) JPEG compression with quality… · p. 11, Table IV · p. 11
• STYLEGAN and STYLEGAN2 images achieve the best attacked classification accuracies, which the authors attribute to their larger original image size.GDWCT images are the smallest and are least robust to attacks; their accuracy remains comparable to the un-attacked result only under JPEG compression.Source: 5) JPEG compression with quality… · p. 11
• A 90° anticlockwise rotation, which introduces no interpolation, unexpectedly improves classification for every considered architecture.The authors suggest this may relate to a major CT direction and say it warrants further investigation.Source: 5) JPEG compression with quality… · p. 11

VI. COMPARISONS WITH DEEPFAKE DETECTION METHODS

• 53% accuracy is the best result from VGG-16 on the Real-versus-all-10-Deepfakes binary task, similar to a random classifier.The passage also describes CNN methods as context-dependent, prone to overfitting, and likely to rely on image semantics.Source: §VI · p. 11
• Accuracy exceeds 90% in all five tested architectures, and the proposed method beats FakeSpotter on average, though it is slightly lower on PROGAN and FACEFORENSICS++.The five architectures are STYLEGAN, STYLEGAN2, STARGAN, PROGAN, and FACEFORENSICS++.Source: §VI · p. 12, Figure 9 · p. 12
• The proposed method obtains better results than Wang et al. on SPADE, IMLE, and CYCLEGAN, architectures that do not produce face images.The comparison reports Average Precision and mean Average Precision across architectures.Source: §VI · p. 12, Figure 10 · p. 12

VII. CONCLUSIONS AND FUTURE WORKS

• Expectation-Maximization extracts Convolutional Traces that can identify Deepfakes and their generating GAN architecture, with reported discriminative power, attack robustness, and semantic independence.The authors state that rotating input images to find the most important direction can yield better results and plan to investigate this in future work.Source: §VII · p. 12