Source-linked AI summary
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, Nenghai Yu
TL;DR
Face forgery detectors can overfit known datasets and struggle with unseen manipulations. SPSL combines RGB images with phase-spectrum information and shallow learning to target common up-sampling artifacts and local textures. Experiments report state-of-the-art cross-dataset detection and improved multi-class classification, with scope limited by its dependence on up-sampling-based forgery generation.
Problem
Existing supervised detectors can overfit datasets and face challenges on unseen samples, motivating more transferable forgery detection.
Method
SPSL combines spatial images with reconstructed phase-spectrum representations and uses shallow networks to emphasize local textures over high-level semantics.
Results
SPSL achieves state-of-the-art performance in cross-dataset forged-face detection and improves multi-class classification.
Takeaways & Limitations
Phase-spectrum information and local-region learning provide the basis for SPSL's reported transferability and multi-class detection improvements.
Takeaways & Limitations
SPSL depends on up-sampling in forgery generation and may lose transferability across different manipulation categories.
Abstract
from arXiv · showhide
The remarkable success in face forgery techniques has received considerable attention in computer vision due to security concerns. We observe that up-sampling is a necessary step of most face forgery techniques, and cumulative up-sampling will result in obvious changes in the frequency domain, especially in the phase spectrum. According to the property of natural images, the phase spectrum preserves abundant frequency components that provide extra information and complement the loss of the amplitude spectrum. To this end, we present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery to improve the transferability, for face forgery detection. And we also theoretically analyze the validity of utilizing the phase spectrum. Moreover, we notice that local texture information is more crucial than high-level semantic information for the face forgery detection task. So we reduce the receptive fields by shallowing the network to suppress high-level features and focus on the local region. Extensive experiments show that SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
1. Introduction
Face forgery detection must generalize beyond training datasets because existing supervised methods can overfit manipulation-specific data. SPSL targets up-sampling artifacts in the phase spectrum and local textures, achieving strong cross-dataset and multi-class results.
- Face forgery methods can synthesize realistic faces, creating security and ethical risks that motivate more general detection.
- Supervised detectors often overfit their training datasets, limiting effectiveness on unseen samples and motivating improved transferability.
- Up-sampling is common in generative face synthesis and leaves frequency-domain traces, while amplitude-spectrum detection loses information.
- SPSL leverages phase-spectrum artifacts because repeated up-sampling produces larger phase-spectrum changes than amplitude-spectrum changes.
- SPSL suppresses high-level semantics by using shallow networks with smaller receptive fields, emphasizing local texture information.
- SPSL achieves state-of-the-art performance in cross-dataset forged-face detection and improves multi-class classification.
2. Related work
Related work mainly detects face forgery through spatial-domain artifacts or frequency-domain representations. Prior frequency-based methods use transforms such as DFT, WT, DCT, or LoG, but cross-dataset performance can remain limited.
- Recent face forgery detectors commonly exploit spatial-domain artifacts, including handcrafted facial features and high-level representations from deep neural networks.
- Frequency-based detectors convert spatial images using DFT, WT, or DCT to capture forgery artifacts.
- Amplitude statistics, DCT features, and LoG enhancement have been used for frequency-based detection, with some methods showing strong compressed-video performance.
- F3-Net achieves state-of-the-art performance on highly compressed videos, but its cross-dataset performance drops greatly.
3. SPSL for Face Forgery Detection
SPSL combines RGB imagery with a phase-spectrum representation to detect up-sampling artifacts, while shallow learning emphasizes local texture over high-level semantics.
- Phase-spectrum input: SPSL reconstructs the phase spectrum into the spatial domain and concatenates it with RGB channels, producing a four-channel RGBP input.The phase representation is obtained with IDFT applied to the frequency spectrum without amplitude.
- Up-sampling artifacts: Up-sampling introduces new or compressed frequency components, and the phase spectrum is identified as more sensitive to these artifacts than the amplitude spectrum.The paper’s derivation links increased spatial resolution with frequency-domain compression and duplicated frequency components.
- Frequency information: Theoretical analysis argues that combining phase and amplitude information lets CNNs learn more frequency components than using the original image alone.The paper states that phase information helps recover components tending toward zero in the amplitude spectrum and supports this with a distributive-law analysis.
- Local-region learning: The method uses shallow networks to suppress high-level semantic features and focus on local low-level cues such as textures and colors.The paper motivates shallowing by noting that semantic characteristics are shared between pristine and forged faces.
- Evaluation: SPSL is evaluated on FaceForensics++ under high- and low-quality compression settings using ACC and AUC.The reported table compares quantitative performance across light and heavy compression conditions.
4. Experiments
Experiments evaluate SPSL on FF++ and cross-dataset transfer to Celeb-DF using ACC, AUC, and recall. SPSL achieves comparable single-dataset performance and state-of-the-art cross-dataset results, with phase-spectrum cues supporting detection across manipulations.
- Experimental settings: Experiments use FF++ and Celeb-DF, with ACC and AUC as primary metrics and recall for multi-class classification.Models are trained on FF++ and evaluated on FF++ and Celeb-DF.
- FF++ evaluation: SPSL matches or outperforms the baseline on FF++ under low-quality compression and obtains comparable results under high-quality compression.The authors attribute the low-quality gains to extra phase information retaining more frequency components than RGB images.
- FF++ evaluation: SPSL achieves comparable performance across four FF++ manipulation methods, while transferability remains its primary objective.The comparison covers DeepFakes, Face2Face, FaceSwap, and NeuralTextures.
- Cross-dataset evaluation: SPSL obtains state-of-the-art AUC on Celeb-DF while retaining good performance on FF++.The cross-dataset setting trains on multiple FF++ manipulations and tests on Celeb-DF.
- Cross-dataset evaluation: Phase-spectrum patterns expose manipulation-specific traces that are less obvious in RGB images, supporting cross-dataset detection and multi-class separation.The phase representation makes differences from cumulative up-sampling more perceptible to convolutional kernels.
5. Ablation Study
The ablation study separates the contributions of phase-spectrum input and network shallowing, then evaluates their combination and backbone generality. Both components improve transferability, while SPSL performs best together and generalizes beyond Xception.
- Multi-class classification: In multi-class classification, SPSL completely surpasses original XceptionNet across raw, high-quality, and low-quality compression settings.The evaluation reports recall for the origin and each manipulation method.
- Multi-class classification: SPSL forms separable feature clusters for all classes, including NeuralTextures images that are nearly indistinguishable from pristine faces in RGB.The phase spectrum shows distinguishing patterns among the manipulation methods.
- Effectiveness of Phase spectrum and Shallow network: Phase-spectrum input and a shallow network each improve Celeb-DF AUC over baseline Xception in ablation experiments.All ablation models are trained on FF++ and tested on Celeb-DF.
- Effectiveness of Phase spectrum and Shallow network: Combining phase spectrum with network shallowing produces the best ablation performance, increasing AUC by about 13%.The comparison removes the phase-spectrum and shallow-operation components step by step.
- Feature visualization: SPSL focuses on more microcosmic regions than baseline Xception, whose Grad-CAM emphasizes global information.This visualization agrees with the method’s motivation to prioritize local texture information.
- Backbone generality: The proposed SPSL framework also works with ResNet34 and ResNet50 after their residual blocks are halved.The authors describe this as evidence of generality across backbones.
6. Limitations
SPSL relies on up-sampling artifacts and may lose transferability across fundamentally different manipulation categories. These constraints limit its scope when forged faces are not generated through the assumed processes.
- Scope limitations: SPSL depends on up-sampling during forged-face generation, so performance may drop for forgeries not based on generative models.This is identified as a limitation of the method.
- Scope limitations: Transferability can drop when the test forgery belongs to an entirely different manipulation type than the training data.The paper gives expression swapping after identity-swap training as an example.
7. Conclusion
SPSL combines spatial and frequency information for face forgery detection, using phase-spectrum cues and local-region emphasis to improve robustness, especially across datasets.
- SPSL combines spatial and frequency information for face forgery detection.
- Its phase spectrum captures duplicated frequency components introduced during up-sampling in forged-face generation.
- SPSL focuses on local microcosmic regions while suppressing global semantic information for greater robustness.
- Extensive experiments demonstrate excellent performance, especially in the challenging cross-dataset evaluation task.