Source-linked AI summary
All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark
Junjiang Wu, Liejun Wang, Zhiqing Guo
TL;DR
Existing proactive forensics methods are described as treating detection, tampering localization, and source tracing separately, despite the need to address these tasks together. The paper combines a landmark-identity watermark with a factorized decoder to perform all three. The authors report a unified, robust, and imperceptible framework for facial forensics.
Problem
Existing proactive forensics methods typically address detection, tampering localization, and source tracing separately, rather than jointly.
Source: §1 · p. 2Method
The framework combines a 152-D landmark-identity watermark with an FHD that uses regression and classification heads to recover landmarks and source identifiers.
Source: §1 · p. 2Results
The authors report a robust, imperceptible framework that unifies facial deepfake detection, tampering localization, and source tracing.
Source: §5 · p. 8Takeaways & Limitations
The framework uses landmark consistency for detection and localization while recovering an identifier for source tracing.
Source: §5 · p. 8Disrupt-first proactive defense risks interfering with benign applications because disruption remains its primary goal.
Source: §2.1 · p. 3
Abstract
from arXiv · showhide
With the rapid advancement of deepfake technology, malicious face manipulations pose a significant threat to personal privacy and social security. However, existing proactive forensics methods typically treat deepfake detection, tampering localization, and source tracing as independent tasks, lacking a unified framework to address them jointly. To bridge this gap, we propose a unified proactive forensics framework that jointly addresses these three core tasks. Our core framework adopts an innovative 152-dimensional landmark-identity watermark termed LIDMark, which structurally interweaves facial landmarks with a unique source identifier. To robustly extract the LIDMark, we design a novel Factorized-Head Decoder (FHD). Its architecture factorizes the shared backbone features into two specialized heads (i.e., regression and classification), robustly reconstructing the embedded landmarks and identifier, respectively, even when subjected to severe distortion or tampering. This design realizes an "all-in-one" trifunctional forensic solution: the regression head underlies an "intrinsic-extrinsic" consistency check for detection and localization, while the classification head robustly decodes the source identifier for tracing. Extensive experiments show that the proposed LIDMark framework provides a unified, robust, and imperceptible solution for the detection, localization, and tracing of deepfake content. The code is available at https://github.com/vpsg-research/LIDMark.
1. Introduction
The introduction argues that existing proactive methods do not jointly provide detection, tampering localization, and source tracing. It presents LIDMark and the Factorized-Head Decoder (FHD) as a unified approach to those three tasks.
1. Introduction
- The introduction frames realistic deepfakes as a threat to public trust and social order through false information, financial fraud, and malicious impersonation.It links that risk to the urgency of robust forensics.Source: §1 · p. 1
- Passive forensics seeks manipulation artifacts but is described as poorly generalizing to unseen techniques, motivating proactive methods that embed imperceptible signals.Those signals can disrupt malicious generation or support downstream forensic tasks.Source: §1 · p. 1
- Existing approaches address single or paired forensic tasks, leaving a gap in methods that also localize manipulated facial regions.The introduction identifies localization as the unanswered question of where an image is fake.Source: §1 · p. 2
- The proposed intrinsic-extrinsic check compares reconstructed original landmarks with landmarks redetected from the potentially manipulated image.A high global Average Euclidean Distance indicates forgery; local inconsistencies support region-based localization.Source: §1 · p. 2
- LIDMark and FHD jointly support deepfake detection, tampering localization, and source tracing in one proactive forensics framework.The framework combines a 136-D tamper-sensitive facial landmark vector with a 16-D robust source identifier; FHD uses dedicated regression and classification heads.Source: §1 · p. 2
2. Related Work
Related work contrasts proactive defense, which aims to disrupt generation, with proactive forensics, which embeds information for downstream tasks. It describes a progression from single-task watermarking to dual-function methods, while noting that an integrated trifunctional solution is sought.
2.1. Proactive Defense
- Proactive defense injects imperceptible perturbations to disrupt generation, but the disrupt-first approach risks interfering with benign applications and lacks key forensic functions.Traceable approaches are described as fragile and insufficient in capacity for meaningful forensic analysis.Source: §2.1 · pp. 2–3
2.2. Proactive Forensics
- Proactive forensics uses watermark fragility to serve different tasks: semi-fragile schemes support detection, robust schemes support tracing, and dual-decoder methods combine both.The review names FaceSigns and Asnani et al. for detection, FakeTagger and DiffMark for tracing, and SepMark and KAD-Net as dual-function examples.Source: §2.2 · p. 3
- The related-work discussion points toward a more integrated trifunctional forensic solution beyond the described single- and dual-function approaches.The passage presents this as a direction of integration rather than detailing a specific method.Source: §2.2 · p. 3
3. Methods
LIDMark combines tamper-sensitive facial landmarks with a robust source identifier in one watermark, then embeds and decodes it through a framework trained against simulated distortions and deepfake manipulations. Its multi-task losses support imperceptible embedding and recovery of both payload components.
3.1. Overview of LIDMark
- LIDMark concatenates a 136-D facial landmark vector for detection and localization with a 16-D source identifier for tracing into one 152-D watermark.The identifier is generated from the image filename using SHA-256, then truncated and mapped to a bipolar vector.Source: §3.1 · p. 3
3.2. Network Architecture
- The encoder fuses image and watermark features to produce a watermarked image, while stochastic manipulation simulates common distortions and deepfake attacks before decoding.The attack operator samples one manipulation per mini-batch; InfoSwap is reserved for testing to assess generalization to an unseen attack.Source: §3.2 · pp. 3–4, Figure 3 · p. 4
- The FHD shares a backbone and separates landmark regression from identifier classification, recovering both payloads from the manipulated image.The framework also uses a discriminator in its adversarial objective.Source: §3.2 · pp. 4–5, Figure 3 · p. 4
3.3. Multi-Task Loss Function
- The multi-task losses combine landmark and identifier recovery with image-fidelity and adversarial objectives, with additional generative-consistency and stability losses during deepfake fine-tuning.The landmark loss uses mean pointwise Euclidean distance, while identifier recovery uses binary cross-entropy; the stability loss addresses forgetting on non-attacked watermarked images.Source: §3.3 · p. 5
4. Experiments
Experiments evaluate LIDMark’s visual quality, three forensic functions, cross-dataset generalization, and decoder losses. The reported results show strong tracing under deepfake attacks, high visual quality, and complementary roles for landmark and identifier losses.
4.1. Implementation Details
- Experiments used CelebA-HQ (30,000 images) as the main training and evaluation benchmark; generalization was tested on 2,000 randomly selected LFW images, with images resized to 128 × 128 and 256 × 256.Source: §4.1 · pp. 5–6
4.2. Visual Quality
- LIDMark achieved high visual quality, surpassing LampMark and DiffMark while embedding a 152-D payload; the watermarked image was visually indistinguishable from the cover image.The visual comparison reports no discernible artifacts between the cover and watermarked images.Source: §4.2 · p. 6, Figure 4 · p. 6
4.3. Trifunctional Evaluation
- LIDMark achieved the best overall average BER under deepfake attacks, with near-perfect recovery for identity-focused swaps but harder recovery under global attribute edits.The evaluation also reports a separable landmark-AED margin between common distortions and deepfake manipulations, with AUC 0.9388 and an optimal threshold of 3.2375 px.Source: Table 3 · p. 7, §4.3 · pp. 7–8
4.4. Cross-Dataset Evaluation
- On unseen LFW data, the model maintained high fidelity and low intrinsic landmark AED, and achieved the lowest average BER among baselines on deepfakes.The model trained on CelebA-HQ was applied directly to LFW, which has different identities and capture conditions.Source: Table 4 · p. 8, Table 5 · p. 8, §4.4 · p. 8
4.5. Ablation Studies
- The full jointly optimized model best balanced all objectives: removing landmark loss broke geometric recovery, while removing identifier loss compromised tracing.The ablation attributes landmark recovery to detection and localization, and identifier recovery to tracing.Source: Table 6 · p. 8, §4.5 · p. 8
5. Conclusion
The conclusion presents LIDMark as a unified proactive-forensics framework for detection, localization, and source tracing. It identifies robustness to unseen deepfakes and extension to video as future work.
5. Conclusion
- LIDMark combines a 152-D landmark-identity watermark with an FHD that extracts landmarks for detection and localization and identifiers for tracing.The conclusion describes the framework as a robust, imperceptible unified approach to facial forensics.Source: §5 · p. 8
A. Scalability Analysis
Expanding LIDMark’s identifier from 16-D to 32-D increases total payload capacity while preserving robust landmark recovery. The larger identifier supports more source identities, with experiments using hashed filenames to simulate tracing.
A. Scalability Analysis
- 10.5% more total payload capacity incurs only a marginal PSNR cost while improving landmark recovery under common distortions and deepfake manipulations, without interfering with the geometric signal.The comparisons cover objective imperceptibility and BER/AED under common distortions and deepfake manipulations.Source: A. Scalability Analysis · p. 11, Table 7 · p. 11, Table 8 · p. 11
- A 32-D identifier accommodates 2^32 ≈ 4.3 × 10^9 identities, compared with 2^16 for the 16-D payload; experiments use hashed filenames to simulate tracing.The identifier can represent arbitrary provenance or copyright data.Source: A. Scalability Analysis · p. 11
- The expanded 168-D payload combines the 136-D landmark vector with a 32-D identifier, while the 136-D landmarks remain sufficient for detection and localization.Source: A. Scalability Analysis · p. 11
B. Complexity Analysis
LIDMark uses fewer parameters than the compared dual-function methods, but incurs higher FLOPs. The authors present this compute trade-off as supporting dense localization and robust watermark recovery, while the shared backbone keeps the model compact.
B. Complexity Analysis
- At 128 × 128 resolution, LIDMark requires only 5.82M parameters, approximately 20% of SepMark’s [40] capacity and 17% of WaveGuard’s [17], respectively.Source: B. Complexity Analysis · p. 11
- LIDMark incurs higher FLOPs, which the authors describe as a necessary investment in dense pixel-level processing for precise localization and high-fidelity watermark recovery.They characterize the latency increase as a trade-off for combining three forensic tasks in one framework.Source: B. Complexity Analysis · pp. 11–12
- The 168-D robustness figure contrasts recovered intrinsic landmarks with re-detected extrinsic landmarks to visualize their spatial differences.Its rows also compare watermarked and cover images and show manipulation results.Source: Figure 6 · p. 12