LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection

1. Introduction

• High-quality deepfakes contain subtle, localized artifacts, but common convolutional architectures tend to learn global features as localized cues fade across layers.The paper identifies capturing these local and subtle artifacts as necessary.Source: §1 · pp. 1–2
• LAA-Net uses a multi-task framework with classification, heatmap, and self-consistency branches to focus on vulnerable pixels and their neighborhoods.The heatmap and self-consistency targets are generated using blending-based fake-data synthesis.Source: §1 · p. 2
• E-FPN aggregates multi-scale features while limiting redundancy, complementing explicit attention with implicit consideration of local cues.The paper describes the attention mechanism and E-FPN as complementary components.Source: §1 · p. 2
• LAA-Net reports better and more stable AUC across deepfake-quality ranges than existing methods, and reports outperforming state-of-the-art methods on several benchmarks.Figure 1 compares AUC across Mask-SSIM ranges; Average Precision is also reported.Source: Figure 1 · p. 1, §1 · p. 2
• The contributions include training with real data only and evaluations on FF++, CDF2, DFD, DFDC, and DFW.The paper reports extensive experiments and comprehensive analysis on these benchmarks.Source: §1 · p. 2

2. Related Works: Attention-based Deepfake Detection

• Earlier deepfake methods often use binary classification, while later strategies include multi-task learning and pseudo-fake generation.The paper notes that purely binary classification has been associated with poor generalization capabilities.Source: §2 · p. 2
• Traditional DNN backbones tend to produce global features, potentially ignoring informative low-level cues and weakening high-quality deepfake detection.The paper argues that adequate strategies are needed to model localized artifacts.Source: §2 · p. 3
• Attention-based methods [55] implicitly focus on low-level artifacts but use a single real-versus-deepfake classifier without pseudo-fake generation or multi-task learning.The paper reports that these methods generalize less well to unseen datasets than other recent techniques [41] [52].Source: §2 · p. 3

3. Localized Artifact Attention Network (LAA-Net)

• LAA-Net combines a classification branch with heatmap and self-consistency branches to direct attention toward vulnerable pixels.Only real and pseudo-fake images are used during training.Source: §3 · p. 3, §3.1.2 · p. 5

3.1.1 Blending-based Data Synthesis

• Blending-based synthesis creates pseudo-fakes from foreground and background images, avoiding actual deepfakes and manual annotations for training.The blend uses a deformed Convex Hull mask; its boundary mask marks pixels more affected by blending, from which vulnerable pixels are selected.Source: §3.1.1 · p. 3, §3.1.2 · p. 4

3.1.2 Proposed Multi-task Learning Framework

• The heatmap represents vulnerable pixels and their neighborhoods, while self-consistency compares a selected vulnerable point with other pixels in the blended mask.The self-consistency branch produces a two-dimensional matrix and uses binary cross-entropy against its predicted counterpart.Source: §3.1.2 · pp. 4–5

3.2. Enhanced Feature Pyramid Network (E-FPN)

• E-FPN propagates relevant information from higher-resolution features and filters lower-level features to enrich representations while limiting redundancy.It applies convolution and transpose convolution to features from later layers, then weights and concatenates features across layers.Source: §3.2 · p. 5

4.1. Experimental Settings

• Experiments train and validate on FF++ and test generalization on CDF2, DFD, DFDC, and DFW, reporting video-level AUC and AP.Training uses real images to dynamically generate pseudo-fakes; the study also evaluates perturbations and component ablations.Source: §4.1 · pp. 5–7

4.2. Comparison with State-of-the-art

• LAA-Net achieves state-of-the-art results on CDF2, DFW, DFD, and DFDC; on CDF2, it exceeds Multi-attentional and SFDG by 27.14% AUC and 19.57% AP.The best performance is reached using SBI for data synthesis; BI is slightly better only on DFD, by 1.08% AUC and 0.4% AP.Source: Table 1 · p. 6, §4.2 · p. 7
• 99.96% AUC is achieved on FF++ by LAA-Net combined with SBI, using only real data for training.Source: Table 1 · p. 6, §4.2 · p. 7

4.3. Ablation Study

• The full model performs best in the cross-dataset ablation, and the heatmap branch contributes more to improving results than the other components.Grad-CAM examples show more precise activation of blending regions when all components are combined.Source: Table 3 · p. 7, §4.3 · p. 7

4.2. Comparison with State-of-the-art

• Color changes such as saturation and contrast do not affect performance, whereas structural perturbations such as Gaussian noise cause high sensitivity.The authors identify structural-perturbation robustness as future work.Source: §4.2 · p. 7

4.4. E-FPN versus Traditional FPN

• E-FPN generally outperforms traditional FPN across integrated feature levels, except for F(5).The comparison supports using multi-scale features while reducing redundancy.Source: Table 4 · p. 8, §4.4 · p. 8

4.5. Sensitivity Analysis

• The model is robust to different λ1 and λ2 values, with best average performance at λ1 = 10 and λ2 = 100.Source: Table 5 · p. 8, §4.5 · p. 8

• LAA-Net combines classification, heatmap, and self-consistency branches with E-FPN to detect high-quality deepfakes while generalizing to unseen manipulations.Source: §5 · p. 8
• The reported results show superiority over state-of-the-art methods across several benchmarks, while future work will investigate noise robustness and temporal information.Source: §5 · p. 8

• The supplement covers self-consistency loss and heatmap ground-truth generation, additional metrics for in-dataset and cross-dataset evaluations, and qualitative E-FPN-versus-FPN comparisons.Source: Overview · p. 9

• A randomly selected vulnerable point anchors the predicted and ground-truth consistency maps, with similarities computed against other features or pixels.The predicted map uses feature similarity; the ground-truth map uses pixel consistency within the blended mask.Source: Figure 7 · p. 9, §6 · p. 9
• The self-consistency loss is binary cross-entropy between the predicted consistency map and its ground truth.Source: §6 · p. 9

7. Ground Truth Generation of Heatmaps

• For each selected vulnerable point, blending-mask height and width define a virtual bounding box; boxes with IoU greater than t = 0.7 are identified, and a tight circle encompassing them determines radius rk.Source: §7 · p. 9
• The heatmap construction begins by selecting a vulnerable point and measuring the blending mask's height and width at that location.Source: §7 · p. 9
• The procedure repeats for every vulnerable point, and the final heatmap H is the superimposition of all point-specific Gaussians g_k.Source: §7 · p. 9

8. Additional Results

• LAA-Net achieves better performance than other state-of-the-art methods in the in-dataset and cross-dataset settings.Tables 6 and 7 report AUC, AP, AR, and mF1; Table 7 covers CDF2, DFW, DFD, and DFDC.Source: §8 · p. 9, Table 7 · p. 10

8.1. Qualitative Results: E-FPN versus FPN

• E-FPN produces more precise activations on blending boundaries than FPN, while FPN also attends to regions outside those boundaries.Source: §8.1 · pp. 9–10

8.2. Qualitative Results: Gaussian Noise

• Gaussian noise highly impacts vulnerable-point detection, and the reported performance decline is attributed as possibly reflecting that detection difficulty.Source: §8.2 · p. 10, Figure 10 · p. 10

8.3. Robustness to Compression

• On the c23 version of FF++, LAA-Net's overall AUC is 89.30%.Source: §8.3 · p. 10