PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Cross-Reconstruction for Generalizable Deepfake Detection

Bingjian Yang, Shilei Zhao, Zheng Wang

arXiv:2610.01544v1cs.CV

TL;DR

Deepfake detectors can generalize poorly to unseen manipulations, while cross-reconstruction methods may align heterogeneous artifacts and exclude them from reconstruction. This paper preserves diverse artifacts through semantically aligned cross-generator reconstruction with artifact-aware masked residual targets. It reports the highest average AUC among compared methods in cross-dataset evaluation.

  • Problem

    Cross-reconstruction methods may align heterogeneous artifacts and exclude artifact representations from reconstruction, overlooking diverse visual cues that could support detection of unseen manipulations.

    Source: Introduction · p. 1

  • Method

    The framework uses semantically aligned cross-generator pairs, incorporates artifact representations into reconstruction, and applies masked high-frequency residual targets.

    Source: Introduction · p. 2, Overview · p. 3, Reconstruction Target · p. 5

  • Results

    90.6% average AUC across five benchmark datasets, the highest among compared approaches in cross-dataset evaluation.

    Source: Cross-Dataset Evaluation · p. 5

  • Takeaways & Limitations

    Diverse manipulation artifacts can contain complementary forensic cues that the framework preserves and exploits for transferable representations.

    Source: Introduction · p. 1, Conclusion · p. 7

Abstract

from arXiv · show

Existing image forgery detectors often suffer from generalization to unseen manipulation methods due to the limited ability to capture transferable forensic cues. Recent cross-reconstruction based methods attempt to improve generalization through semantic-artifact disentanglement, but typically align heterogeneous artifacts across generators and exclude artifact representations during reconstruction, which may overlook the inherent diversity and visual cues of manipulation artifacts. In this work, we revisit cross-reconstruction and introduce an artifact-oriented disentanglement framework for robust image forgery detection. We argue that \textbf{artifact diversity}, i.e., the intrinsic variations of manipulation artifacts introduced by different generation processes, contains complementary forensic cues rather than undesirable domain variations. Instead of enforcing explicit artifact alignment, our framework preserves diverse artifact characteristics through semantically aligned cross-generator reconstruction. Furthermore, we incorporate artifact representations into the reconstruction process and introduce a masked frequency-aware reconstruction strategy to emphasize manipulation-related residuals while reducing semantic interference. This design enables the model to learn transferable forensic representations from diverse artifacts. Extensive experiments on multiple benchmark datasets demonstrate improvements under both cross-dataset and cross-generator evaluation settings. Further analysis and ablation studies validate the effectiveness of artifact diversity preservation and artifact-aware cross-reconstruction.

Introduction

Artifact diversity comprises generator-specific variations that can provide complementary forensic cues, so aligning artifacts may discard useful information. The paper proposes preserving those variations through artifact-aware cross-reconstruction and masked residual supervision.

Introduction

  • Artifact diversity—the intrinsic variation of manipulation artifacts across generation processes—can provide complementary forensic cues rather than undesirable domain variation.Examples include color inconsistency and unnatural facial details such as teeth.Source: Introduction · p. 1, Figure 1 · p. 1
  • Aligning artifacts across generators may suppress transferable cues, bias representations toward training-generator statistics, and impair generalization to unseen manipulations.Existing methods also exclude artifact representations from reconstruction, despite their meaningful visual information.Source: Introduction · pp. 1–2
  • The framework preserves artifact diversity using semantically aligned cross-generator pairs and incorporates artifact representations into reconstruction.Masked high-frequency residual targets emphasize manipulation-related details while reducing semantic interference.Source: Introduction · p. 2

Related Works

Prior work explores artifact features, pretrained vision models, and disentanglement to improve deepfake detection generalization. Reconstruction-based approaches separate semantic and forgery representations, but their objectives may not specifically emphasize forensic cues.

Generalizable Deepfake Detection

  • Artifact-focused features, pretrained vision models, and semantic-artifact disentanglement are among strategies for generalizing to unseen methods and datasets.Prior work also separates generator-specific and shared forensic components, an approach that assumes a shared artifact space can be learned from limited seen generators.Source: Generalizable Deepfake Detection · p. 2

Reconstruction Based Representation Learning in Deepfake Detection

  • Reconstruction methods disentangle representations; deepfake approaches use cross-reconstruction for shared forgery features, demographic disentanglement, or multi-scale semantic-artifact separation.The cited approaches include UCF, PFG, and CFB.Source: Reconstruction Based Representation… · p. 2
  • Existing reconstruction objectives are not tailored to forensic learning, and RGB reconstruction may provide suboptimal supervision because forgery artifacts often appear in high-frequency components.Source: Reconstruction Based Representation… · p. 2

Method

The framework preserves diverse artifacts through semantically aligned cross-generator reconstruction, while using artifact-aware classification and masked high-frequency targets to learn transferable forensic representations.

Overview

  • The framework extracts semantic and artifact features, exchanges artifact representations between paired samples, and reconstructs masked high-frequency targets.A frozen CLIP visual encoder uses artifact-aware attention biases for forgery prediction, while classification, artifact contrastive, and cross-reconstruction losses jointly optimize the framework.Source: Overview · p. 3, Figure 3 · p. 4, Decoder · p. 4

Training Pair Formation

  • Semantically aligned samples from different manipulation methods are paired so their facial pose, expression, and background remain consistent while their generators differ.Training pairs are formed within FF++ groups: pristine with reenactment, or face swapping separately.Source: Training Pair Formation · p. 3

Diversity Preserving Cross-Reconstruction

  • Artifact queries and visual tokens form spatially corresponding semantic and artifact features, which the decoder uses for cross-reconstruction while preserving semantic representations.The framework assigns separate semantic and artifact query tokens; AdaIN injects exchanged artifact representations into semantic representations.Source: Encoder · p. 3, Decouple Head · p. 3, Decoder · p. 4
  • The reconstruction target suppresses low-frequency image content, and a union mask restricts supervision to paired manipulation regions where artifacts may occur.The joint training objective optimizes classification, artifact contrastive learning, and cross-reconstruction.Source: Reconstruction Target · pp. 4–5, Overall Objectives · p. 5

Overall Objectives

  • Three hyperparameters control the contributions of classification, artifact contrastive learning, and cross-reconstruction losses.The overall objective jointly optimizes all three capabilities.Source: Overall Objectives · p. 5

Experiments

The model is trained on FF++ and evaluated across unseen datasets and generators, as well as at video level. Results and ablations support artifact-aware cross-reconstruction for generalizable deepfake detection.

Settings

  • Training uses FF++ c23; the evaluation spans five unseen benchmarks and four full-face synthesis datasets.The implementation uses ViT-Tiny/16 for visual feature extraction and frozen CLIP ViT-L/14 for classification.Source: Dataset · p. 5, Implementation Details · p. 5

Evaluations

  • 90.6% average AUC across five benchmarks is the highest overall among compared methods; CDF-v1 and CDF-v2 AUCs are 93.0% and 90.3%.The evaluation trains on FF++ and tests on unseen datasets.Source: Cross-Dataset Evaluation · p. 5, Table 1 · p. 6
  • 95.1% AUC on CDF-v2 and 89.5% on DFDC are state-of-the-art video-level results.The evaluation tests whether frame-level representations generalize to video-level prediction.Source: Cross-Dataset Evaluation · pp. 5–6, Table 2 · p. 6
  • On unseen diffusion generators SiT and DiT, the method outperforms existing approaches, while its GAN performance is comparable.The comparison covers frame- and video-level AUC across VQGAN, StyleGAN, SiT, and DiT.Source: Cross-Generator Evaluation · p. 6, Table 3 · p. 6

Ablation Study

  • Cross-reconstruction loss provides the dominant improvement: CDF-v1 AUC rises from 85.6% to 91.4%, and CDF-v2 from 85.3% to 89.3%.Removing both cross-reconstruction and contrastive losses causes a clear performance drop.Source: Impact of Loss Functions · p. 6, Table 4 · p. 7
  • Semantically identical fake-fake pairs raise CDF-v1 AUC from 87.1% to 90.5% versus real-fake pairs; residual reconstruction further improves generalization.Combining fake-fake pairs with residual reconstruction achieves the highest AUC on four of five benchmarks and raises CDF-v1 AUC from 87.1% to 93.0%.Source: Impact of Cross-Reconstruction Settings · p. 7, Table 5 · p. 7

Visualization

Conclusion

The framework revisits cross-reconstruction by preserving diverse manipulation artifacts as complementary forensic cues. It incorporates artifact representations into semantically aligned cross-generator reconstruction and uses masked high-frequency residual targets; experiments cover cross-dataset and cross-generator evaluation.

  • The framework treats diverse manipulation artifacts as complementary forensic cues to preserve and exploit.It proposes artifact-aware disentanglement for image forgery detection.Source: Conclusion · p. 7
  • Cross-generator reconstruction incorporates artifact representations and uses masked high-frequency residual targets to learn transferable forensic representations while maintaining artifact diversity.Source: Conclusion · p. 7
  • Experiments on multiple benchmarks demonstrate effectiveness under both cross-dataset and cross-generator evaluation.Source: Conclusion · p. 7

Related papers

Loading 2610.01544v1…