PaperlayerView exact revision on arXiv

Source-linked AI summary

FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models

Zhipei Xu, Xuanyu Zhang, Runyi Li, Zecheng Tang, Qing Huang, Jian Zhang

arXiv:2410.02761v4cs.CVcs.AI

TL;DR

Existing IFDL methods can be black-box and limited across diverse tampering types. FakeShield combines authenticity assessment, masks, and textual rationale, and the paper reports it surpasses most previous IFDL methods across multiple tampering types.

  • Problem

    Existing IFDL methods often provide no explanation of their detection principles and are typically limited to particular tampering techniques.

    Source: §1 · p. 2

  • Method

    FakeShield pairs image-authenticity judgments and tampered-region masks with rationale, using GPT-4o-enriched MMTD-Set data and domain-tagged detection and multimodal localization modules.

    Source: §1 · pp. 2–3

  • Results

    FakeShield accurately analyzes tampering clues and surpasses most previous IFDL methods in detection and localization across copy-move, splicing, removal, DeepFake, and AIGC-based editing.

    Source: §1 · p. 3

  • Takeaways & Limitations

    FakeShield offers an explainable detection-and-localization approach across the varied tampering types reported in the paper.

    Source: §1 · pp. 2–3

    The framework performs suboptimally on complex DeepFake manipulations such as identity switching and full-face generation.

    Source: §A.1 · p. 16

Abstract

from arXiv · show

The rapid development of generative AI is a double-edged sword, which not only facilitates content creation but also makes image manipulation easier and more difficult to detect. Although current image forgery detection and localization (IFDL) methods are generally effective, they tend to face two challenges: \textbf{1)} black-box nature with unknown detection principle, \textbf{2)} limited generalization across diverse tampering methods (e.g., Photoshop, DeepFake, AIGC-Editing). To address these issues, we propose the explainable IFDL task and design FakeShield, a multi-modal framework capable of evaluating image authenticity, generating tampered region masks, and providing a judgment basis based on pixel-level and image-level tampering clues. Additionally, we leverage GPT-4o to enhance existing IFDL datasets, creating the Multi-Modal Tamper Description dataSet (MMTD-Set) for training FakeShield's tampering analysis capabilities. Meanwhile, we incorporate a Domain Tag-guided Explainable Forgery Detection Module (DTE-FDM) and a Multi-modal Forgery Localization Module (MFLM) to address various types of tamper detection interpretation and achieve forgery localization guided by detailed textual descriptions. Extensive experiments demonstrate that FakeShield effectively detects and localizes various tampering techniques, offering an explainable and superior solution compared to previous IFDL methods. The code is available at https://github.com/zhipeixu/FakeShield.

1 INTRODUCTION

Existing IFDL methods provide limited explanations and often handle only one tampering type, despite the diversity of real-world manipulations. FakeShield pairs authenticity judgments and tamper masks with textual rationale, using GPT-4o-enriched data and dedicated detection and localization modules.

1 INTRODUCTION

  • Prior IFDL methods are often black boxes and limited to particular tampering techniques, leaving users to reassess images and select detectors themselves.The paper identifies Photoshop, DeepFake, and AIGC editing among the diverse real-world techniques.Source: §1 · pp. 1–2
  • FakeShield's explainable IFDL task combines image-authenticity evaluation, suspected-region masks, and rationale from pixel-level artifacts and image-level semantic errors.The framework decouples detection and localization while providing a judgment basis.Source: §1 · p. 2
  • GPT-4o enriches existing IFDL datasets into MMTD-Set image-mask-description triplets focused on distinct features of different tampering types.The descriptions provide analysis of edited regions alongside tampered images and masks.Source: §1 · p. 2
  • DTE-FDM uses domain tags to address different forgery types, while MFLM aligns visual-language features to pinpoint tampered areas.The modules are presented as addressing data-domain conflict and text-guided localization, respectively.Source: §1 · p. 3
  • FakeShield accurately analyzes tampering clues and surpasses most previous IFDL methods in detection and localization across copy-move, splicing, removal, DeepFake, and AIGC editing.The paper reports this result across many tampering types.Source: §1 · p. 3

2 RELATED WORKS

Prior work spans manipulation-specific and broader IFDL methods, while multimodal LLM research includes visual understanding and text-guided masks. The paper identifies missing explainability and limited cross-domain coverage in these research strands.

2.1 IMAGE FORGERY DETECTION AND LOCALIZATION

  • Prior IFDL work includes methods for specific manipulation types and universal localization, but the surveyed methods do not explain their judgments and show performance disparities across test domains.The review also notes that these methods offer no interaction.Source: §2.1 · p. 3

2.2 LARGE LANGUAGE MODEL

  • M-LLM research includes text-to-mask grounding (Lai et al., 2024; Rasheed et al., 2024), while DeepFake-focused approaches cannot generalize to Photoshop and AIGC editing or accurately locate tampered areas (Huang et al., 2024).The review describes universal tamper detection and localization using M-LLMs as unexplored.Source: §2.2 · p. 3
  • The MMTD-Set construction illustration shows tampered image-mask pairs from Photoshop, DeepFake, and AIGC benchmarks used with domain tags to guide GPT-4o's judgment basis.The descriptions focus on pixel-level details and image-level content.Source: Figure 2 · p. 4

3 METHODOLOGY

FakeShield builds GPT-4o-generated tamper descriptions into a dataset and separates image detection and explanation from mask localization. Its domain-tagged detection module and text-guided localization module are trained with distinct objectives.

3.1 CONSTRUCTION OF THE PROPOSED MMTD-SET

  • MMTD-Set pairs authentic and tampered images across Photoshop, DeepFake, and AIGC-Editing with GPT-4o descriptions guided by type-specific prompts.Descriptions cover detection results, tampered-area location and content, and visible artifacts or semantic errors.Source: §3.1 · p. 4

3.2 OVERALL FRAMEWORK OF FAKESHIELD

  • FakeShield separates detection and explanation from localization: DTE-FDM produces Odet, then MFLM uses it with image features to generate Mloc.The framework is decoupled because joint training of the tasks increases optimization difficulty and causes interference between them.Source: §3.2 · p. 5, Figure 3 · p. 5

3.3 DOMAIN TAG-GUIDED EXPLAINABLE FORGERY DETECTION MODULE

  • DTE-FDM assigns one of three domain tags and feeds it with image tokens and an instruction to an LLM that returns detection results, location, and rationale.The LLM is frozen and fine-tuned with LoRA for efficiency.Source: §3.3 · pp. 5–6

3.4 MULTI-MODAL FORGERY LOCALIZATION MODULE

  • MFLM sends the tamper description and image tokens through TCM, whose <SEG> embedding prompts SAM to generate a tamper mask.TCM aligns long-text features with visual modalities to improve SAM's localization of forgery areas.Source: §3.4 · p. 6

3.5 TRAINING OBJECTIVES

  • Training uses cross-entropy for domain-tagged detection and prompt generation, and BCE plus Dice losses for masks; Table 1 reports best detection accuracy across three tampered-data domains.The detection objective balances LLM output and domain-tag prediction losses with λ.Source: §3.5 · pp. 6–7, Table 1 · p. 7

4 EXPERIMENT

Experiments report detection across Photoshop, DeepFake, and AIGC-Editing, alongside explanation and localization results. Robustness tests and ablations assess degradations, domain tags, and the decoupled module design.

4.1 EXPERIMENTAL SETUP

  • Evaluation covers Photoshop, DeepFake, and AIGC-Editing datasets, using ACC/F1 for detection, IoU/F1 for localization, and CSS for interpretability.The IFDL baselines are retrained on MMTD-Set; training uses staged LoRA fine-tuning and a full-parameter Domain Tag Generator.Source: §4.1 · p. 7

4.2 COMPARISON WITH IMAGE FORGERY DETECTION METHOD

  • FakeShield reports ACC 0.08 and F1 0.05 against MVSS-Net on IMD2020, with detection accuracy of 0.93 on DeepFake and 0.98 on AIGC.The comparison also reports that FakeShield significantly outperforms other methods on DFFD and Seq-DeepFake.Source: Figure 4 · p. 8, §4.2 · p. 8, Table 2 · p. 8

4.3 COMPARISON WITH M-LLMS

  • FakeShield achieves the best explanation CSS across nearly all test sets, including 0.8873 on DSO, surpassing InternVL2-26B.The descriptions are compared against GPT-4o-generated tampering descriptions used as ground truth.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 COMPARISON WITH IMAGE FORGERY LOCATION METHOD

  • Figure 5 shows clean, complete masks, while PSCC-Net produces dispersed attention, blurred segmentations, and overly broad tampering areas.The examples span IMD2020, CASIA1+, Columbia, NIST16, Korus, DSO, and DeepFake.Source: Figure 5 · p. 9, §4.4 · p. 9
  • IoU 0.12 and F1 0.1 advantages over OSN on IMD2020; on CASIA1+, FakeShield leads OSN with IoU 0.07 and F1 0.09.Across almost all test datasets, FakeShield surpasses other localization methods.Source: §4.4 · p. 9, Table 4 · p. 9

4.5 ROBUSTNESS STUDY

  • JPEG compression and Gaussian noise, including noise variances 5 and 10, have minimal effect on FakeShield’s performance despite degraded data not being specifically added during training.The evaluation reports explanation and location performance under different degradations.Source: §4.5 · p. 10, Table 5 · p. 10

4.6 ABLATION STUDY

  • Removing the Domain Tag Generator lowers detection across domains, including IMD2020 ACC by 0.12 and F1 by 0.11.The authors attribute the declines to greater difficulty differentiating domains and more pronounced data conflicts without DTG.Source: §4.6 · p. 10
  • Without the LLM, localization lags throughout training and converges earlier than the original framework on CASIA1+.The ablation compares a joint MFLM setup against the framework's decoupled module design.Source: §4.6 · p. 10, Figure 6 · p. 10

5 CONCLUTION

The conclusion presents FakeShield as an M-LLM application for explainable IFDL, combining detection, explanations, and localization across manipulation types.

5 CONCLUTION

  • The authors present FakeShield as the first application of an M-LLM to explainable IFDL.They describe this as an advancement in the field.Source: §5 · p. 10
  • FakeShield is described as excelling at tampering detection while providing comprehensive explanations, precise localization, and generalization across manipulation types.The conclusion characterizes it as versatile for diverse real-world applications.Source: §5 · p. 10
  • The authors suggest FakeShield could support digital-manipulation policy, legal evidence collection, and efforts to correct misinformation.They connect these potential uses with a clearer and more trustworthy online environment.Source: §5 · p. 10

A.1 LIMITATIONS AND FUTURE WORKS

The authors identify complex deepfake manipulations as a current weakness and outline dataset and model changes for future work.

A.1 LIMITATIONS AND FUTURE WORKS

  • FakeShield performs suboptimally on complex deepfakes, particularly identity switching and full-face generation.The authors state that these manipulation types introduce challenges the model currently struggles to address.Source: §A.1 · p. 16
  • The authors plan to add Chain-of-Thought reasoning to help detect subtler deepfake manipulations.This is proposed as an enhancement to the model's reasoning abilities.Source: §A.1 · p. 16
  • Future work will broaden the deepfake training data and optimize framework modules for complex manipulation types.The stated goal is improved generalization across a wider range of deepfake domains.Source: §A.1 · p. 16

A.2 DATA SOURCES

The data sources include real and manipulated face images and a constructed AIGC-editing set, with dataset details summarized for training and evaluation.

A.2 DATA SOURCES

  • Table 7 summarizes the datasets used for training and evaluation, drawing on the sources described in Section 4.1.Source: §A.2 · p. 16, Table 7 · p. 16
  • FaceApp and FFHQ are part of DFFD; the authors follow the original configuration for their training and validation split.FFHQ contains real face images, while FaceApp contains fake faces generated by its facial-attribute manipulation tool.Source: §A.2 · p. 16
  • The AIGC-Editing construction uses 20,000 COCO images for partial Stable-Diffusion-Inpainting edits and a separate 20,000 real images for balance.SAM segments targets, and the target with the third-largest area is selected for editing.Source: §A.2 · p. 16

A.3 MORE ABLATION EXPERIMENTS

The ablations examine whether correcting detection descriptions improves localization and how alternative language-model inputs affect it. Using the analyzed tampering basis with the image performs better than the tested alternatives, while explicit description correction does not improve localization predictions.

A.3 MORE ABLATION EXPERIMENTS

  • Correcting Odet during MFLM training does not improve Mloc predictions.Table 8 reports the localization ablation; the authors suggest interference between mask and text optimization as a possible explanation.Source: §A.3 · p. 17, Table 8 · p. 17
  • MFLM is observed to correct minor inaccuracies in Odet at test time, including incorrect tampered-location descriptions.The authors attribute this capability to training with potentially inaccurate Odet as input while predicting Mloc.Source: §A.3 · p. 17
  • Using {Odet, Timg} yields better localization results than either {Tins, Timg} or {Tins, Ttag}.The {Tins, Timg} combination is slightly better than {Tins, Ttag}; Table 9 presents the input-combination ablation.Source: §A.3 · p. 17, Table 9 · p. 17

A.4 MORE AIGC-EDITING GENERALIZATION EXPERIMENTS

The generalization test evaluates FakeShield on ControlNet and SDXL inpainting images not encountered during training. The authors report leading detection and localization performance on these unseen datasets.

A.4 MORE AIGC-EDITING GENERALIZATION EXPERIMENTS

  • FakeShield demonstrates leading detection and localization performance on unseen ControlNet and SDXL inpainting datasets.Table 10 presents the generalized performance comparison.Source: §A.4 · p. 17, Table 10 · p. 17
  • The expanded test set contains 2,000 tampered images created with ControlNet and SDXL inpainting, which the model had not encountered before.The comparison methods were MVSS-Net, CAT-Net, and HiFi-Net, selected for their AIGC-Editing-domain performance in Table 1.Source: §A.4 · p. 17

A.5 ANSWER ANALYSIS

FakeShield’s generated answers reflect the vocabulary found in MMTD-Set descriptions and address both semantic plausibility and pixel-level artifacts. The answer analysis highlights terms associated with each kind of tampering clue.

A.5 ANSWER ANALYSIS

  • FakeShield’s answers assess image-level semantic plausibility, including physical laws and texture.The analysis attributes this capability to fine-tuning guided by MMTD-Set descriptions.Source: §A.5 · p. 18
  • FakeShield’s answers also use pixel-level tampering artifacts, including edges and resolution, to assess whether an image is real.Figure 7 presents adjective and noun word clouds for MMTD-Set descriptions and FakeShield answers.Source: §A.5 · p. 18, Figure 7 · p. 18
  • The high-frequency noun and verb vocabulary in MMTD-Set descriptions is basically the same as in FakeShield’s generated answers.Figure 7 compares the word clouds for the dataset descriptions and generated answers.Source: Figure 7 · p. 18

A.6 PROMPTS

The appendix shows category-specific GPT-4o prompts and example inputs for MMTD-Set construction, alongside FakeShield dialogue examples across tampering types and authentic images. The examples also compare mainstream M-LLM responses and show sample MMTD-Set data.

A.6 PROMPTS

  • GPT-4o prompts are tailored to tampering categories, with guides for PhotoShop and AIGC-Editing, DeepFake, and authentic scene or face images.Figures 8 and 9 show the category-specific analysis prompts.Source: §A.6 · p. 18, Figure 8 · p. 19, Figure 9 · p. 20

A.7 EXAMPLES

  • In the cited examples, FakeShield alone correctly identifies both a spliced CASIA1+ image and a real IMD2020 image among the compared mainstream M-LLM responses.The spliced example has a black animal pasted at the bottom; the real-image example is from IMD2020.Source: §A.7 · p. 18, Figure 10 · p. 21, Figure 11 · p. 22
  • PhotoShop dialogue examples include CASIA1+ copy-move and splicing, Columbia splicing, IMD2020 copy-move, and authentic scene images.The examples identify pasted yellow petals, a penguin, an elliptical post-processing area, and a car copied from another car.Source: §A.7 · p. 18, Figure 12 · p. 23, Figure 13 · p. 24
  • DeepFake dialogue examples show DFFD faces tampered with by FaceAPP and authentic DFFD face images.The tampered examples add a beard, glasses, or hair on the right side of a face.Source: §A.7 · p. 18, Figure 15 · p. 26, Figure 16 · p. 27
  • AIGC-Editing dialogue examples use Stable-Diffusion-Inpainting images, including a redrawn window and clock area.The examples come from the paper’s self-generated AIGC-Editing dataset.Source: §A.7 · p. 18, Figure 17 · p. 28
  • MMTD-Set samples show both tampered and untampered images for PhotoShop, AIGC-Editing, and DeepFake data.The examples use CASIAv2, a self-generated AIGC-Editing dataset, and DFFD, respectively.Source: §A.7 · p. 18, Figure 18 · p. 29, Figure 19 · p. 30

A.6 PROMPTS

  • The PhotoShop/AIGC-Editing prompt provides a tampered image and binary mask, with white marking tampered pixels and black marking untampered pixels.The prompt asks for the tampering location, its contents, and visible altered details.Source: §A.7 · p. 19, Figure 8 · p. 19
  • The DeepFake prompt likewise supplies a face image and binary mask, then asks for the location, contents, and visible details of the tampered areas.Figure 9 includes guidance for DeepFake-tampered images and real face images.Source: §A.7 · p. 20, Figure 9 · p. 20

Related papers

Loading 2410.02761v4…