SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model

Introduction

• Generative AI [9] can produce highly realistic images resembling real-world events, enabling deceptive content that may mislead public opinion or distort historical records.The paper links these risks to malicious misuse of generated images.Source: §1 · p. 1
• Figure 1 contrasts prior methods limited to detection, localization, or both with SIDA’s combined detection, localization, and explanation capabilities.The paper proposes SIDA as a VLM-based framework that detects images, localizes potential manipulations, and explains its judgments.Source: Figure 1 · p. 1, §1 · p. 2
• Existing datasets often focus on faces [7] [24] or simple, non-social-media imagery using older generators, and commonly emphasize detection or localization over explanations [21] [22].The introduction identifies insufficient diversity and limited comprehensiveness as the two main challenges.Source: §1 · pp. 1–2
• SIDA achieves state-of-the-art performance on SID-Set, generalizes effectively across other benchmarks, and demonstrates superior or equivalent performance across SID-Set and other benchmarks.The paper presents SIDA as a baseline model for SID-Set and other benchmarks.Source: §1 · p. 2
• SID-Set contains 300K images—100K real, 100K synthetic, and 100K tampered—with extensive annotations and textual descriptions of the model’s judgment basis.The images include subtle alterations, natural-looking local manipulations, and complex scenes, and the synthetic and tampered examples are described as indistinguishable to humans.Source: §1 · p. 2

2.1. Image Deepfake Datasets

• SID-Set targets social-media images with recent state-of-the-art generation models, high-quality images, and more comprehensive, diverse annotations than prior deepfake datasets.Earlier datasets include facial-focused ForgeryNet [24], Deep-FakeFace, and DFFD [7], as well as non-facial datasets such as GenImage, HiFi-IFDL [21], and DiffForensics.Source: §2.1 · p. 2

2.2. Image Deepfake Detection and Localization

• Existing detection methods use classification, spatial or frequency features, and varied training strategies, but generalization remains difficult and large public datasets for realistic social-media localization are scarce.Prior work has also used masks to localize tampered regions, but such datasets are concentrated on facial data; this motivates a dataset covering diverse manipulations beyond faces.Source: §2.2 · p. 3

2.3. Large Multimodal Models

• LLMs [15] [16] and VLMs [15] [16] [30] support multimodal comprehension; LLaMA [15] [16], LLaVA [36] [37], and LISA [30] [33] [70] advance language, visual question answering, and segmentation.Grounding large multimodal models localize content from linguistic information.Source: §2.3 · p. 3
• AntifakePrompt [5], ForgeryGPT, and FakeShield apply multimodal models to deepfake detection or localization, with FakeShield also providing interpretable insights.SIDA distinguishes itself by pairing a social-media-specific, comprehensively annotated dataset with VLM-based detection, forgery localization, and explanations.Source: §2.3 · p. 3

3.1. Motivation

• SID-Set addresses prior datasets’ limited realism, narrow focus on full-image generation, and lack of authenticity criteria by covering subtle tampering and both synthetic and tampered images.These gaps can hinder detection of manipulations to particular regions or objects and the interpretability of authenticity judgments.Source: §3.1 · pp. 3–4

3.2. Benchmark Construction

• Tampered images are generated by extracting caption objects with GPT-4o, creating masks with Language-SAM, defining replacement and attribute-modification dictionaries, then regenerating images with Latent Diffusion.The pipeline uses COCO classes for object replacement and attribute modification.Source: Figure 3 · p. 4, §3.2 · p. 4
• SID-Set contains 100K real, 100K highly realistic synthetic, and 100K tampered images to evaluate image-type classification and altered-region localization.The real images come from OpenImages V71, and the synthetic images were generated with FLUX.Source: §3.2 · p. 4
• After five human experts reviewed 1,000 images from each generative model, FLUX emerged as top performer, producing 100K synthetic images based on Flickr30k and COCO that experts found indistinguishable from real ones.Source: §3.2 · p. 4
• GPT-4o generated judgment-basis descriptions for 3,000 SID-Set images, equally divided across the three image types, and five annotation experts quality-checked the images and descriptions.The descriptions support SIDA’s explainability.Source: §3.2 · p. 5

4.1. Architecture

• SIDA adds <DET> and <SEG> tokens to a vision-language model, which produces a textual description and classifies images as real, fully synthetic, or tampered.The model receives an image and a prompt asking for authenticity classification and, if tampered, a mask of the altered region.Source: §4.1 · p. 5
• For tampered images, SIDA fuses detection and segmentation features with single-layer multihead attention and a residual connection, then decodes them with frozen-encoder visual features into a mask.Detection features serve as the attention query, while segmentation features serve as key and value.Source: §4.1 · pp. 5–6

4.2. Training

• Training first uses cross-entropy detection and weighted BCE–DICE mask losses end-to-end, then fine-tunes text generation against detailed descriptions from 3,000 images.The total loss balances detection, segmentation, and text-generation components.Source: §4.2 · p. 6
• SIDA is trained on 300k SID-Set images combined with quality-filtered MagicBrush data, with LLM-generated descriptions for 3,000 randomly selected images.The combined data supports training for detection and localization of synthetic content.Source: §4.2 · p. 6

5.1. Detection Evaluation

• SIDA achieves better or comparable SID-Set detection results across evaluated methods; retrained LGrad leads tampered-image accuracy and F1 but performs worse on other metrics.The authors attribute LGrad’s high recall and false positives to misclassifying other types as tampered.Source: §5.1 · p. 6, Table 2 · p. 7

5.2. Localization Results

• SIDA achieves the best SID-Set forgery-localization performance among the compared methods.The authors suggest LISA’s general segmentation abilities lack specialized features for subtle manipulations, limiting fine-tuned localization.Source: §5.2 · p. 7, Table 3 · p. 7

5.3. Robustness Study

• Without training on degraded data, SIDA remains resilient to JPEG compression, resizing, and Gaussian noise across six tested scenarios.The scenarios use JPEG quality levels 70 and 80, resizing scales 0.5 and 0.75, and Gaussian-noise variances 5 and 10.Source: §5.3 · p. 7, Table 4 · p. 7

5.4. Test on Other Benchmark

• On DMimage, SIDA achieves superior performance against the compared deepfake-detection methods using their original hyperparameters and pretrained weights.The comparison uses equal numbers of real and fake images sampled from DMimage.Source: §5.4 · p. 7, Table 5 · p. 8

5.5. Ablation Study

• Removing SIDA’s attention module or replacing it with fully connected layers significantly reduces performance, supporting attention’s role in feature interaction and detection and localization accuracy.The ablation compares both variants with the attention-module model.Source: §5.5 · pp. 7–8, Table 6 · p. 8
• SIDA’s experiments set λdet to 1, λbce to 2.0, and λdice to 0.5 to balance detection and localization losses and maintain stability.The paper summarizes results for different weight configurations in Table 7.Source: §5.5 · p. 8, Table 7 · p. 8

5.6. Qualitative Results

• Qualitative examples show SIDA identifying tampered regions and explaining its decisions, alongside challenging cases where it fails to localize the tampering accurately.Additional visualization results are reported in the appendix.Source: §5.6 · p. 8, Figure 6 · p. 8

6. Conclusion and Discussions

• SID-Set contains 100k real, 100k fully synthetic, and 100k tampered images, and the paper proposes SIDA, a VLM-based framework for deepfake tasks.The authors describe VLM integration as a promising avenue for future research.Source: §6 · p. 8
• SIDA detects fake types, localizes tampered regions, and explains the criteria behind its decisions.Source: §6 · p. 8
• The authors plan to expand SID-Set and add generation techniques beyond FLUX, whose exclusive use may skew data and impair performance on diverse datasets.They report that skew was not significantly evident in their experiments.Source: §6 · pp. 8–9
• SIDA performs relatively strongly on the authors’ dataset, but some tampered regions are not reliably detected and localization needs improvement.Source: §6 · p. 9

7. Appendix

• Appendix Sections A–C provide experimental settings and configurations, detailed related-work comparisons, and additional SIDA examples, including failures and their analysis.Source: §7 · p. 9
• Appendix Sections D and E detail dataset creation and report experts’ and human evaluation.Source: §7 · p. 9

A. Experiment Settings

• AIGCDetecBenchmark2 GitHub tested or retrained CnnSpot, FreDect [18], Fusing [27], Gram-Net [39], UnivFD, LGrad, and LNP [2]; AntifakePrompt [5] used official settings.Detection methods used classification labels only because they could not handle localization; AntifakePrompt performance was recorded across classes.Source: A. Experiment Settings · p. 9
• CNNSpot, FreDect [18], Fusing [27], and Gram-Net [39] used blur and JPEG augmentation probabilities of 0.1, batch size 64, crop size 224, and Adam.Testing noise was set to None; augmentation used blur sigma 0.0–3.0 and JPEG quality 30–100. LNP [2] and LGrad used zero blur and JPEG probabilities, while UnivFD followed CNNSpot settings after preprocessing.Source: A. Experiment Settings · p. 9
• Detection methods trained for 10 epochs on one NVIDIA A100 40GB GPU, taking approximately 36 hours without preprocessing or 48 hours with preprocessing.LGrad [61], LNP [2], and UnivFD [48] required preprocessing; the other methods did not.Source: A. Experiment Settings · p. 9
• MVSS-Net and HIFI-Net [21] used pretrained models, PSCC-Net [38] followed official GitHub training settings, and LISA [30] used LISA-7B-v1 fine-tuned on SID-Set.LISA fine-tuning used learning rate 0.0001, batch size 2, and 10 gradient accumulation steps.Source: A. Experiment Settings · p. 9

B. Detailed Comparison

• Unlike detection methods [2] [5] [18] [27] [39] that often target specific generators, SIDA handles varied manipulations and provides both detection and localization.The paper says this broader focus supports generalization across datasets and manipulations.Source: B. Detailed Comparison · p. 9
• Compared with IFDL methods [21] [38], SIDA covers fully synthetic, tampered, and real images and uses LLMs to pair localization masks with detailed explanations.The paper describes this combination as adding interpretability to localization results.Source: B. Detailed Comparison · pp. 9–10
• Against FFAA [25], FakeShield, and ForgeryGPT, SIDA emphasizes social-media manipulations beyond faces, more realistic images, fine-grained outputs, and a large-scale dataset.The paper identifies object manipulation and partial tampering as examples of scenarios beyond facial deepfakes.Source: B. Detailed Comparison · p. 10
• After retraining PSCC-Net [38] and LISA [30] on SID-Set for 10 epochs, SIDA produced more precise and clearer tampered-area borders.The comparison uses their localization outputs, shown in Figure 8.Source: B. Detailed Comparison · p. 10

C. Additional Visual Examples

• Figures 9 and 10 show SIDA outputs for tampered images, while Figure 11 presents failure cases.These are additional visual examples of SIDA.Source: C. Additional Visual Examples · p. 10
• Figure 11 shows failures to detect tampered areas, including cases with no mask output, and failures to produce fine-grained masks.The first row illustrates missed or absent masks; the second row shows masks that lack detail.Source: C. Additional Visual Examples · p. 10
• The authors attribute these limitations to possibly insufficient training data—SID-Set has 100k tampered images—and difficulty with subtle or intricate tampering, and propose improving data and methods.They aim to improve training-data quality and quantity and develop more sophisticated methodologies and enhancement strategies.Source: C. Additional Visual Examples · p. 10

D. Detailed DataSet Creation Process

• GPT-4o prompts were designed separately for real, fully synthetic, and tampered images, with generated description examples shown for each category.Source: D. Detailed DataSet Creation Process · p. 10
• Fully synthetic images were generated with FLUX5 using Flickr30k and COCO [35] data, a specified style and negative prompt, and two NVIDIA A100 GPUs.Source: D. Detailed DataSet Creation Process · pp. 10–11
• Tampered images were created in separate directories using object replacement, such as swapping similar-class objects, and attribute replacement, such as changing an animal’s emotion or activity.Source: D. Detailed DataSet Creation Process · p. 11
• GPT-4o generated ground-truth descriptions for real images with prompts, synthetic images with corresponding prompts, and tampered images with prompts and tampered masks; image segments from Magicbrush added diversity.Source: D. Detailed DataSet Creation Process · p. 11

E. Experts and Human Evaluation

• Experts selected FLUX and latent-diffusion after examining approximately 1,000 images for output consistency and quality.Source: E. Experts and Human Evaluation · p. 11
• Experts rated image realism on a scale from 0 to 5; images scoring below 3 received secondary review, and identified flawed images were excluded.The paper describes this as a five-point rating scale.Source: E. Experts and Human Evaluation · p. 11
• Experts reviewed 3,000 GPT-4-generated descriptions for accuracy, clarity, and consistency with similar prompts or scenarios across the dataset.Source: E. Experts and Human Evaluation · pp. 11–12
• The provided prompt asks an AI visual-authenticity expert to analyze a fully synthetic or whole-deepfake image and explain its artificiality using technical and human-centered perspectives.Source: E. Experts and Human Evaluation · p. 17

• Figures 13–14 present prompts for fully synthetic images and tampered images.Source: Figure 13 · p. 17, Figure 14 · p. 18
• Figures 15–17 show SIDA analysis outputs for a real image, a fully synthetic image, and a tampered image.Source: Figure 15 · p. 19, Figure 16 · p. 20, Figure 17 · p. 21
• Figures 18–19 show SID-Set directories for object replacement and attribute modification.Source: Figure 18 · p. 22, Figure 19 · p. 22