PaperlayerView exact revision on arXiv

Source-linked AI summary

FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant

Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, Wenming Yang, Jiaya Jia

arXiv:2408.10072v2cs.CVcs.AI

TL;DR

Open-world face forgery analysis is challenged by unknown techniques and varied image conditions, while binary outputs and heatmaps offer limited insight into model decisions. The paper introduces a VQA dataset and FFAA, combining a fine-tuned MLLM with MIDS and hypothetical prompts. Experiments report improved accuracy and robustness over previous methods, while the paper notes longer inference times and face-only scope.

  • Problem

    Unknown forgery techniques, varied facial attributes, and complex environments challenge open-world analysis, while binary results or heatmaps make model decisions difficult to understand.

    Source: §1 · p. 1

  • Method

    The paper introduces OW-FFA-VQA and FFA-VQA, then builds FFAA by fine-tuning an MLLM with hypothetical prompts and using MIDS to select among its responses.

    Source: §1 · pp. 1–2

  • Results

    Experiments report that FFAA improves accuracy and robustness over previous methods while providing explainable results.

    Source: §1 · p. 2

  • Takeaways & Limitations

    OW-FFA-VQA requires authenticity judgments with supporting analysis, making the decision process part of the face-forgery task output.

    Source: §1 · p. 1

    Detailed analysis and decision transparency lead to longer inference times and limited suitability for large-scale screening; the method is restricted to face images.

    Source: §6 · p. 8

Abstract

from arXiv · show

The rapid advancement of deepfake technologies has sparked widespread public concern, particularly as face forgery poses a serious threat to public information security. However, the unknown and diverse forgery techniques, varied facial features and complex environmental factors pose significant challenges for face forgery analysis. Existing datasets lack descriptive annotations of these aspects, making it difficult for models to distinguish between real and forged faces using only visual information amid various confounding factors. In addition, existing methods fail to yield user-friendly and explainable results, hindering the understanding of the model's decision-making process. To address these challenges, we introduce a novel Open-World Face Forgery Analysis VQA (OW-FFA-VQA) task and its corresponding benchmark. To tackle this task, we first establish a dataset featuring a diverse collection of real and forged face images with essential descriptions and reliable forgery reasoning. Based on this dataset, we introduce FFAA: Face Forgery Analysis Assistant, consisting of a fine-tuned Multimodal Large Language Model (MLLM) and Multi-answer Intelligent Decision System (MIDS). By integrating hypothetical prompts with MIDS, the impact of fuzzy classification boundaries is effectively mitigated, enhancing model robustness. Extensive experiments demonstrate that our method not only provides user-friendly and explainable results but also significantly boosts accuracy and robustness compared to previous methods.

1. Introduction

Open-world face forgery analysis must handle unknown techniques, varied facial attributes, and complex environments, while producing explanations beyond binary labels or heatmaps. The paper introduces a VQA task and FFAA, and reviews gaps in existing datasets, detectors, and MLLM approaches.

1. Introduction

  • Unknown forgery techniques, varied facial attributes, and complex environments challenge open-world accuracy and robustness, while binary outputs or heatmaps obscure decision-making.The paper frames these as central obstacles to reliable and understandable face forgery analysis.Source: §1 · p. 1
  • OW-FFA-VQA requires authenticity judgments and supporting analysis, and FFAA combines a fine-tuned MLLM with MIDS to select among responses under hypothetical prompts.The FFA-VQA dataset provides diverse real and forged images with descriptions and forgery reasoning.Source: §1 · pp. 1–2
  • Experiments report improved accuracy and robustness over previous methods, alongside user-friendly, explainable results.The evaluation includes OW-FFA-Bench, built from real and forged images across seven public datasets.Source: §1 · p. 2

2. Related Work

  • Existing face-forgery datasets lack necessary textual descriptions, while manual reasoning annotations are cumbersome, potentially biased, and insufficient for difficult open-world cases.The related-work discussion contrasts these limitations with DD-VQA’s manually annotated reasoning for faces discernible by common sense.Source: §2 · p. 2
  • Prior detectors show limited open-world generalization, and MLLM prompt-based face-forgery analysis has suboptimal accuracy; fine-tuning MLLMs for this task remains underexplored.Existing detectors also commonly provide binary outputs or heatmaps that are difficult to interpret.Source: §2 · p. 2

3. Dataset

The dataset is built in two stages: gathering diverse face-forgery images, then generating and scrutinizing structured analysis to create FFA-VQA. Its descriptions and reasoning are reported to help MLLMs reduce overfitting and improve generalization.

3.1. Data Collection

  • The Multi-attack dataset contains 95K images drawn from four source datasets and covers identity exchange, facial attribute manipulation, and entire face synthesis.The source datasets are FF++, Celeb-DF-v2, DFFD, and GanDiffFace.Source: §3.1 · pp. 2–3

3.2. GPT4-assisted Analysis Generation

  • GPT-4o generates structured face-forgery analyses from prompts, target images, and reference images; invalid or unreliable responses are filtered and experts scrutinize the remainder.Qualified analyses are organized into VQA format.Source: §3.2 · p. 3
  • FFA-CoT structures analysis as image description, forgery reasoning, and an analysis result, while quality checks filter erroneous, low-probability, or irrational responses.The analysis result includes classification, probability, and forgery type; experts assess rationality using the image and its authenticity and forgery type.Source: §3.2 · pp. 3–4
  • 20K high-quality analyses are supplemented with semantically identical inquiry prompts to form FFA-VQA, whose descriptions and reasoning are reported to reduce overfitting and enhance generalization.The paper reports this quality-verification finding for MLLMs fine-tuned on FFA-VQA.Source: §3.2 · p. 4

4. Method

FFAA addresses unstable open-world face judgments by fine-tuning an MLLM with hypothetical prompts and using MIDS to select the answer that best matches the image. Its method combines prompt-based analysis with image-answer fusion and classification.

4. Method

  • FFAA combines a hypothetical-prompt fine-tuned MLLM with MIDS, which selects among generated answers using image-answer match scores.The candidates include an anchor answer without a hypothesis and answers generated under varying hypotheses.Source: §4 · p. 4

4.1. Fine-tuning MLLM with Hypothetical Prompts

  • Fine-tuning uses non-hypothetical and hypothetical prompts; changing the real/fake hypothesis reveals hard cases where judgments vary, while easy cases retain their judgment.One-third of FFA-VQA conversations are randomly selected and modified to use a hypothesis aligned with the face’s authenticity.Source: §4.1 · p. 5

4.2. Multi-answer Intelligent Decision System

  • MIDS encodes answer text and image features, fuses them through dual cross-attention, and classifies image-answer pairs into four categories.The method masks the answer’s “Analysis result” when encoding text features, to discourage reliance on the final classification alone.Source: §4.2 · p. 5
  • Table 1 reports ACC/AUC (%) on OW-FFA-Bench and MA, while Table 2 presents OW-FFA-Bench data distribution and sources.The Table 1 caption identifies gray-shaded areas as FFAA without MIDS.Source: Table 1 · p. 6, Table 2 · p. 6

5. Experiments

FFAA is evaluated on in-domain and open-world face-forgery data, with comparisons, ablations, and qualitative analyses. The reported results emphasize improved accuracy, generalization, and robustness, alongside more explainable judgments.

5.1. Experimental Setup

  • Evaluation uses a 1K in-domain MA test set and OW-FFA-Bench; ACC and AUC measure performance, while sACC measures accuracy variability across test sets.The benchmark is described as a set of diverse real and forged face images from seven public datasets.Source: §5.2 · p. 6, §5.1 · p. 6, §5 · p. 6

5.2. Comparison with Competitive Methods

  • FFAA improves accuracy and robustness over prior methods while clarifying decision-making; the comparison reports limited cross-domain generalization and high accuracy variability for other method categories.The authors attribute FFAA’s robustness to comparing answers under different hypothetical prompts with MIDS.Source: §5.2 · pp. 6–7

5.3. Ablation Study

  • Adding image descriptions and forgery reasoning improves cross-domain generalization over analysis-results-only training, and complete FFA-CoT improves it further.The ablation compares R, DR, FR, and complete FFA-CoT on MA and OW-FFA-Bench.Source: §5.3 · pp. 7–8
  • MIDS significantly improves accuracy on hard images while preserving high accuracy on easy images, and its masking and visual-encoder design choices affect generalization and robustness.Without masking the analysis result, ACC is 78.2%, 8.3% below the masked scenario; unfreezing the final two CLIP-ViT layers helps only with masking.Source: Figure 6 · p. 7, §5.3 · p. 8

5.4. Qualitative Study

  • In real-world examples, FFAA provides multifaceted facial analysis and forgery clues, while the compared MLLMs are described as lacking evidential support or often avoiding definitive judgments.FFAA also supplies a match score and difficulty type as indicators of response reliability.Source: Figure 5 · p. 7, §5.4 · p. 8
  • Grad-CAM heatmaps indicate that MIDS attends to multiple image aspects and features related to face authenticity when guided by enriched textual features.Figure 7 pairs images and Fvl/Fvg heatmaps with key information from FFAA’s answer.Source: §5.4 · p. 8, Figure 7 · p. 8
  • FFAA’s hard samples often combine variable image quality with convincing forgeries, whereas easy fake samples show more obvious signs and easy real samples have higher image quality.Figure 8 displays easy and hard samples in separate rows.Source: §5.4 · p. 8, Figure 8 · p. 8

6. Conclusion

The conclusion presents FFAA as an explainable face-forgery analysis assistant built around a VQA task, dataset, and benchmark. It reports improved generalization, accuracy, and robustness, while noting inference-time and modality limits.

6. Conclusion

  • FFAA combines a fine-tuned MLLM and MIDS, trained using the GPT4-assisted FFA-VQA dataset of real and forged faces with descriptions and forgery reasoning.The paper introduces OW-FFA-VQA and a corresponding benchmark to support analysis of model decision-making.Source: §6 · p. 8
  • Experiments show improved generalization, accuracy, and robustness over previous methods, with user-friendly and explainable results.Source: §6 · p. 8
  • Detailed analysis and decision transparency lengthen inference time, limiting suitability for large-scale screening; the method is restricted to face images.The authors identify multimodal inputs and reducing inference time as future-work directions.Source: §6 · p. 8

7. Dataset Construction

The paper constructs FFA-VQA using GPT-4o-assisted generation and expert approval, and establishes OW-FFA-Bench from sampled face images. The two resources support training and open-world evaluation, respectively.

7. Dataset Construction

  • Algorithm 1 samples image-question prompts, obtains GPT-4o answers, filters for format, label, forgery type, and confidence, then retains answers approved by an expert.The final qualified examples form the VQA dataset.Source: §7 · p. 11
  • FFA-VQA extracts facial regions from images and video frames in public datasets, resizes them uniformly to 224×224, and randomly selects 2,000 images classified by forgery type according to source.Source: §7 · p. 11
  • FFA-VQA contains 20,142 VQA pairs across real, facial-attribute-manipulation, entire-face-synthesis, and identity-exchange categories.The category counts are 8,073, 4,014, 4,025, and 4,030, respectively.Source: §7 · p. 11
  • OW-FFA-Bench samples 1,000 images from each of its source datasets, cropping faces and resizing them to 224×224 for testing.The benchmark passage refers to source composition in Table 2; the supplied Table 4 caption identifies Multi-attack dataset composition.Source: §7 · p. 11, Table 4 · p. 11

8. Implementation Details

Implementation details cover fine-tuning the MLLM, training MIDS, and training the comparison baselines. The reported configurations specify hardware, optimization settings, and selected model parameters.

8. Implementation Details

  • LLaVAv1.6-mistral-7B [26] is fine-tuned on FFA-VQA with LoRA (rank 32, alpha 48) using two RTX 3090 GPUs, at learning rate 1e-4 and batch size 16.Source: §8 · p. 12
  • MIDS is trained on 30K mined image-answer pairs for two epochs using AdamW, two RTX 3090 GPUs, and a batch size of 48.Training uses learning rate 1e-4, weight decay 1e-5, and three self-attention iterations.Source: §8 · p. 12
  • Baseline training uses method-specific epoch or step counts, optimizers, learning rates, batch sizes, and weight decay settings.The supplied baseline details span ResNet-34, EfficientNet-B4, Xception, RECCE, DCL, Exposing, NPR, UIA-ViT, CLIPping, F3Net, Two-Stream, and FADD.Source: §8 · p. 12

9. Answer Analysis

Answers from FFAA and GPT-4o are semantically enriched and diverse, covering facial features, forgery characteristics, image quality, and physical-environment factors.

9. Answer Analysis

  • FFAA’s answers have semantic richness comparable to GPT-4o’s.The comparison is presented through a word cloud of nouns and adjectives in noun-verb and adjective-verb pairs.Source: §9 · p. 12
  • The answers include descriptions of facial features such as eyes and hair.Source: §9 · p. 12
  • The answers also describe forgery-related characteristics, image quality, and physical-environment factors.Examples include artifacts, sharpness, lighting, and resolution.Source: §9 · p. 12

10. Prompts

The paper presents four prompt components: system, forgery-specific, question, and reference prompts.

10. Prompts

  • The prompt setup includes system and forgery-specific prompts.Source: §10 · p. 12
  • The prompt setup also includes question and reference prompts.Source: §10 · p. 12
  • Together, the four components are shown in Figure 12.Source: §10 · p. 12

11. Examples

The examples illustrate FFA-VQA dataset annotations and FFAA dialogues across varied image conditions, including real-world faces with unknown forgery techniques. Additional heatmaps show the visual clues emphasized by MIDS.

11. Examples

  • FFAA dialogue examples cover varied lighting, image editing, generated images, low resolution, and evident forgery traces.The examples also include real-world celebrity face swaps and high-fidelity AIGC-generated faces made with unknown techniques.Source: §11 · p. 12, Figure 13 · p. 15, Figure 14 · p. 16
  • The examples illustrate FFAA providing evidence and identifying forgery types in cases involving unknown deepfake technologies.Real-world examples include celebrity face swaps and high-fidelity AIGC-generated faces.Source: §11 · p. 12, Figure 17 · p. 19, Figure 18 · p. 20
  • FFA-VQA examples pair face images with descriptions of facial features, image quality, and surroundings, plus forgery reasoning.Source: §11 · p. 12, Figure 11 · p. 13
  • MIDS heatmaps show Fvl focusing on specific features, surroundings, and local clues, while Fvg emphasizes abstract cues such as smoothness and lighting.In some examples, both heatmaps rely on the same features.Source: §11 · p. 12, Figure 21 · p. 23

Related papers

Loading 2408.10072v2…