Beyond Generation: Using Multimodal LLMs as Detectors of LLM-Generated Deepfake Facial Images

1.1 Background and Motivation

• The study asks whether high MLLM detection accuracy reflects genuine image-structure reasoning or reliance on shortcuts such as metadata and embedded watermarks.The authors identify insufficient current evidence to answer this question.Source: §1.1 · p. 2

1.2 Research Hypotheses

• The hypotheses predict variation across detector architectures, an advantage for pairwise classification, measurable watermark effects, and possible gaps between justifications and reasoning.The study focuses on fully synthetic facial images generated from text prompts rather than face swaps or identity manipulation.Source: §1.2 · p. 2

1.3 Research Objectives

• The objectives cover a 400-image corpus, six MLLM detectors, single-image and pairwise tests, Gemini watermark ablations, and manual audits of model justifications.The Gemini experiments remove visible logos and embedded SynthID watermarks to assess their effects on detection accuracy.Source: §1.3 · p. 3

1.4 Report Structure

• The report proceeds from literature review and methodology to results, discussion, and conclusion.The methodology includes dataset development and single-image, pairwise-image, and ablation experiments.Source: §1.4 · p. 3

2 Literature Review

• Generative AI can produce media difficult to detect, creating demand for MLLM-based approaches alongside conventional pixel-based methods.The review describes MLLMs as capable of processing text and images.Source: §2 · p. 3, §2.1 · p. 3

2.1 Evolution of Generative AI

• Generative models have become harder to detect, as early GAN-based outputs were more visibly identifiable than later high-resolution generated images.The review traces the development from language models and latent diffusion toward multimodal language models.Source: §2.1 · p. 3

2.2 Historical Overview of Deepfakes

• Deepfakes have shifted from visible face-swap artefacts to high-fidelity text-prompted synthetic images, motivating higher-level semantic detection.The study focuses on third- and fourth-generation synthetic media.Source: §2.2 · pp. 3–4

2.3 Automated Deepfake Detection

• No recent controlled ablation study had isolated whether MLLM detection accuracy reflects structural reasoning or watermark exploitation; this study addresses that gap.Prior findings also report large accuracy drops on high-quality diffusion images and sensitivity to slight image alterations.Source: §2.3 · p. 4

2.4 Watermarks and Their Importance

• Watermark signals may deteriorate under basic image alterations, and reliance on embedded watermarks alone leaves detection vulnerable to removal attacks.The review identifies SynthID as a prominent example of invisible image watermarking.Source: §2.4 · p. 4

2.5 Explainability and Justification Quality

• MLLM explanations can include confident but factually incorrect image descriptions, including distortions not present in the image.The review also flags shortcut use as a concern when models identify fakes.Source: §2.3 · p. 4, §2.5 · p. 5

2.6 Ethics, Diversity and Industry Context

• Dataset construction affects detector learning, while demographic underrepresentation and lighting bias can contribute to higher false-positive rates for underrepresented groups.The review notes that FFHQ contains approximately 70% Caucasian individuals.Source: §2.6 · p. 5

3.1 Dataset Construction

• The custom dataset contains 400 images: 350 synthetic images from seven generators and 50 authentic FFHQ photographs manually selected for demographic balance.The balancing addresses FFHQ’s reported demographic bias and aims to prevent detectors from using demographic features as a classification shortcut.Source: §3.1.1 · p. 6, §3.1 · p. 6, §3.1.3 · p. 7

3.1.1 Synthetic Corpus Generation.

• The 350 synthetic images were generated through official developer APIs to avoid UI-specific watermarks, interface alterations, and compression; Experiment 3 additionally used Gemini’s web interface to capture its visible logo watermark.The seven generators were Gemini 3, Gemini 2.5, Flux.2 Pro, GPT-Image-1, GPT-Image-1.5, Seedream 4.0, and Seedream 4.5.Source: §3.1.1 · p. 6

3.1.2 Prompt Matrix Design.

• The 50-prompt matrix varied age, gender, physical traits, ethnicity, and professional roles while using consistent photographic qualifiers to limit stereotypical outputs and demographic shortcuts.Examples of the shared qualifiers include natural window light and centred composition.Source: §3.1.2 · p. 7

3.1.3 Real Image Control Group.

• The FFHQ control group supplied high-resolution authentic portraits comparable to generated headshots, with manual selection used to balance age, gender, and ethnicity.The passage reports that base FFHQ is nearly 70% Caucasian and 4.2% Black.Source: §3.1.3 · p. 7

3.2 Experimental framework

• The study’s three experiments test single-image detection and justification reliability, pairwise image advantage, and the effects of visible and invisible watermarks.Experiment 1 establishes a zero-shot baseline; Experiment 2 compares performance with that baseline, and Experiment 3 isolates watermark signals.Source: §3.2 · p. 7
• Experiment 1 used zero-shot single-image classification, then collected natural-language justifications and manually checked them against visual evidence for correctness or hallucinated artefacts.The justification audit used ten fixed images, with five independent runs per image per model.Source: §3.2.1 · pp. 7–8

3.2.2 Experiment 2: Pairwise Image Analysis (Contextual Advantage).

• Experiment 2 used randomized real–synthetic image pairs in a two-alternative forced-choice task to test whether a real reference image improves detection accuracy.Image A/B placement was randomized to limit positional bias; the experiment did not include justification analysis.Source: §3.2.2 · p. 8
• Five Gemini pools and five comparisons were designed to isolate visible sparkle logos, invisible SynthID, and their combined effects on detection.The pools vary whether each marker is present, cropped, or disrupted; Figure 2 visualizes the five-pool design.Source: §3.2.3 · p. 9, Figure 2 · p. 10
• Datasets 4–5 used LDM image-to-image reconstruction at strength 0.15 to disrupt SynthID while preserving visual quality, facial biometrics, and scene metadata.The reconstruction used Realistic Vision V5.1 via Hugging Face Diffusers.Source: §3.2.3 · p. 9

3.3.1 Automation Pipeline.

• A custom Python pipeline called the OpenAI, Gemini, and Replicate APIs and converted extracted outputs into a binary correctness matrix using JSON schemas.Replicate hosted the LDM reconstruction models used for the watermark-ablation experiment.Source: §3.3.1 · p. 10

3.3.2 Parameter Standardisation and Bias Control.

• Image generation used temperature 0 where available, detectors generally retained default settings, Gemini justification used 0, prompts were uniform, and pairwise positions were shuffled.Position shuffling was intended to neutralize positional bias in the two-alternative forced-choice protocol.Source: §3.3.2 · pp. 10–11

3.4 Statistical Methodology

• Binary accuracy was analyzed with non-parametric tests at p<0.05: Mann–Whitney U for independent samples, Wilcoxon signed-rank for paired samples, and rank-biserial correlation for effect size.The tests addressed differences across model families and within-model changes under experimental conditions.Source: §3.4 · p. 11

4.1 Overview of Experimental Results

• Pairwise conditions brought performance gains in the GPT family and accuracy declines in the Gemini ecosystem; Table 1 summarizes shifts across all six detectors.Source: §4.1 · p. 11, Figure 4 · p. 12, Table 1 · p. 12

4.2.1 Quantitative Zero-Shot Baseline.

• 78.8% was GPT-5.2’s mean single-image accuracy, while GPT-5 and Gemini 3 each exceeded 60%.Source: §4.2.1 · p. 13
• Detection was inconsistent across models: Seed 1.6 had the highest standard deviation, 6.70, and some detectors struggled to exceed 50% chance accuracy.The authors conclude standalone zero-shot single-image classification is becoming unreliable as a forensic methodology.Source: §4.2.1 · pp. 12–13
• Flux.2 Pro was the most easily identifiable generator, with a cross-model mean detection rate of 80.6%; detection was difficult for six study detectors in a case with a 50.4% mean.The results suggest generator architecture influences how detectable an image is.Source: §4.2.1 · p. 13

4.2.2 Qualitative Reasoning.

• 93.8% was GPT-5.2’s score for correct qualitative justifications, which cited structural artefacts such as unusual geometry, unnatural texture, and blurred backgrounds.GPT-4o scored 58.3%, and the authors identify model version as a stronger factor in reasoning quality than generator ecosystem.Source: §4.2.2 · p. 14
• The manual audit recorded no hallucinations; even misclassifications were accompanied by structurally coherent reasoning about image inconsistencies.Two of 50 events were excluded after Gemini 3’s safety block on an authentic image.Source: §4.2.2 · p. 14, Table 3 · p. 14
• Four of six detectors repeatedly classified a GPT-Image-1.5 image as real, while Gemini 3 and GPT-5.2 identified hair blurring and implausible background geometry.Source: §4.2.2 · p. 14, Figure 5 · p. 14

4.2.3 Hypothesis Verdicts: H1 and H4.

• Same-ecosystem detection did not show a consistent advantage: GPT-5.2 detected GPT-Image-1 images at 84.3%, while it reached 87.9% on Flux.2 Pro images.Gemini 2.5 detected Gemini-generated images at 79.9%, but neither same-ecosystem result established a consistent advantage over external generators.Source: §4.2.3 · p. 15

Appendix A.2.1 for full test output).

• H1 was partially supported and H4 partially rejected: same-ecosystem advantages were modest, while no hallucinations were recorded and errors were clean misclassifications.The manual justification audit was conducted by one researcher, creating a confirmation-bias risk.Source: Appendix A.2.1 · p. 15

4.3 Experiment 2 Results: Pairwise Classification and Contextual Advantage

• GPT-5.2 achieved 82.5% mean pairwise accuracy and a 97.9% peak against Flux.2 Pro, the study’s highest single detection rate.All GPT models gained accuracy under pairwise conditions; GPT-5 gained 4.9 percentage points and GPT-4o gained 3.0 points over their single-image baselines.Source: Table 4 · p. 15, §4.3 · p. 15

4.3.1 Unexpected Pairwise Behaviour in Gemini Models.

• Under pairwise presentation, Gemini 2.5 accuracy fell by 17.1 percentage points, from 62.6% to 45.5%, below the 50% random-chance threshold; Gemini 3 declined by 5.0 points to 64.5%.GPT-family gains of +3.0 to +4.9pp were not statistically significant; pairwise presentation significantly declined Gemini performance, so H2 was rejected for the Gemini family.Source: §4.3.1 · p. 16, §4.3.3 · p. 16

4.3.2 Generator Difficulty Under Pairwise Conditions.

• Gemini 3 images were hardest to detect at a 43.6% cross-model mean, while Gemini 2.5 images were easiest at 85.7%, ahead of Flux.2 Pro at 83.6%.Source: §4.3.2 · p. 16

4.4.2 Stability and Baseline (C1).

• C1 found no significant difference between baseline groups (p = 0.4354, r = 0.078), supporting attribution of later differences to watermark manipulation rather than image variability.Source: §4.4.2 · p. 17

4.4.3 Visible Watermark Effect (C2).

• Removing Gemini’s visible sparkle logo reduced detection from 100% to 90% (p = 0.0075, r = 0.378), indicating that the logo improves detection consistency.Source: §4.4.3 · p. 18

4.4.4 SynthID as a Secondary Signal (C4).

• With the visible logo present, SynthID still added a statistically significant secondary signal (p = 0.0455, r = 0.283), though both datasets had 100% binary detection.Source: §4.4.4 · p. 18

4.4.6 Pure SynthID Impact (C5).

• Gemini detected its own images at 100% with all watermarks removed, while SynthID-only images reached 90% detection (p = 0.0020, r = 0.437).The watermark-free result supports detection through structural analysis; the SynthID-only comparison identifies the invisible signal as a significant forensic cue.Source: §4.4.6 · p. 18, §4.4.5 · p. 18
• The apparent contradiction may reflect masking: neutralizing SynthID likely amplified native structural artefacts, while embedding may have diminished high-frequency details.The text contrasts the reconstruction used to neutralize SynthID in D5 (Gemini-Full-Ablation) with SynthID embedding in D3 (Gemini-Web-Cropped).Source: §4.4.6 · p. 18
• C5 suggests SynthID is both a forensic signal and a subtle inhibitor of the structural artefacts that support successful detection.The embedding process may have diminished high-frequency details used for structural identification.Source: §4.4.6 · p. 18

4.4.7 Hypothesis Verdict: H3.

• Visible watermarks boosted detection (C2: p=0.0075, r=0.378), and SynthID provided an independent forensic signal (C5: p=0.0020, r=0.437).These results led the authors to partially reject H3, which predicted that any watermark would increase detection accuracy.Source: §4.4.7 · pp. 18–19
• 100% detection remained after all watermarks were removed, showing that native artefact analysis can detect AI-generated images without watermark evidence.The authors conclude that watermarks are supplementary, not essential, to MLLM-based forensic detection.Source: §4.4.7 · pp. 18–19
• The authors advise designing future invisible watermarks with care because they may suppress structural signals used by MLLM forensic tools.Source: §4.4.7 · p. 18

5.1 Revisiting the Research Question

• GPT-5.2 detected external Flux.2 Pro images at 87.9%, versus 72.1% for its own GPT-Image-1.5, supporting image quality as a stronger factor than ecosystem alignment.The discussion says same-ecosystem detection showed no consistent advantage.Source: §5.1 · p. 19

5.2 Key Findings

• Gemini 2.5’s accuracy fell by 17.1% and below 50% when a real photo accompanied the generated image, suggesting comparison noise can hinder some architectures.The proposed explanation is that a second image can confuse rather than assist the model.Source: §5.2 · p. 19
• 100% detection remained after visible and invisible watermarks were stripped from Gemini images, which the authors interpret as recognition of Gemini’s generative artefacts.The authors describe this as a ceiling effect.Source: §5.2 · p. 19
• 90% detection with SynthID present and the visible logo removed (D3) was below the fully ablated control (D5) at 100%.The authors report a statistical association between SynthID and reduced accuracy compared with images carrying no watermark (p=0.0020, r=0.437).Source: §5.2 · p. 19
• The manual audit found no hallucinated justifications among the six models; incorrect classifications were grounded in identifiable structural artefacts.GPT-5.2 scored 93.8% in the qualitative audit, while GPT-4o scored 58.3%.Source: §5.2 · pp. 19–20

5.3 Relation to Existing Literature

• The authors report that Gemini 3 images increased detection difficulty, while visible watermarks served as shortcuts and GPT-5.2 showed structural reasoning.The discussion says its results do not align with hallucination concerns in prior literature, while noting possible explanations including the portrait style or model-generation improvements.Source: §5.3 · p. 20

5.4 Limitations of the Study

• The study uses high-quality studio-style portraits, and it remains unclear whether GPT-5.2’s performance would hold under real forensic conditions with degraded imagery.The 400-image dataset is small relative to MLLM training corpora, and possible FFHQ training exposure could affect control-image accuracy.Source: §5.4 · p. 20

5.5 Future Work

• Future work proposes testing lower-quality imagery, comparing MLLMs with human forensic experts, and using multiple auditors for justification reviews.Further studies could also control pairwise comparison noise and expand watermark ablations to examine forensic watermark design.Source: §5.5 · p. 21

6 Conclusion

• Structural reasoning is presented as more reliable for forensic detection than pixel-level or metadata-based approaches.Source: §6 · p. 21
• The Gemini Paradox and inconsistent accuracy across model generations indicate that these tools are not yet robust, despite meaningful improvement in physical-inconsistency detection with model generation.Source: §6 · p. 21
• MLLMs should not yet replace human forensic experts, though they provide auditable natural-language justifications for each classification decision.Source: §6 · p. 21
• The masking phenomenon suggests invisible watermarking should be assessed for effects on the structural signals many MLLM detectors depend upon, not durability alone.Source: §6 · p. 21

A.1 System Prompt

• The prompt asks MLLMs to act as digital-forensics analysts and classify a single image as a real photograph or AI-generated fake.Source: §A.1 · p. 23
• Figure 7 locates the prompt within the detection Python script.Source: §A.1 · p. 23, Figure 7 · p. 23

A.2 Statistical Results (Experiments 1 and 2)

• The appendix section covers baseline performance metrics for Experiments 1 and 2.Source: §A.2 · p. 23
• A Mann–Whitney U test was used to compare same-ecosystem detector–generator pairs (N=10) with cross-ecosystem pairs.Source: §A.2.1 · p. 23
• Table 6 reports Mann–Whitney U test results for ecosystem alignment (H1).Source: Table 6 · p. 24

• The analysis compares each detector’s pairwise (TAFC) accuracy with its corresponding single-image baseline.Source: §A.2.2 · p. 24
• Table 7 presents Wilcoxon signed-rank test results for contextual advantage (H2).Source: Table 7 · p. 24
• The Mann–Whitney U results are described as showing no statistically significant advantage.Source: §A.2.3 · p. 24

A.2.3 Description of Statistical Outcomes.

• Same-ecosystem pairing showed no statistically significant advantage over cross-ecosystem pairing (p = 0.3835; r = 0.1875).Ecosystem membership was not a reliable predictor of baseline accuracy.Source: §A.2.3 · p. 24
• Descriptive gains across the GPT family were not statistically significant; GPT-5.2’s reported gain was +3.69% (p = 0.4062; r = 0.3929).Source: §A.2.3 · p. 24
• Gemini 2.5’s accuracy fell 17.09%, a significant decline attributed to comparison noise from the reference baseline.Reported statistics: W = 0.0, p = 0.0156, r = −1.0000.Source: §A.2.3 · p. 24

A.3 Ablation Study Statistical Results

• Figure 8 presents the full non-parametric Wilcoxon and Mann-Whitney U statistical results for the Gemini ablation study.Source: Figure 8 · p. 25