From GAN-Based Models to GPT-5: A Taxonomy-Guided Study of Human Performance in Detecting Deepfake Face Images

1. Introduction

• Photorealistic deepfake faces are a consequential focus, while automated detectors can be unavailable or unreliable under distribution shifts.Users may still need to judge authenticity themselves.Source: §1 · p. 1
• Prior human-focused studies rarely examined modern generators or reported detection performance separately by model.Existing evidence leaves unclear whether human detection varies across generative models.Source: §1 · p. 1
• The study evaluates five models and training effects in an online main experiment and a five-month follow-up.The samples included 524 main-study participants and 342 follow-up participants.Source: §1 · p. 2
• Human detection performance varies across generators, and newer, more advanced models are not necessarily harder for people to detect.The study establishes a cross-model baseline spanning five representative generators.Source: §1 · p. 2
• DFAT-guided training improves detection accuracy, with model-specific training outperforming general-purpose training.The interventions are designed to teach human-recognizable artifacts across deepfake models.Source: §1 · p. 2
• Five months after training, detection gains persisted without increased perceived stress.The follow-up included 342 participants.Source: §1 · p. 2

2.1. Deepfake Generation Models

• The study samples five models across GAN, diffusion, and MLLM families to operationalize the face-synthesis landscape.The set includes StyleGAN2, StyleGAN3, GRAM-HD, SD3.5, and GPT-5.Source: §2.1 · p. 2

2.2. Automated Deepfake Detection

• Automated detectors may lose performance under distribution shifts and may be inaccessible in everyday settings.These constraints motivate complementary human-centered support.Source: §2.2 · p. 3

2.3. Human Detection of Deepfakes

• Reported training and feedback gains vary across studies, with some interventions yielding negligible improvement.Prior work includes feedback-based and artifact-focused guidance, but effects depend on protocols and evaluation conditions.Source: §2.3 · p. 3
• Human deepfake-face research has concentrated on GANs, especially StyleGAN2, and has not systematically tested model-specific performance or training effects.The review identifies nine studies reporting quantitative human accuracy for photorealistic facial deepfakes.Source: §2.3 · p. 3
• DFAT organizes human-recognizable artifacts across model families to guide training and consistent cross-model analysis.Earlier taxonomy work covered broader AI-generated photographs or diffusion outputs rather than fine-grained, model-specific face artifacts.Source: §2.3 · pp. 3–4

3.1. Taxonomy Construction Methodology

• DFAT was built from manual inspection of 1,000 images: 500 real photographs and 500 generated faces from five models.Recurring observations were grouped into a three-level hierarchy of domains, categories, and visual manifestations.Source: §3.1 · p. 4

3.2. Taxonomy Overview

• The taxonomy comprises four artifact domains: face-related, style-related, background-related, and accessory-related.Five annotators labeled images, and one co-author reviewed and finalized the annotations.Source: §3.2 · p. 4

3.2.1. Face-related Artifacts.

• Face-related cues include malformed teeth, unnatural hair or skin, anomalous eyes, and ears blurred or fused with hair.StyleGAN3 skin may show oval- or diamond-shaped local minitextures that become recognizable under magnification.Source: §3.2.1 · pp. 4–5

3.2.2. Style-related artifacts.

• Style-related artifacts include an overly perfect, idealized aesthetic in SD3.5 and GPT-5 images that departs from natural photographic variation.The contrast is an overall visual signature rather than a localized glitch.Source: §3.2.2 · p. 5

3.2.3. Background-related Artifacts.

• Background artifacts include blurred face-background boundaries, unreal or simplified scenery, and inconsistent lighting.The account links these issues to generators prioritizing faces and difficulty learning semantically consistent backgrounds.Source: §3.2.3 · pp. 5–6

3.2.4. Accessory-related Artifacts.

• Accessory artifacts include asymmetric or implausible earrings and clothing, and eyeglasses with inconsistent geometry or frames merged into skin.These cues reflect difficulties maintaining semantic coherence in rendered accessories.Source: §3.2.4 · p. 6

3.3.1. Selecting Learnable Artifact Cues.

• Training cues were selected for frequency, visual salience, and human recognizability, including background consistency, style, facial texture, and accessory symmetry.Shared domains support general-purpose guidance, while model-specific manifestations motivate targeted materials.Source: §3.3 · p. 6, §3.3.1 · p. 6

3.3.2. Constructing the Training Materials.

• The general-purpose set uses a five-minute video and ten practice questions; each model-specific set uses a one- to three-minute video and five questions.Videos follow a global-to-local inspection routine, moving from image-level cues to localized face and accessory cues.Source: §3.3.2 · p. 6

4.1. Study Design

• The mixed design varied training type and generator between participants, and testing session and presentation format within participants.The five model conditions were StyleGAN2, StyleGAN3, GRAM-HD, SD3.5, and GPT-5.Source: §4.1 · pp. 6–7
• Single-image judgments tested authenticity without a reference, whereas side-by-side judgments asked participants to identify the real image in a pair.Participants completed both formats before and after training.Source: §4.1 · p. 7

4.5. Statistical Analyses

• Detection performance was measured as the proportion of correct judgments, while confidence and stress were self-reported on five-point scales.Non-parametric analyses were used because accuracy scores were non-normally distributed.Source: §4.1 · p. 7, §4.5 · p. 8

4.2. Participants

• The main study analyzed 524 participants, and 342 members of that sample completed the five-month follow-up.Participants were recruited online and assigned to a training group and model condition.Source: §4.2 · p. 7

4.3.1. Main Study.

• The main study used consent and demographics, a 20-question baseline, DFAT-guided video and practice, and a post-training test with newly sampled images.Practice questions included explanations of correct answers, and participants reported confidence and perceived stress.Source: §4.3.1 · p. 7

4.3.2. Follow-up Study.

• At follow-up, participants repeated the post-training test without reviewing materials and completed 20 additional single-image trials with new images.The additional trials assessed transfer to unseen images.Source: §4.3.2 · p. 7

4.4. Test Stimuli

• Stimuli comprised 30 deepfake images from the assigned model and 30 FFHQ photographs, with non-overlapping pre- and post-training test images.Images were 1024 × 1024, and participants within each model condition saw the same image set.Source: §4.4 · pp. 7–8

5.1.1. Baseline Detectability Is Model-Dependent.

• Baseline accuracy differed by model: SD3.5 and GPT-5 images were easier to detect than StyleGAN2 and StyleGAN3 images, with model ordering broadly consistent across formats.Side-by-side references improved baseline accuracy, but did not remove model-dependent differences.Source: §5.1.1 · pp. 8–9, Figure 2 · p. 9

5.2.1. Training Gains Are Uneven Across Models.

• Significant accuracy gains occurred in 13 of 20 conditions; GAN-based models showed more consistent gains, while SD3.5 and GPT-5 gains were more conditional.StyleGAN3 improved by 22.8% with model-specific training in the side-by-side format; SD3.5 gains appeared only with model-specific training in the single-image format (+8.0%).Source: §5.2.1 · p. 9, Figure 3 · p. 9

5.2.2. Model-Specific Training Helped Most in Single-Image Subtasks.

• In single-image tasks, model-specific training outperformed general-purpose training for SD3.5 by 14.8% and for GPT-5 by 11.3%.GAN-model training-material gain differences were not statistically significant after correction, despite descriptive numerical advantages for model-specific training.Source: §5.2.2 · pp. 9–10

5.2.3. Format and Demographic Effects Are Limited.

• Training gains were comparable across presentation formats for all models except SD3.5, and demographic variables showed no consistent relationship with gains.Only six of 60 demographic interaction tests reached p < .05, without a consistent pattern.Source: §5.2.3 · p. 10

5.3.1. Training Shifted Sensitivity Toward Deepfake Im-

• Deepfake-image accuracy increased in nine of ten single-image conditions, while general-purpose training reduced real-image accuracy for SD3.5, StyleGAN3, and GPT-5.No Holm-corrected decrease in real-image accuracy occurred under model-specific training.Source: §5.3.1 · p. 10, Figure 4 · p. 11

5.3.2. Confidence and Stress Level Change After Train-

• Model-specific training increased confidence without a corrected stress increase, whereas general-purpose training significantly increased self-reported stress.Confidence rose from 3.41 to 3.78 after model-specific training; general-purpose training's confidence change was not significant.Source: §5.3.2 · p. 10

5.4.1. Detection Gains Showed Little Decay.

• None of 20 corrected paired comparisons showed a significant detection-accuracy decline at five months, and self-reported stress did not increase at follow-up.New-image trials showed a similar descriptive pattern, but were interpreted as transfer evidence rather than paired retention evidence.Source: §5.4.1 · p. 11, §5.4.2 · p. 11

5.5. Participant Feedback

• Most participants found the training useful; 89.9% said it improved their ability to identify deepfake faces, and unnatural skin remained a frequently reported cue five months later (78.4%).The most frequently reported cues immediately after training also included background problems and accessory rendering anomalies.Source: §5.5 · pp. 11–12

6.1. Key Findings

• Human detectability varies across generative models, and newer models are not necessarily harder to detect; local, subtle artifacts are especially difficult for untrained users to notice.The authors identify cross-model evaluation as important for studying human deepfake detection.Source: §6.1 · p. 12

6.2. Practical Implications

• Model-specific training produced better outcomes than general-purpose training, possibly because mixed artifact cues increase cognitive load and undermine training effectiveness.The authors suggest DFAT can guide concrete inspection cues and help organize platform prompts, educational materials, and moderation guidelines.Source: §6.1 · p. 12, §6.2 · p. 13

6.3. Extending DFAT

• DFAT is designed to be extensible: researchers can add artifact types and refine categories as generative models evolve.The authors also propose future interactive modules or assistive interfaces that guide image inspection.Source: §6.3 · p. 13

6.4. Limitations

• The study covered five models and fully generated photorealistic faces, and its controlled online experiment did not cover real-world contexts such as social feeds or time pressure.Participant dropout may introduce selection bias into the five-month retention findings.Source: §6.4 · p. 13

7. Conclusion

• Detection accuracy varied strongly by model, and some newer, more advanced models were easier for people to detect than older ones.The evaluation covered five representative models across three architectures.Source: §7 · p. 13
• DFAT-guided training significantly improved detection accuracy, with model-specific materials especially effective.The study compared general-purpose and model-specific interventions.Source: §7 · p. 13
• Training effects persisted after five months, with no significant decline in accuracy or increase in perceived stress reported.The conclusion describes the longer-term effects as sustainable.Source: §7 · pp. 13–14
• The authors advocate shifting public-awareness efforts toward concrete, model-specific, artifact-focused interventions rather than generic alerts.They also propose interface baselines for side-by-side comparison and tools that direct attention to harder-to-recognize anomalies.Source: §7 · p. 14

Appendix A. Detailed Review of Prior Related Studies

• Prior studies mainly examined older GAN-based models or did not report model-specific human performance.Table 4 compares nine closely related empirical studies.Source: Appendix A · p. 16
• Existing training interventions provide limited evidence about model-specific training and longer-term retention.Source: Appendix A · p. 16
• The reviewed studies were selected for including photorealistic face-image conditions, quantitative human detection accuracy, and controlled experimental settings.Source: Appendix A · p. 16

Appendix B. Data Quality and Incentives

• Table 4 is an overview of prior work categorized by training strategy, while Table 5 gives the main-study participants’ demographic breakdown.The supplied captions identify table scope but do not provide cell values.Source: Table 4 · p. 17, Table 5 · p. 17
• Participants were excluded if they failed at least two attention checks in either study.The main study had four checks across testing sessions; the follow-up had three.Source: Appendix B · p. 17
• Submissions completed in under six minutes were treated as invalid for insufficient effort.Source: Appendix B · p. 17
• Submissions with missing responses, inconsistent self-reported demographics, or an unfinished procedure were removed.Source: Appendix B · p. 17
• 524 participants remained for the main-study analysis and 342 for the five-month follow-up after quality-control screening.Source: Appendix B · p. 17
• Main-study participants received £5, and follow-up participants received £3, with additional performance-based bonuses in both studies.Bonuses depended on participants’ rankings within their training group and model condition.Source: Appendix B · p. 17

Appendix C. Platform and Questionnaires

• The appendix presents the platform and questionnaire items used in both the main and follow-up studies.Source: Appendix C · p. 17

C.1. Platform User Interface

• Figure 6 shows the single-image and side-by-side tests and practice-task feedback interface.After practice responses, participants saw the correct answer and an explanation.Source: §C.1 · p. 17, Figure 6 · p. 18

C.2. Confidence and Perceived-Stress Questions

• After each testing session, participants rated confidence and perceived stress on five-point Likert scales.Source: §C.2 · pp. 17–18

C.3. Main-Study Feedback Questionnaire

• The main-study feedback questionnaire asked whether deepfake technology is a significant societal problem and whether training helped identification.It also asked which training aspects were useful and what clues participants looked for.Source: §C.3 · p. 18
• Main-study feedback used five response options from strongly disagree to strongly agree for closed-ended questions.Source: §C.3 · p. 18

C.4. Follow-Up Feedback Questionnaire

• The follow-up questionnaire asked which clues participants used, whether they remembered the training, and what aspects they recalled.Responses included selectable clue categories and open-ended text.Source: §C.4 · p. 18

Appendix D. Examples of StyleGAN3 Texture Artifacts

• StyleGAN3 skin can show subtle oval- or diamond-like local patterns instead of natural pores, visible in magnified crops.Figure 7 presents two representative examples.Source: Appendix D · p. 18, Figure 7 · p. 18

Appendix F. Ethics Considerations

• The Prolific experiments received Cambridge ethics approval and informed consent; images were screened, and participant identifiers were removed from released data.Annotators voluntarily labeled images without compensation; that task collected no personal or sensitive data.Source: Appendix F · p. 19

Appendix G. Meta-Review

• The program committee prepared the meta-review as part of the 2027 IEEE Symposium on Security and Privacy review process.Source: Appendix G · p. 19

• The study systematically examined deepfake artifacts, created a taxonomy, and used it to guide training in a multi-month human-performance study.Source: §G.1 · p. 19
• The online study included 524 participants and examined generalized versus specialized training, pre- and post-training tasks, and follow-up.Source: §G.1 · p. 19
• The review reports that taxonomy-guided training was useful and its benefits lasted for months.Source: §G.1 · p. 19

G.2. Scientific Contributions

• The paper presents a taxonomy and publicly usable dataset as resources for future science.Source: §G.2 · p. 19, §G.3 · p. 19

G.3. Reasons for Acceptance

• Reviewers described the work as a valuable step toward understanding human performance in deepfake-image detection.Source: §G.2 · p. 19, §G.3 · p. 19
• The taxonomy enumerates artifacts and offers an approach for updating training materials, while the open-sourced artifact is documented for public use and future research.Source: §G.3 · p. 19