PaperlayerView exact revision on arXiv

Source-linked AI summary

Is Artificial Intelligence Generated Image Detection a Solved Problem?

Ziqiang Li, Jiazhen Yan, Ziwen He, Kai Zeng, Weiwei Jiang, Lizhi Xiong, Zhangjie Fu

arXiv:2505.12335v2cs.CVcs.CR

TL;DR

AIGIBench asks whether AIGI detectors’ high controlled-setting accuracy transfers to realistic conditions. It benchmarks 11 detectors across four practical tasks and diverse generated and real-world data, finding substantial real-world performance degradation and limited or uneven benefits from augmentations and pre-processing. The benchmark exposes generalization and reliability gaps while providing a standardized evaluation framework, with current training settings limited to ProGAN and ProGAN plus SD-v1.4.

  • Problem

    Reported AIGI-detection accuracies often exceed 95%, but their effectiveness under realistic sources, degradations, and manipulation conditions remains questionable.

  • Method

    AIGIBench evaluates 11 detectors across multi-source generalization, degradation robustness, augmentation variation, and test-time pre-processing using diverse synthetic and real-world image subsets.

  • Results

    Detectors suffer significant performance degradation under real-world conditions, while augmentation benefits are limited and pre-processing effects are nuanced.

  • Takeaways & Limitations

    AIGIBench provides a unified realistic framework that reveals detector limitations and supports standardized comparisons toward more robust and generalizable detection methods.

  • Takeaways & Limitations

    AIGIBench currently considers only ProGAN and ProGAN plus SD-v1.4 as training settings, with broader datasets and detectors planned for future work.

Abstract

from arXiv · show

The rapid advancement of generative models, such as GANs and Diffusion models, has enabled the creation of highly realistic synthetic images, raising serious concerns about misinformation, deepfakes, and copyright infringement. Although numerous Artificial Intelligence Generated Image (AIGI) detectors have been proposed, often reporting high accuracy, their effectiveness in real-world scenarios remains questionable. To bridge this gap, we introduce AIGIBench, a comprehensive benchmark designed to rigorously evaluate the robustness and generalization capabilities of state-of-the-art AIGI detectors. AIGIBench simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing. It includes 23 diverse fake image subsets that span both advanced and widely adopted image generation techniques, along with real-world samples collected from social media and AI art platforms. Extensive experiments on 11 advanced detectors demonstrate that, despite their high reported accuracy in controlled settings, these detectors suffer significant performance drops on real-world data, limited benefits from common augmentations, and nuanced effects of pre-processing, highlighting the need for more robust detection strategies. By providing a unified and realistic evaluation framework, AIGIBench offers valuable insights to guide future research toward dependable and generalizable AIGI detection.Data and code are publicly available at: https://github.com/HorizonTEL/AIGIBench.

1 Introduction

AIGIBench questions whether high reported AIGI-detection accuracy transfers to realistic conditions. It introduces a benchmark and pipeline targeting unknown sources, degradations, and practical evaluation gaps.

  • 1 Introduction: AIGIBench evaluates AIGI detectors across multi-source generalization, image degradation, data augmentation, and test-time pre-processing.The pipeline augments real and synthetic training images, then crops or resizes test images before binary authenticity classification.
  • 1 Introduction: Despite reported accuracies exceeding 95%, existing detectors face significant challenges on real-world manipulations and in-the-wild content.The benchmark reports notable degradation on DeepFakes and in-the-wild data, with no method consistently outperforming others across all generative scenarios.
  • 1 Introduction: AIGIBench covers 23 subsets spanning advanced and widely adopted image-generation techniques, plus fake images collected from social media and AI-art platforms.Its scope includes GAN-based, diffusion-based, deepfake, personalized-generation, and open-platform sources.
  • 1 Introduction: Unlike earlier benchmarks, AIGIBench incorporates 11 recent detection techniques while comprehensively simulating state-of-the-art image-generation methods.The benchmark is positioned as a broader evaluation framework than prior benchmarks that mainly focused on methods developed before 2022.

2 AIGIBench

AIGIBench is constructed to evaluate AIGI detectors under diverse, realistic data and testing conditions. Its dataset and four tasks cover multiple generation sources, real images, degradations, augmentations, and inference pre-processing.

  • 2 AIGIBench: Two training settings use 72K ProGAN images or 144K images combining SD-v1.4 and ProGAN across four object categories.The benchmark also re-trains all evaluated methods under both settings using their original hyperparameters.
  • 2 AIGIBench: The benchmark spans 23 fake-image subsets covering GAN noise-to-image, diffusion text-to-image, GAN deepfake, and diffusion personalized-generation techniques.Examples include ProGAN and StyleGAN3, SD-XL and DALLE-3, FaceSwap and SimSwap, and InstantID and IP-Adapter.
  • 2 AIGIBench: Real images are balanced against fake images and sampled from FFHQ, CelebA-HQ, and Open Images V7 to combine facial and broader object diversity.Fake-image collection uses generator, web-API, and open-platform sources, with CLIP similarity filtering to remove near-duplicates.
  • 2.4 Task Definition: AIGIBench evaluates detectors through four tasks: multi-source generalization, multi-degradation robustness, augmentation variation, and test-data pre-processing.The tasks target unknown distributions, JPEG/noise/sampling degradations, overfitting-related augmentation choices, and compatibility between test images and trained detectors.

3 Experiments and Discussion

AIGIBench evaluates detector generalization and robustness across diverse testsets, degradations, augmentations, and preprocessing choices. Results show substantial variability, real-world failures, and trade-offs that limit dependable AIGI detection.

  • 3.1 Task 1: Generalization Assessment: SAFE achieves the highest overall accuracy at 79.2% in Setting-I and 79.9% in Setting-II, while AIDE reaches 82.7% mean A.P. in Setting-II and 69.0% mean F.Acc. in Setting-I.These comparisons are reported across 25 AIGIBench testsets.
  • 3.1 Task 1: Generalization Assessment: No detector consistently dominates across all scenarios, and performance degrades substantially on face-swap and in-the-wild manipulations despite stronger results on GAN and diffusion generation tasks.The findings motivate complementary strategies including frequency analysis, self-supervised learning, and large-scale pre-training.
  • 3.2 Task 2: Robustness Assessment: Under JPEG compression and Gaussian noise, fake-image accuracy often approaches 0% while real-image accuracy remains near 100%, indicating a strong bias toward predicting real.The perturbation settings use JPEG quality factor 50 and Gaussian noise with σ = 4; down-up sampling halves image size before restoring resolution.
  • 3.2 Task 2: Robustness Assessment: Across perturbations, most detectors retain R.Acc. ≥90% but often fall below 35% F.Acc., showing compromised reliability under image degradation.FreqNet is described as more robust because its frequency-domain features are less sensitive to spatial perturbations.
  • 3.3 Task 3: Data Augmentation Variation Assessment: Augmentation generally improves R.Acc. but can reduce F.Acc.; combining rotation, color-jitter, and masking provides no clear advantage and may impair consistency.The effects are model-dependent, especially for detectors sensitive to semantic or frequency cues.
  • 3.4 Task 4: Test Data Pre-processing Assessment: Crop preprocessing generally outperforms Resize by improving R.Acc., but F.Acc. remains unchanged or declines for some detectors.For SAFE, R.Acc. rises from 63.3% to 96.8%; for FreqNet, it rises from 65.9% to 84.6%.

4 Conclusions, Limitations, and Board Impacts

AIGIBench is a realistic benchmark for testing AIGI detectors across diverse generators, real-world samples, and four practical evaluation tasks. Its results expose substantial real-world performance degradation, while its current training settings and detector coverage remain limited.

  • 4 Conclusions, Limitations, and Board Impacts: AIGIBench combines 23 diverse synthetic-image subsets, real-world samples, and four tasks covering generalization, degradation robustness, augmentation sensitivity, and test-time preprocessing.The benchmark includes both cutting-edge and widely adopted generation techniques, with samples from social media and AI art platforms.
  • 4 Conclusions, Limitations, and Board Impacts: Existing detectors suffer significant performance degradation under real-world conditions, confirming that AIGI detection remains a formidable challenge.The benchmark is intended to support standardized evaluation and guide further detector development.
  • 4 Conclusions, Limitations, and Board Impacts: AIGIBench currently covers only two training settings: ProGAN alone and ProGAN combined with SD-v1.4.Future work plans larger, more diverse training datasets, broader detector and dataset coverage, and a public leaderboard.
  • 4 Conclusions, Limitations, and Board Impacts: The benchmark is positioned to enable standardized comparisons and support more robust and generalizable forensic methods addressing risks such as misinformation and digital fraud.These impacts are presented as intended benefits of the evaluation framework.

A Appendix

Appendix materials include generalization results for detectors trained on 144K images generated by SD-v1.4 and ProGAN. The appendix table reports F.Acc. and R.Acc. comparisons across detectors.

  • A Appendix: Table 7 reports detector generalization using Acc. and A.P. in Setting-II, trained on 144K images generated by SD-v1.4 and ProGAN.

A.1 More Results on Task 1: Generalization Assessment

Generalization remains difficult: detectors trained on limited generator distributions often overfit, while CLIP-, frequency-, and advanced methods improve cross-model performance but still struggle on deepfake data.

  • Generalization Assessment: Training-domain performance above 97% can fall below 55% accuracy and 60% average precision on unseen generators for classical CNN detectors.The reported examples include R3GAN, FaceSwap, and StyleGAN-XL.
  • Generalization Assessment: CLIPDetection reaches 91.2% average precision on R3GAN and 95.2% on StyleSwim, outperforming conventional CNN baselines on out-of-distribution models.
  • Generalization Assessment: SAFE achieves 93.9% accuracy and 98.2% average precision on R3GAN, with consistently high scores across GAN- and diffusion-based models.AIDE and LaDeDa also maintain robust and balanced cross-model performance.
  • Generalization Assessment: Generalization to deepfake models remains poor even for advanced detectors such as SAFE and AIDE, including on DALLE-3, BlendFace, and SocialRF.The results point to limitations in current methods and a need for greater training diversity and more robust feature representations.
  • Generalization Assessment: Training exclusively on ProGAN images produces significant overfitting and poor generalization across diverse generative models and datasets.The benchmark reports results under both F.Acc./R.Acc. and Acc./A.P. settings.

A.2 More Results on Task 2: Robustness Assessment

Detector robustness drops substantially under realistic image degradations, especially JPEG compression and Gaussian noise, although AIDE and DFFreq provide more stable fake-image detection.

  • Robustness Assessment: JPEG compression, Gaussian noise, and up-down sampling are used to simulate common real-world image degradations.The evaluation applies JPEG quality factor 50, Gaussian noise with σ = 4, and half-resolution downsampling followed by upsampling.
  • Robustness Assessment: Mean fake-image accuracy reaches 51.5% for both DFFreq and AIDE across perturbations, while JPEG compression causes near-zero fake accuracy for nearly all detectors.JPEG compression is identified as the most challenging perturbation.
  • Robustness Assessment: CNNDetection falls from 99.0% real-image accuracy on clean images to 0.8% fake-image accuracy under JPEG compression.
  • Robustness Assessment: AIDE, DFFreq, and SAFE show improved robustness and are better suited to diverse, degraded inputs than traditional CNN-based detectors.

A.3 More Results on Task 3: Data Augmentation Variation Assessment

Standard data augmentations provide limited gains for AIGI detection and can create performance trade-offs across detector architectures and augmentation combinations.

  • Data Augmentation Variation Assessment: Standard augmentation techniques generally provide limited improvement and can sometimes reduce AIGI detection performance.The evaluated strategies include rotation, color-jitter, masking, and their combinations under Setting-II.

A.4 More Results on Task 4: Test Data Pre-processing Assessment

Test-time preprocessing has an asymmetric effect: Crop generally improves real-image accuracy, while fake-image accuracy changes little or can decline.

  • Test Data Pre-processing Assessment: Crop preprocessing substantially increases real-image accuracy for LaDeDa and SAFE, from 72.5% to 98.9% and from 58.4% to 91.1%, respectively.Their fake-image accuracy increases only modestly or slightly decreases.
  • Test Data Pre-processing Assessment: Crop generally outperforms Resize because its gains primarily come from improved real-image accuracy, while fake-image accuracy remains unchanged or declines.The reported asymmetry is evaluated under Training Setting-I, where detectors train solely on ProGAN images.
  • Test Data Pre-processing Assessment: Preserving local structures and high-frequency textures may help authentic-content detection, whereas heterogeneous synthetic artifacts produce mixed fake-image outcomes.

A.5 More Confirmatory Analyses

These analyses examine how dataset curation, augmentation, and test-time preprocessing affect AIGI detector evaluation. They show that curated data increases difficulty, augmentations offer mixed benefits, and preprocessing effects vary by metric and detector.

  • A.5 More Confirmatory Analyses: CLIP-based near-duplicate removal and aesthetic filtering retain high-resolution, visually diverse images to increase benchmark difficulty and discriminative power.Images are filtered using a cosine similarity threshold of 0.98, and only high-resolution 4K images with high aesthetic scores are retained.
  • A.5 More Confirmatory Analyses: Curated datasets are more challenging for detectors than original unfiltered datasets, as indicated by positive F.Acc subscripts in Table 17.The subscript records the performance difference between unfiltered and curated data; positive values mean worse performance on the curated dataset.
  • A.5 More Confirmatory Analyses: Under Training Setting-I, crop preprocessing primarily improves R.Acc. but has limited or negative effects on F.Acc. across several detectors.The comparison uses a training set of 72k ProGAN-generated images.

A.6 Details of Generation Methods Used in Our Evaluation Datasets

AIGIBench constructs balanced real–fake datasets from multiple image sources and forgery paradigms, including samples collected from real-world online scenarios. Its curated evaluation data is explicitly designed to challenge detectors under realistic conditions.

  • A.6 Details of Generation Methods Used in Our Evaluation Datasets: Each sub-dataset contains equal numbers of real and fake images, with real images drawn from FFHQ, CelebA-HQ, and Open Images V7.The sources combine high-resolution facial images with diverse object and scene categories.
  • A.6 Details of Generation Methods Used in Our Evaluation Datasets: Table 17 compares detector F.Acc. on original unfiltered and curated datasets, using subscripts to indicate the curated-data performance difference.Positive subscripts mean detectors perform worse on the curated dataset, which is therefore more challenging.
  • A.6 Details of Generation Methods Used in Our Evaluation Datasets: Fake images span 25 forgery methods across GANs, diffusion models, deepfakes, and customized generation techniques.The benchmark also includes real and manipulated images collected from social media platforms and AI communities.

A.6.1 GANs for Noise-to-image Generation

The GAN subset covers six state-of-the-art noise-to-image generators and applies refinement procedures to produce diverse, high-quality evaluation images. The included methods span progressive-growing, style-based, transformer-based, and regularized GAN designs.

  • A.6.1 GANs for Noise-to-image Generation: The GAN evaluation subset includes ProGAN, StyleGAN3, StyleGAN-XL, StyleSwim, R3GAN, and WFIR.These six methods are selected as advanced GAN-based noise-to-image generators.
  • A.6.1 GANs for Noise-to-image Generation: ProGAN progressively grows generator and discriminator layers from low to high resolution, improving training stability and image quality.It is included partly because it is used to train certain detection models, supporting domain consistency in evaluation.
  • A.6.1 GANs for Noise-to-image Generation: StyleGAN3 uses Fourier feature mapping and rotation-invariant design principles to improve high-frequency rendering and geometric consistency.The supplied passage characterizes these choices as supporting highly realistic and coherent synthesis.
  • A.6.1 GANs for Noise-to-image Generation: StyleGAN-XL controls image features across scales with AdaIN, progressive growing, and stochastic noise injection, enabling images up to 1024×1024.Its style-based synthesis network maps latent codes through a multilayer perceptron.
  • A.6.1 GANs for Noise-to-image Generation: StyleSwin combines local and shifted-window attention with a wavelet discriminator to improve coherence and reduce blocking artifacts.The architecture targets efficient high-resolution synthesis with details up to 1024×1024.
  • A.6.1 GANs for Noise-to-image Generation: R3GAN uses a lightweight generator and regularized relativistic loss to stabilize training, mitigate mode collapse, and produce fine-grained images.The dataset uses the official pre-trained model for generation.
  • A.6.1 GANs for Noise-to-image Generation: The post-processed R3GAN subset contains 4,500 generated images and 4,500 real images.
  • A.6.1 GANs for Noise-to-image Generation: WFIR evaluates real-versus-AI-generated face discrimination using StyleGAN synthesis with progressive growing and AdaIN-based layers.The method generates realistic faces with varied attributes such as expressions and lighting.

A.6.2 Diffusion for Text-to-image Generation

The diffusion subset generates text-conditioned images with seven representative models using diverse prompts and post-generation quality filtering. It covers models ranging from GLIDE and Stable Diffusion variants to commercial systems such as DALL·E 3 and Midjourney V6.

  • A.6.2 Diffusion for Text-to-image Generation: Diffusion models generate images by iteratively denoising random noise under textual guidance, making prompt quality and diversity important for dataset construction.The benchmark uses Gemini to produce diverse descriptions through carefully designed instructions.
  • A.6.2 Diffusion for Text-to-image Generation: The benchmark uses GLIDE, DALLE-3, Imagen3, FLUX1-dev, Midjourney V6, SD3, and SDXL for text-to-image generation.Prompts cover people, animals, objects, and landscapes, with 1,500 unique sentences per category.
  • A.6.2 Diffusion for Text-to-image Generation: CLIP-based redundancy detection and aesthetic evaluation remove visually similar and low-quality outputs after generation.The pipeline ranks image quality quantitatively and excludes samples with subpar visual attributes.
  • A.6.2 Diffusion for Text-to-image Generation: GLIDE uses classifier-free guidance and supports text-driven inpainting within a diffusion-based generation framework.
  • A.6.2 Diffusion for Text-to-image Generation: DALLE-3 combines text-conditioned diffusion, improved caption quality, and classifier-free guidance to enhance prompt alignment and output diversity.
  • A.6.2 Diffusion for Text-to-image Generation: Imagen 3 progressively denoises latent representations and uses multi-stage upsampling to produce photorealistic images up to 1024×1024.
  • A.6.2 Diffusion for Text-to-image Generation: FLUX.1-dev combines diffusion with transformer layers and flow matching to improve training stability and fine-grained detail rendering.Rotary positional embeddings and parallel attention support complex prompts and outputs up to 1024×1024.
  • A.6.2 Diffusion for Text-to-image Generation: Midjourney V6 uses diffusion, enhanced language encoding, upsampling, and attention mechanisms to generate detailed, semantically aligned images up to 2048×2048.

A.6.3 GANs for Deepfake

AIGIBench includes diverse deepfake data sources and face-swapping methods to evaluate detection under realistic conditions. Its construction combines GAN-based and conventional manipulation pipelines with balanced real and generated images, while the benchmark remains limited to two training settings.

  • Dataset sources and methods: AIGIBench includes DFDC and five face-swapping methods spanning GAN-based, deep-learning, and traditional computer-vision pipelines.The methods include BlendFace, E4S, FaceSwap, InSwap, and SimSwap.
  • GAN-based face swapping: BlendFace produces realistic swaps by disentangling source identity from target attributes and enforcing identity consistency during generation.Its identity encoder reduces attribute bias, while a discriminator evaluates visual fidelity and identity preservation.
  • GAN-based face swapping: E4S performs fine-grained swaps in StyleGAN's latent space by separately representing facial shape and regional texture.A mask-guided encoder and injection module project facial-region textures into regional style codes.
  • Traditional face swapping: FaceSwap uses 68 facial landmarks, a fitted Candide 3D face model, and image blending rather than deep learning.The pipeline estimates pose, facial shape, and expression before projecting and blending source-face texture.
  • Real-world data collection: SocialRF and CommunityAI each contain 4,500 generated and 4,500 real images collected from targeted social-media searches and AI-art platforms.Authentic images use tags such as #nature and #realphoto, while synthetic images come from AI-related hashtags and platforms including ArtStation, Civitai, and Liblib.
Loading 2505.12335v2…