🔥 News

• [2025-09-19]🎉🎉🎉 AIGIBench is accepted by NeurIPS 2025 Datasets and Benchmarks.

This repository is the official repository of the AIGIBench.

This repository contains the AIGIBench dataset and the evaluated methods.

AIGIBench dataset contains two types of training and 25 test subsets. This dataset has the following advantages:

• Comprehensive generate types: including GAN-based Noise-to-Image Generation, Diffusion for Text-to-Image Generation, GANs for Deepfake, Diffusion for Personalized Generation, and Open-source Platforms.
• State-of-the-art Generators: MidjourneyV6, Stable Diffusion 3, Imagen, DALLE3, InstantID, FaceSwap, StyleGAN-XL and so on.
• Completely unknown generation method: Crawl pictures from communities and social media to build datasets CommunityAI & SocialRF, making detection more challenging.

If this project helps you, please fork, watch, and give a star to this repository.

📚Dataset

The training set and testing set used in the paper can be downloaded on Huggingface/Baidu Netdisk.

Each folder contains compressed files. After unzip the file, files under the data root directory can be organized as follows.

Train

AIGIBench introduces two training dataset settings: (i) Setting-I: Training on 144K images generated by ProGAN across four object categories—car, cat, chair, and horse. (ii) Setting-II: Training on 144K images generated by both SD-v1.4 and ProGAN, covering the same four object categories. The data of ProGAN comes from ForenSynths, and the data of sdv1.4 comes from GenImage. In order to maintain the fairness of the training data, we randomly select the sdv1.4 training images of GenImage to keep the same number as ProGAN, and then merge the data. The file directory is as follows:

├── train
│   ├── car
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── cat
│   │   ├── ...
│   ├── chair
│   │   ├── ...
│   ├── horse
│   │   ├── ...
│   ├── sdv1.4
│   │   ├── 0_real
│   │   ├── 1_fake
├── val
│   ├── ...
│   │   ├── 0_real
│   │   ├── 1_fake
│   │   ...

Test

AIGIBench comprehensively tests the performance of the detector and builds a test dataset from five perspectives: GAN-based Noise-to-Image Generation, Diffusion for Text-to-Image Generation, GANs for Deepfake, Diffusion for Personalized Generation, and Open-source Platforms. The file directory is as follows:

├── test
│   ├── ProGAN
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── R3GAN
│   │   ├── ...
│   │   ...
│   ├── BlendFace
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── InSwap
│   │   ├── ...
│   │   ...
│   ├── FLUX1-dev
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── Midjourney-V6
│   │   ├── ...
│   │   ...
│   ├── BLIP
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── Infinite-ID
│   │   ├── ...
│   │   ...
│   ├── CommunityAI
│   │   ├── 0_real
│   │   ├── 1_fake
│   ├── SocialRF
│   │   ├── ...

Note: The test set count in the paper contained some errors, which we are correcting here. Please note that the number of real images and generated images are consistent; only the number of generated images is listed below.

🔍Detection Methods

We use the official code for all detection codes and make unified modifications to the input and output. The code we use for training in Setting-II is publicly available above, the corresponding pre-trained checkpoints are publicly available on Huggingface. Of course, if you need the code from the original paper, the following is the corresponding detection code in the paper：

• ResNet-50: Deep Residual Learning for Image Recognition
• CNNDetection: CNN-generated images are surprisingly easy to spot...for now
• GramNet: Global Texture Enhancement for Fake Face Detection in the Wild
• LGrad: Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection
• CLIPDetection: Towards Universal Fake Image Detectors that Generalize Across Generative Models
• FreqNet: FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform
• NPR: Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection
• DFFreq: Dual Frequency Branch Framework with Reconstructed Sliding Windows Attention for AI-Generated Image Detection
• LaDeDa: Real-Time Deepfake Detection in the Real-World
• AIDE: A Sanity Check for AI-generated Image Detection
• SAFE: Improving Synthetic Image Detection Towards Generalization: An Image Transformation Perspectives
• Effort: Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection

⏳Detection Results (Continuously updating)

To ensure a fair comparison, we retrain all baseline methods on the Setting-II of AIGIBench.

If your retrained results differ significantly from those shown, please contact us.

For specific reasons, in the following method, we directly utilize the official pre-trained weights for inference.

Citation

@inproceedings{li2025artificial,
  title={Is Artificial Intelligence Generated Image Detection a Solved Problem?},
  author={Li, Ziqiang and Yan, Jiazhen and He, Ziwen and Zeng, Kai and Jiang, Weiwei and Xiong, Lizhi and Fu, Zhangjie},
  booktitle={Advances in Neural Information Processing Systems},
  year={2025}
}

Contact

If you have any question about this project, please feel free to contact 247918horizon@gmail.com