Dataset Card for OpenFake

Known issues

Prompt–image misalignment in the synthetic split (reported November 2025, fix pending)

For five of the eighty generators, the prompt field attached to synthetic images does not correspond to the prompt actually used to generate that image. Affected generators:

• flux-realism
• sd-3.5
• sdxl-realvis-v5
• sd-1.5-dreamshaper
• sd-1.5-epicdream

This affects approximately 19.77% of synthetic images. It was first reported in discussion #2.

Cause. Per-image prompts were added to the image split after generation, in response to user requests, by reconstructing the image-to-prompt association from generation records rather than from a mapping logged at generation time. For these five generators the reconstruction join was incorrect, so prompts were attached in the wrong order within the affected shards.

Scope. The defect is confined to the prompt column of the image split. It does not affect:

• real/fake labels
• the generator/model name field, which is derived from shard structure
• the standalone released prompt bank (a separate CSV), which is correct
• any detection benchmark result, which depends only on images, labels, and generator identity

Status. A corrected mapping is available at corrections/prompt_mapping_fix.parquet, keyed by image identifier, split, and generator name. Because some embedded images share a basename, the current misaligned prompt is also included in the join key to disambiguate those rows. The in-place prompt column will be updated in a versioned release; until then, join against the corrections file for the affected generators. We are not modifying existing data files while the associated paper is under review.

Loading the dataset and applying the corrected mapping

from pathlib import Path

import pandas as pd
from datasets import DatasetDict, Image, load_dataset
from huggingface_hub import hf_hub_download

core = load_dataset("ComplexDataLab/OpenFake", "core")  # train / validation / test
wild = load_dataset(
    "ComplexDataLab/OpenFake", "reddit", split="test"
)  # in-the-wild test dataset
prompts = load_dataset(
    "ComplexDataLab/OpenFake",
    data_files="prompt-image_bank.csv",
    split="train",
)  # extra prompt bank to generate new images

mapping_path = hf_hub_download(
    repo_id="ComplexDataLab/OpenFake",
    repo_type="dataset",
    filename="corrections/prompt_mapping_fix.parquet",
)
mapping = pd.read_parquet(mapping_path)
prompt_fixes = {
    (row.split, row.image_id, row.model, row.original_prompt): row.prompt
    for row in mapping.itertuples(index=False)
}


def apply_prompt_fixes(batch, split_name):
    batch["prompt"] = [
        prompt_fixes.get(
            (split_name, Path(image["path"]).name, model, original_prompt),
            original_prompt,
        )
        for image, model, original_prompt in zip(
            batch["image"], batch["model"], batch["prompt"]
        )
    ]
    return batch


fixed_splits = {}
for split_name, dataset in core.items():
    # decode=False exposes the published image filename used by the mapping.
    dataset = dataset.cast_column("image", Image(decode=False))
    dataset = dataset.map(
        apply_prompt_fixes,
        batched=True,
        fn_kwargs={"split_name": split_name},
        desc=f"Applying prompt fixes to {split_name}",
    )
    fixed_splits[split_name] = dataset.cast_column("image", Image())

core = DatasetDict(fixed_splits)

Removal requests

If you appear in an image or a prompt in this dataset, or believe an image/prompt should not be included, contact victor.livernoche@mail.mcgill.ca with the image/prompt identifier or a description. We will remove the record and issue a corrected release. Requests are handled regardless of whether the image remains available at its original upstream source.

Note that the real images in this dataset are redistributed as image files rather than as URLs, so removals upstream do not propagate automatically to this copy. This is why the channel above exists.

OpenFake is a dataset and benchmark for detecting AI-generated images, with a focus on politically and socially salient content where misinformation risk is highest. It pairs real photographs with synthetic counterparts produced by a wide range of frontier proprietary generators, open-source diffusion models, and community fine-tunes. A separate in-the-wild test set is sourced from Reddit to evaluate detector performance on naturally circulated synthetic media.

Versions

This is the v2 release. The original v1 release remains accessible at the v1.0 tag:

from datasets import load_dataset
ds = load_dataset("ComplexDataLab/OpenFake", revision="v1.0")

What changed in v2:

• New real-image sources (Pexels added to training; DOCCI and ImageNet used for OOD test reals)
• Many new generators including frontier proprietary models (nano-banana family, GPT Image 1.5 and 2.0) and recent open-source releases (Flux.2, Z-Image, HiDream variants, Chroma, etc.)
• Large addition of community fine-tunes and LoRAs sourced from Civitai, including video-generator outputs
• Restructured splits with a held-out OOD-model test set and a separate in-the-wild Reddit test config
• Standardized schema across all splits and configs

A detailed changelog is at the bottom of this card.

Configurations

Two configs are exposed:

• core (default): the main detection benchmark. Three splits: train, validation, test. The test split is held-out OOD models (no overlap with training generators) paired with real images from sources distinct from training reals. This is the cleanest setup for measuring generalization to unseen generators.
• reddit: in-the-wild test set only (test split). Synthetic images scraped from AI-generation subreddits, real images scraped from photography subreddits. Use this to evaluate how detectors trained on core transfer to naturally circulated content with platform compression and unknown provenance.

core = load_dataset("ComplexDataLab/OpenFake", "core")          # train / validation / test
wild = load_dataset("ComplexDataLab/OpenFake", "reddit", split="test")        # in the wild test dataset
prompts = load_dataset("ComplexDataLab/OpenFake", data_files="prompt-image_bank.csv", split="train")       # extra prompt bank to generate new images

You also have the extra prompt bank in

Dataset Details

Description

OpenFake covers high-resolution real and synthetic images centered on politically relevant media: faces of public figures, news-type events such as disasters and protests, and multimodal meme-style images with text overlays. Each image carries structured metadata with the prompt, source model (for synthetic), model type (base / fine-tune / LoRA / video), and release date.

• Authors: Victor Livernoche; Akshatha Arodi; Andrey Ambartsumov; Andreea Musulan; Zachary Yang; Adam Salvail; Gaétan Marceau Caron; Jean-François Godbout; Reihaneh Rabbany
• Curated by: Victor Livernoche; Andrey Ambartsumov; Akshatha Arodi; Jie Zang
• Funded by: CIFAR AI Chairs Program; Centre for the Study of Democratic Citizenship (CSDC); IVADO; Canada First Research Excellence Fund; Mila
• Language(s): English (prompts and captions)
• License: CC-BY-NC-4.0. Subsets produced with proprietary generators are released under non-commercial terms due to provider non-compete clauses; see the paper for the full breakdown.

Sources

• Repository: https://huggingface.co/datasets/ComplexDataLab/OpenFake
• Arena (crowdsourced adversarial platform): https://huggingface.co/spaces/CDL-AMLRT/OpenFakeArena
• Paper (v1): https://arxiv.org/abs/2509.09495

Schema

All splits in all configs share the same columns:

There are no image overlaps between splits.

Real-image sources

Multiple sources are used because the OOD design depends on real images coming from different distributions in train versus test:

• LAION (filtered). Politically salient or newsworthy images filtered from ReLAION-5B using Qwen2.5-VL. Carries authentic web compression artifacts. Used in core/train and core/validation.
• Pexels. High-quality stock photographs added in v2. Used in core/train and core/validation to expose detectors to clean professional photography.
• DOCCI. Densely captioned natural images. Used as a held-out real-image source for core/test so that OOD evaluation does not share real-image distribution with training.
• ImageNet. Used alongside DOCCI in core/test for diversity of natural-image content.
• Reddit photography subreddits. Real images for reddit/test. Carries platform-specific compression and naturalistic capture conditions.

Synthetic-image sources

Synthetic images come from a deliberately diverse generator set, grouped below by category. Specific model strings appear in the model column.

Frontier proprietary models

• OpenAI: gpt-image-1, gpt-image-1.5, gpt-image-2.0
• Google: nano-banana, nano-banana-pro, imagen-3, imagen-4
• Midjourney: midjourney-6, midjourney-7
• Ideogram: ideogram-2.0, ideogram-3.0
• Recraft: recraft-v2, recraft-v3
• Black Forest Labs: flux-1.1-pro
• xAI: grok-2-image-1212
• DALL·E 3

Open-source flagships

• Stable Diffusion family: sd-1.4, sd-1.5, sd-2.1, sdxl, sdxl-turbo, sd-3.5
• Flux family: flux.1-dev, flux.1-schnell, flux.2-dev, flux.2-klein-4b, flux.2-klein-9b
• Other: z-image-turbo, qwen-image, chroma, mystic, hidream-i1-full, playground-v2.5, kolors, lumina, aurora, seedream

Community fine-tunes and LoRAs (sampled from Civitai)

• SDXL fine-tunes: sdxl-epic-realism, sdxl-juggernaut, sdxl-realvis-v5, sdxl-touchofrealism
• SD 1.5 fine-tunes: sd-1.5-dreamshaper, sd-1.5-epicdream, realistic-vision-v5.1
• Illustrious / anime fine-tunes: many variants
• Specialized LoRAs (flux-amateursnapshotphotos, flux-realism, etc.)

Video-generation models (frames extracted from output)

• veo-3, wan-2.1, wan-2.2, wan-2.5, kling, hunyuan-video, ltxv-2.3, seedream-video, and others

For training distribution, see the per-model row counts at the statistics page (if published) or compute directly from the loaded dataset.

Splits and OOD design

The core/test set isolates two axes of OOD shift simultaneously: unseen generators and unseen real-image distribution. This produces a stricter generalization measure than holding out generators alone. Models held out for core/test include several frontier proprietary generators (gpt-image-1.5, gpt-image-2.0, nano-banana-pro), recent open-source releases (flux.2-klein-9b, z-image-turbo), and a set of dated-version generators (recraft-v2, recraft-v3, midjourney-7, ideogram-2.0, plus a small set of date-tagged Civitai entries).

Uses

Direct use

• Binary classification of real vs synthetic images, particularly on politically and socially salient content.
• Cross-generator generalization studies via the held-out OOD test split.
• Real-world transfer evaluation via the reddit config.
• Robustness studies (compression, resolution, content type) using metadata fields.
• Training adversarially robust detectors via community submissions through the OpenFake Arena.

Out-of-scope use

• Training generative models on this data without consent of the original real-image rights holders.
• Use of personal imagery in ways that violate platform rules or privacy laws.
• Identity-based facial recognition or biometric profiling on real subjects appearing in the data.

Dataset Creation

Curation rationale

OpenFake fills a gap in deepfake detection benchmarks by combining (1) high-quality politically salient content rather than ImageNet-style abstract imagery, (2) frontier proprietary generators that prior datasets lacked access to, (3) community fine-tunes representing the actual long tail of generators encountered in deployment, and (4) a separate in-the-wild test set that measures performance on naturally circulating content rather than only lab-generated samples.

Real-image curation

LAION images were filtered with Qwen2.5-VL to retain faces and politically salient or newsworthy events. Detailed captions were produced to drive both T2I generation and Arena prompts. Pexels, DOCCI, and ImageNet sources were sampled and integrated into specific splits as described above. Reddit real images were collected from photography-focused subreddits over a defined time window.

Synthetic-image generation

Synthetic images were generated from a shared prompt bank derived from the captioned real images. Open-source generations follow documented sampler and step settings. Proprietary generations use each provider's standard inference API at default or near-default settings, documented per-model in the paper.

A continual-extension pipeline tracks newly released open-source models on Hugging Face and runs them through the OpenFake prompt bank, keeping the dataset current with the generator landscape. The OpenFake Arena complements this by collecting human-adversarial submissions.

Personal and sensitive information

Source data was filtered to reduce personal or sensitive content. The dataset focuses on public figures and public events; private individuals appearing incidentally in real images are subject to the licensing terms of the original sources. See the paper's ethics section for the full discussion.

Bias, Risks, and Limitations

• Western political events are likely overrepresented because of source distribution.
• Synthetic images inherit the biases of the generators that produced them; this is a feature for evaluation purposes but a limitation for any other use.
• Generator labels are pipeline-assigned rather than exhaustively human-verified.
• Reddit-sourced labels rely on subreddit context (AI-generation subreddits assumed synthetic, photography subreddits assumed real); some label noise is expected at the percent level.
• Adversarial use as training data for evading detection is a risk; the license and the focus on detection are partial mitigations.

Recommendations

Use caution when interpreting political narratives in the imagery. The dataset is for detection research; do not use it for content generation, identity-linked facial analysis, or any purpose involving real subjects without additional review.

Citation

BibTeX:

@misc{livernoche2025openfakeopendatasetplatform,
      title={OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection},
      author={Victor Livernoche and Akshatha Arodi and Andreea Musulan and Zachary Yang and Adam Salvail and Gaétan Marceau Caron and Jean-François Godbout and Reihaneh Rabbany},
      year={2025},
      eprint={2509.09495},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.09495},
}

APA:

Livernoche, V., Arodi, A., Musulan, A., Yang, Z., Salvail, A., Marceau Caron, G., Godbout, J.-F., & Rabbany, R. (2025). OpenFake: An open dataset and platform toward real-world deepfake detection. arXiv. https://arxiv.org/abs/2509.09495

A v2 paper update is in preparation.

v1.0 (frozen at tag v1.0)

• Original release as described in the v2 paper

Contact

For questions, errors, or contributions: victor.livernoche@mail.mcgill.ca, or open a discussion on this dataset's HF page.