RAID is the largest & most comprehensive dataset for evaluating AI-generated text detectors. It contains over 10 million documents spanning 11 LLMs, 11 genres, 4 decoding strategies, and 12 adversarial attacks. It is designed to be the go-to location for trustworthy third-party evaluation of both open-source and closed-source generated text detectors.

Load the dataset

To load the dataset, install the library datasets with pip install datasets. Then,

from datasets import load_dataset
raid = load_dataset("liamdugan/raid")

Data fields

The RAID dataset has the following fields

1. `id`: A uuid4 that uniquely identifies the original content of the generation
2. `adv_source_id`: uuid4 of the source of the generation if adversarial
3. `source_id`: uuid4 of the human-written source text
4. `model`: The model that generated the text
   - Choices: `['chatgpt', 'gpt4', 'gpt3', 'gpt2', 'llama-chat', 'mistral', 'mistral-chat', 'mpt', 'mpt-chat', 'cohere', 'cohere-chat']`
5. `decoding`: The decoding strategy used 
    - Choices: `['greedy', 'sampling']`
6. `repetition_penalty`: Whether or not we use a repetition penalty of 1.2 when generating
    - Choices: `['yes', 'no']`
7. `attack`: The adversarial attack used
    - Choices: `['homoglyph', 'number', 'article_deletion', 'insert_paragraphs', 'perplexity_misspelling', 'upper_lower', 'whitespace', 'zero_width_space', 'synonym', 'paraphrase', 'alternative_spelling']`
8. `domain`: The genre from where the prompt/text was taken
    - Choices: `['abstracts', 'books', 'code', 'czech', 'german', 'news', 'poetry', 'recipes', 'reddit', 'reviews', 'wiki']`
9. `title`: The title of the article used in the prompt
10. `prompt`: The prompt used to generate the text
11. `generation`: The text of the generation

Data splits

Citation

BibTeX:

@inproceedings{dugan-etal-2024-raid,
    title = "{RAID}: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors",
    author = "Dugan, Liam  and
      Hwang, Alyssa  and
      Trhl{\'\i}k, Filip  and
      Zhu, Andrew  and
      Ludan, Josh Magnus  and
      Xu, Hainiu  and
      Ippolito, Daphne  and
      Callison-Burch, Chris",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-long.674",
    pages = "12463--12492",
}