MGT-Clean Dataset

License: MIT

Overview

This repository provides a cleaned dataset, which is intended to be used for text classification, language modeling, and AI-generated content detection tasks. The dataset covers various fields such as STEM, Social Sciences, and Humanities, and contains datasets from different categories, each of which has been processed and cleaned for easy use. Move to our codebase fro more information (github)

Official Repo for On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing

Primary Sources

• Arxiv:
  • [Arxiv Data]
• Project Gutenberg:
  • Gutenberg Official Site
• Wiki:
  • [Wikipedia Data]

Dataset Classification

The dataset is organized into three major categories:

• STEM (Science, Technology, Engineering, Mathematics)

  • Physics (Arxiv available, Wiki)
  • Math (Arxiv available, Wiki)
  • Computer Science (Arxiv available, Wiki)
  • Biology (Arxiv available, Wiki)
  • Chemistry ( Wiki)
  • Electrical Engineering (Arxiv available, Wiki)
  • Medicine ( Wiki)
  • Statistics (Arxiv available, Wiki)

• Social Sciences

  • Education (Project Gutenberg, Wiki)
  • Management ( Wiki)
  • Economy and Finance (Arxiv available, Wiki)

• Humanities

  • Literature (Project Gutenberg, Wiki)
  • Law (Project Gutenberg, Wiki)
  • Art (Project Gutenberg, Wiki)
  • History (Project Gutenberg, Wiki)
  • Philosophy (Project Gutenberg, Wiki)

Data Cleaning Process

The datasets were cleaned to ensure high-quality human content by implementing the following detailed steps:

1. Removal of Short and Incomplete Texts:

   • For machine-generated texts, any text shorter than 50 words (split by spaces) was removed, as such texts often result from failed or incomplete API queries.

2. Truncation to Ensure Text Completeness:

   • Every text was truncated to a maximum of 2048 tokens. Starting from the last token, we back-traced until the first period was found, ensuring that the text ended at a complete sentence. This prevents truncated, incomplete texts from being included in the dataset, which could make detection tasks easier.

3. Keyword-Based Filtering:

   • Specific keywords and phrases that indicate machine-generated content (e.g., “The revised content is:”, “After editing”, “Retrieved from”, “ISBN”) were used to remove the machine-generated text with such identifiers. For generation identifiers, the closest colon was located, and all content before the colon was removed to ensure the removal of obvious machine-generation patterns.

4. Format-Based Filtering:

   • Texts containing certain formatting symbols commonly associated with machine-generated content (e.g., &, $, ====, **, ##) were entirely removed if they appeared more than 50 times or in specific machine-generated formats like markdown tags.

5. Consistency and Structure:

   • All datasets were restructured for consistency across domains (STEM, Social Sciences, and Humanities), ensuring that data was aligned according to its original source (e.g., Arxiv for STEM, Project Gutenberg for Humanities, etc.). This made it easier to categorize the data and apply different detection algorithms based on the domain-specific structures.

Reference

If you find this repo and dataset useful, please consider cite our work

@inproceedings{he2024mgtbench,
author = {He, Xinlei and Shen, Xinyue and Chen, Zeyuan and Backes, Michael and Zhang, Yang},
title = {{Mgtbench: Benchmarking machine-generated text detection}},
booktitle = {{ACM SIGSAC Conference on Computer and Communications Security (CCS)}},
pages = {},
publisher = {ACM},
year = {2024}
}
@misc{liu2025generalizationadaptationabilitymachinegenerated,
      title={On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing}, 
      author={Yule Liu and Zhiyuan Zhong and Yifan Liao and Zhen Sun and Jingyi Zheng and Jiaheng Wei and Qingyuan Gong and Fenghua Tong and Yang Chen and Yang Zhang and Xinlei He},
      year={2025},
      eprint={2412.17242},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2412.17242}, 
}

Contact

if you have any questions, please contact:

Xinlei He: xinleihe@hkust-gz.edu.cn