"A unified evaluation suite for membership inference attacks
and machine-generated text detection."

---

This is the official repository of our paper, "Machine Text Detectors are Membership Inference Attacks".

• We theoretically prove that MIA and machine text detection share the same optimal metric.
• We empirically demonstrate strong cross-task transferability (ρ ≈ 0.7) across diverse domains and generators.
• Notably, a machine text detector outperforms a state-of-the-art MIA on MIA benchmarks.

These results call for cross-task developments and fair evaluation across both tasks.

---

📢 Updates

• 🎉 May 2026: Accepted to ICML 2026 MemFM Workshop, Seoul 🇰🇷!
• 🗣️ Feb 2026: Talk invited at Google Research! [slide]

💨 Quick Start

Build environment (Python>=3.9):

$ git clone https://github.com/ryuryukke/mint.git
$ cd mint
$ python -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt

Set your Hugging Face cache directory:

$ export HF_HOME=/path/to/huggingface_cache

Run evaluation on all methods for MIA:

$ python run.py --task mia --domain arxiv --methods all --model_name pythia-160m

Run evaluation on all methods for detection:

$ python run.py --task detection --domain wiki --methods all --model_name llama-chat

Please see more details for options in scripts/*.sh.

🌿 MINT Supports

We currently cover 4 common baselines, 7 state-of-the-art MIAs, and 5 state-of-the-art machine text detectors. Please submit an issue for more method support.

📚 Datasets

We employ the MIMIR benchmark for MIAs and the RAID benchmark for detection.

🏃‍♂️ Running on a custom dataset

You can add a custom dataset by adding new if-else block to load_evaluation_data() in run.py.

🏃‍♂️ Running a custom attack or detector

You can add a custom attack or detector by creating a new directory under methods/ and registering it in src/method.py. Please follow the shared format defined in src/method.py.

✍️ Citation

If you find our code or ideas useful in your research, please cite our work:

@misc{koike2025machinetextdetectorsmembership,
      title={Machine Text Detectors are Membership Inference Attacks}, 
      author={Ryuto Koike and Liam Dugan and Masahiro Kaneko and Chris Callison-Burch and Naoaki Okazaki},
      year={2025},
      eprint={2510.19492},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.19492}, 
}

Acknowledgements

This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200005. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein. These research results were also obtained from the commissioned research (No.22501) by National Institute of Information and Communications Technology (NICT), Japan. In addition, this work was supported by JST SPRING, Japan Grant Number JPMJSP2106.