IdeaShift-X

IdeaShift-X tests idea-provenance detection outside English. It holds about 10,000 human-written web pages in 24 languages from FineWeb2. Each page has two AI versions in the same language, written by Gemini 3.7 Flash:

• Level 0: written from a brief naming only the document type, so the ideas are the AI's.
• Level 5: written from the page's full role-labelled outline with an instruction to add nothing, so the ideas are the page's and the prose is the AI's.

A detector should not flag the human pages. It should flag level 0 for AI ideas. An idea detector should not flag level 5; a prose detector should.

Construction

The dataset merges two builds.

• 9-language build: Arabic, Chinese, German, Indonesian, Japanese, Russian, Spanish, Tamil, Urdu. About 610-740 pages per language.
• 24-language build: adds 15 languages at about 250 pages each.

Pages were format-classified with WebOrganizer's annotation prompt on an LLM. Only pages in the eight long-form formats IdeaLens was trained on were kept, some of them force-fitted to the closest format. Outlines are always extracted in English, whatever the page's language.

Splits

Rows in different splits share seed_id; a few pages lack one of their AI versions. language selects a language, e.g. load_dataset("rishanthrajendhran/IdeaShift-X", split="level_5").filter(lambda r: r["language"] == "fra_Latn").

Fields

Caveats

• URLs are not included. The FineWeb2 id identifies each page in the source dataset.
• Outlines are the untreated outlines each document was scored on, not the paraphrased outlines used for training. The level-5 prompt is assembled from a template for each document and was not stored, so system_prompt and prompt are empty there.
• Scores are stored at full precision. Rounding them ties the very small P(AI) values and moves the AUC. pangram4_p_ai is empty for a few documents Pangram returned nothing for.
• Flag a document at the 1% global cut, as in the paper: IdeaLens when idealens_p_ai is above 0.86288, Pangram 4 when pangram4_p_ai is at least 0.5, ProseLens when proselens_p_ai is above 0.39461.
• Outlines and page text can contain web addresses or e-mail addresses that were on the original page.

Scoring

The idealens package (PyPI) runs IdeaLens, ProseLens and the other released detectors on these documents: pip install idealens, then idealens run on a JSONL file with a text field.

All IdeaLens models and datasets are in the IdeaLens collection.

License

The dataset is released under CC BY-NC-SA 4.0: free to share and adapt for non-commercial purposes, with attribution and under the same license. The human pages come from FineWeb2 (ODC-By 1.0) and remain subject to the terms of their original sources and of Common Crawl. The AI versions were produced with Google Gemini models.

Citation

@article{idealens2026,
  title         = {IdeaLens: Detecting AI Ideas in Long-form Writing},
  author        = {Rajendhran, Rishanth and Choi, Minjoon and Russell, Jenna and Namuduri, Ramya and B{\"o}l{\"o}ni-Turgut, Deniz and Karpinska, Marzena and Wieting, John and Iyyer, Mohit},
  journal       = {arXiv preprint arXiv:2610.06778},
  year          = {2026},
  eprint        = {2610.06778},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.06778}
}