rishanthrajendhran/TwiceTold

Dataset

TwiceTold

0

10 commits

1 linked in READMEs

updated Oct 6, 2026

See the code

README

TwiceTold

TwiceTold holds 50 short stories that people wrote from scratch, each working only from the outline of an AI-generated story, together with the 50 AI stories those outlines came from. The ideas in every story are the AI's. The prose of the 50 rewrites is human. A detector of who wrote the prose should call the rewrites human. A detector of whose ideas they are should call them AI. The name comes from Nathaniel Hawthorne's Twice-Told Tales.

Construction

  • AI stories: GPT-5.6 Sol (high reasoning effort) wrote a 500-word story for each of 50 genres, from the prompt "Write me a 500 word story in {genre}".
  • Outlines: an outline of each AI story was extracted: an ordered list of [Role] content items.
  • Rewrites: 17 computer science graduate students and faculty each received one or more outlines, and wrote the story in person rather than through crowdsourcing, to reduce the chance of AI use during writing.

Writers are identified only by pseudonymous ids (W01-W17). No other information about them is included.

Splits

SplitRowsWhat it holds
human_rewrite50the stories people wrote from the outlines
ai_original50the AI stories those outlines came from

A rewrite and its original share story. load_dataset("rishanthrajendhran/TwiceTold", split="human_rewrite") loads the rewrites.

Fields

FieldDescription
idtwicetold_NN_human (the rewrite) or twicetold_NN_ai (the original AI story)
storypair number (1-50): a rewrite and its original share it
kindwhich of the two this row is
prose_authorhuman for rewrites, ai for originals
idea_authorai for both
writerpseudonymous writer id (rewrites only)
genrethe story's genre
generation_promptthe prompt that produced the original AI story
writer_outlinethe outline the writer worked from, one [Role] content line per item
words, textthe story
idealens_p_ai, proselens_p_aiIdeaLens and ProseLens P(AI), as reported in the paper
editlens_score, pangram4_p_aiEditLens-3B score; Pangram 4 weighted P(AI), its AI fraction plus half its AI-assisted fraction (flagged as AI at 0.5 or above)

Caveats

  • Fifty stories in one domain (short fiction) is a small evaluation set.

Scoring

The idealens package (PyPI) runs IdeaLens, ProseLens and the other released detectors on these documents: pip install idealens, then idealens run on a JSONL file with a text field.

All IdeaLens models and datasets are in the IdeaLens collection.

License

The dataset, including the human-written stories, is released under CC BY-NC-SA 4.0: free to share and adapt for non-commercial purposes, with attribution and under the same license. The original AI stories and outlines were produced with OpenAI models.

Citation

@article{idealens2026,
  title         = {IdeaLens: Detecting AI Ideas in Long-form Writing},
  author        = {Rajendhran, Rishanth and Choi, Minjoon and Russell, Jenna and Namuduri, Ramya and B{\"o}l{\"o}ni-Turgut, Deniz and Karpinska, Marzena and Wieting, John and Iyyer, Mohit},
  journal       = {arXiv preprint arXiv:2610.06778},
  year          = {2026},
  eprint        = {2610.06778},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.06778}
}
ai-text-detection
creative-writing
idea-provenance

rishanthrajendhran/TwiceTold

Dataset

TwiceTold

0

10 commits

1 linked in READMEs

updated Oct 6, 2026

See the code

README

TwiceTold

TwiceTold holds 50 short stories that people wrote from scratch, each working only from the outline of an AI-generated story, together with the 50 AI stories those outlines came from. The ideas in every story are the AI's. The prose of the 50 rewrites is human. A detector of who wrote the prose should call the rewrites human. A detector of whose ideas they are should call them AI. The name comes from Nathaniel Hawthorne's Twice-Told Tales.

Construction

  • AI stories: GPT-5.6 Sol (high reasoning effort) wrote a 500-word story for each of 50 genres, from the prompt "Write me a 500 word story in {genre}".
  • Outlines: an outline of each AI story was extracted: an ordered list of [Role] content items.
  • Rewrites: 17 computer science graduate students and faculty each received one or more outlines, and wrote the story in person rather than through crowdsourcing, to reduce the chance of AI use during writing.

Writers are identified only by pseudonymous ids (W01-W17). No other information about them is included.

Splits

SplitRowsWhat it holds
human_rewrite50the stories people wrote from the outlines
ai_original50the AI stories those outlines came from

A rewrite and its original share story. load_dataset("rishanthrajendhran/TwiceTold", split="human_rewrite") loads the rewrites.

Fields

FieldDescription
idtwicetold_NN_human (the rewrite) or twicetold_NN_ai (the original AI story)
storypair number (1-50): a rewrite and its original share it
kindwhich of the two this row is
prose_authorhuman for rewrites, ai for originals
idea_authorai for both
writerpseudonymous writer id (rewrites only)
genrethe story's genre
generation_promptthe prompt that produced the original AI story
writer_outlinethe outline the writer worked from, one [Role] content line per item
words, textthe story
idealens_p_ai, proselens_p_aiIdeaLens and ProseLens P(AI), as reported in the paper
editlens_score, pangram4_p_aiEditLens-3B score; Pangram 4 weighted P(AI), its AI fraction plus half its AI-assisted fraction (flagged as AI at 0.5 or above)

Caveats

  • Fifty stories in one domain (short fiction) is a small evaluation set.

Scoring

The idealens package (PyPI) runs IdeaLens, ProseLens and the other released detectors on these documents: pip install idealens, then idealens run on a JSONL file with a text field.

All IdeaLens models and datasets are in the IdeaLens collection.

License

The dataset, including the human-written stories, is released under CC BY-NC-SA 4.0: free to share and adapt for non-commercial purposes, with attribution and under the same license. The original AI stories and outlines were produced with OpenAI models.

Citation

@article{idealens2026,
  title         = {IdeaLens: Detecting AI Ideas in Long-form Writing},
  author        = {Rajendhran, Rishanth and Choi, Minjoon and Russell, Jenna and Namuduri, Ramya and B{\"o}l{\"o}ni-Turgut, Deniz and Karpinska, Marzena and Wieting, John and Iyyer, Mohit},
  journal       = {arXiv preprint arXiv:2610.06778},
  year          = {2026},
  eprint        = {2610.06778},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.06778}
}
ai-text-detection
creative-writing
idea-provenance