Companion dataset for imbue-ai/ai-detection-demo
— a research artifact comparing logit-based vs hidden-state-based
classification heads for AI-text-edit detection.
Each row is one of three types:
human_written — verbatim text from a pre-2022 public dataset (so we can
treat it as genuinely human-authored, pre-ChatGPT).ai_edited — the human source rewritten by an LLM following one of 301
editing instructions.ai_generated — a fresh LLM write, given only metadata extracted from
the human source (Emi & Spero-style synthetic mirroring) — anchored to the
same topic and length as the human text but with no copied wording.| Split | Rows |
|---|---|
train | 75,316 |
val | 3,234 |
test | 7,658 |
Each split is prompt-disjoint — no editing prompt appears in two splits.
| column | type | meaning |
|---|---|---|
text_id | str | md5 of the text |
text | str | the input text |
text_type | str | human_written / ai_edited / ai_generated |
label | int | 0 = human, 1 = ai_generated, -1 = ai_edited |
cosine_score | float | Linq-Embed-Mistral cosine distance from text to its human source — the supervision signal for the 4-bucket head |
model | str | generator: claude-sonnet-4-6, gpt-5.3-chat-latest, gemini-3-flash-preview, or human |
provider | str | claude / gpt / gemini / human |
source | str | underlying source domain (see attribution below) |
source_id | str | md5 of the human source |
source_text | str | the human-authored source text this row is based on |
source_word_count | int | length of source_text in words |
edit_prompt | str | the editing instruction (only set for ai_edited) |
extracted_metadata | str | the metadata used to seed ai_generated (per-domain JSON or string) |
split | str | train / val / test |
All human text is sourced from established public HuggingFace datasets, with pre-2022 publication dates (pre-ChatGPT) so that human-labeled rows are genuinely human-authored.
source column | underlying public dataset | original citation |
|---|---|---|
amazon_reviews | amazon_polarity | Zhang et al. 2015 |
yelp_reviews | Yelp/yelp_review_full | Zhang et al. 2015 |
reddit_writing_prompts | euclaise/writingprompts | Fan et al. 2018 |
fineweb_edu | HuggingFaceFW/fineweb-edu (sample-10BT) | Lozhkov et al. 2024 |
news | EdinburghNLP/xsum + abisee/cnn_dailymail | Narayan 2018; See 2017 |
twitter | cardiffnlp/tweet_topic_multi, filtered to pre-Nov-2022 | Antypas et al. 2022 |
For every human source row, the dataset includes one ai_edited and one
ai_generated counterpart, with the generator model assigned by round-robin
across claude-sonnet-4-6, gpt-5.3-chat-latest, gemini-3-flash-preview.
ai_edited wraps the human source in:
"Apply the following editing instruction to the text below. Output ONLY the edited text — no preamble, no explanation, no headers, no markdown. Editing instruction: {prompt} Text to edit: {source}"
and sends to the assigned model. Prompts are drawn uniformly from a
301-prompt pool covering tone & style, concision, fluency, paraphrasing, etc.
ai_generated uses a two-stage Emi-&-Spero-style mirroring:
[WP] prompt for writing prompts, topic for tweets, etc.).So ai_generated text shares topic and length with its human source but no
copied wording — which is why the cosine distance for ai_generated rows is
near 1.0 (semantically distinct, even though topically related).
The cosine score is discretized into 4 buckets via thresholds lo=0.03,
hi=0.15:
cosine ≤ 0.03 — undetectable edit, supervised as "looks human"0.03 < cosine < 0.090.09 ≤ cosine < 0.15cosine ≥ 0.15 — heavy edit / fresh generationApproximate bucket distribution for ai_edited rows: 25% / 39% / 20% / 16%.
human_written rows are all bucket 0 (cosine = 0). ai_generated rows are
all bucket 3 (cosine ≈ 1).
CC BY-NC-SA 4.0. Non-commercial research use. Derivatives must use the same license.
Companion dataset for imbue-ai/ai-detection-demo
— a research artifact comparing logit-based vs hidden-state-based
classification heads for AI-text-edit detection.
Each row is one of three types:
human_written — verbatim text from a pre-2022 public dataset (so we can
treat it as genuinely human-authored, pre-ChatGPT).ai_edited — the human source rewritten by an LLM following one of 301
editing instructions.ai_generated — a fresh LLM write, given only metadata extracted from
the human source (Emi & Spero-style synthetic mirroring) — anchored to the
same topic and length as the human text but with no copied wording.| Split | Rows |
|---|---|
train | 75,316 |
val | 3,234 |
test | 7,658 |
Each split is prompt-disjoint — no editing prompt appears in two splits.
| column | type | meaning |
|---|---|---|
text_id | str | md5 of the text |
text | str | the input text |
text_type | str | human_written / ai_edited / ai_generated |
label | int | 0 = human, 1 = ai_generated, -1 = ai_edited |
cosine_score | float | Linq-Embed-Mistral cosine distance from text to its human source — the supervision signal for the 4-bucket head |
model | str | generator: claude-sonnet-4-6, gpt-5.3-chat-latest, gemini-3-flash-preview, or human |
provider | str | claude / gpt / gemini / human |
source | str | underlying source domain (see attribution below) |
source_id | str | md5 of the human source |
source_text | str | the human-authored source text this row is based on |
source_word_count | int | length of source_text in words |
edit_prompt | str | the editing instruction (only set for ai_edited) |
extracted_metadata | str | the metadata used to seed ai_generated (per-domain JSON or string) |
split | str | train / val / test |
All human text is sourced from established public HuggingFace datasets, with pre-2022 publication dates (pre-ChatGPT) so that human-labeled rows are genuinely human-authored.
source column | underlying public dataset | original citation |
|---|---|---|
amazon_reviews | amazon_polarity | Zhang et al. 2015 |
yelp_reviews | Yelp/yelp_review_full | Zhang et al. 2015 |
reddit_writing_prompts | euclaise/writingprompts | Fan et al. 2018 |
fineweb_edu | HuggingFaceFW/fineweb-edu (sample-10BT) | Lozhkov et al. 2024 |
news | EdinburghNLP/xsum + abisee/cnn_dailymail | Narayan 2018; See 2017 |
twitter | cardiffnlp/tweet_topic_multi, filtered to pre-Nov-2022 | Antypas et al. 2022 |
For every human source row, the dataset includes one ai_edited and one
ai_generated counterpart, with the generator model assigned by round-robin
across claude-sonnet-4-6, gpt-5.3-chat-latest, gemini-3-flash-preview.
ai_edited wraps the human source in:
"Apply the following editing instruction to the text below. Output ONLY the edited text — no preamble, no explanation, no headers, no markdown. Editing instruction: {prompt} Text to edit: {source}"
and sends to the assigned model. Prompts are drawn uniformly from a
301-prompt pool covering tone & style, concision, fluency, paraphrasing, etc.
ai_generated uses a two-stage Emi-&-Spero-style mirroring:
[WP] prompt for writing prompts, topic for tweets, etc.).So ai_generated text shares topic and length with its human source but no
copied wording — which is why the cosine distance for ai_generated rows is
near 1.0 (semantically distinct, even though topically related).
The cosine score is discretized into 4 buckets via thresholds lo=0.03,
hi=0.15:
cosine ≤ 0.03 — undetectable edit, supervised as "looks human"0.03 < cosine < 0.090.09 ≤ cosine < 0.15cosine ≥ 0.15 — heavy edit / fresh generationApproximate bucket distribution for ai_edited rows: 25% / 39% / 20% / 16%.
human_written rows are all bucket 0 (cosine = 0). ai_generated rows are
all bucket 3 (cosine ≈ 1).
CC BY-NC-SA 4.0. Non-commercial research use. Derivatives must use the same license.