hotchpotch/bekko-system-one-dataset-v0

Dataset

Bekko System One dataset v0

5

12 commits

26 linked in READMEs

updated Oct 8, 2026

See the code

README

Bekko System One dataset v0

This dataset is used to train the bekko-system-one-v0 model family.

Public Hugging Face dataset distribution.

Loading and storage

from datasets import load_dataset

dataset = load_dataset("hotchpotch/bekko-system-one-dataset-v0", "laya__typed_decisions")
test = dataset["test"]

Select a configuration explicitly. Parquet uses Zstandard compression and dictionary encoding; all decoded values and row order match the local Arrow release. Only training-manifest.json defines active membership. Configurations listed in quarantine-manifest.json are retained for audit and must not enter evaluation. Source attribution and license scope are retained in sources.json, SOURCES.md, dataset_metadata.json and _sources/. README has one leading YAML block listing every config and split. Hub manifests point to Parquet files; _sources/local-arrow/ preserves the historical Arrow manifests.

Typed decisions over shared state, with Choice, Noul and numeric Score supervision.

Train uses a 300,000-judgment cap per subset with complete cases/groups. Validation is sampled from source validation. Test matches the active S1MB dataset. Use training-manifest.json for membership.

RoleSubsetsCasesJudgments
train1536,589,1888,421,771
evaluation12522,21041,938
unlabeled000

Columns

ColumnPurpose
inputShared state and typed decisions; the only inference input.
targetsAligned target distributions and annotation metadata.
splittrain, validation or test.
case_idStable case identifier.
group_idRelated-case grouping.
languageLanguage code.
input_hashStable input identity for audits.

Score values come from the explicit numeric criteria, not candidate indices. Noul retains authored false/true definitions. Preserve soft targets and candidate order. Repeated strings use physical dictionary encoding; decoded values remain strings.

Sources

See sources.json, SOURCES.md and dataset_metadata.json for upstream datasets, revisions, attribution and license scope. The laya__ namespace denotes the Laya-source conversion pipeline; it does not imply Laya authored those upstream datasets. laya__typed_decisions contains four synthetic workflows published by LocalLLaMA. No blanket relicensing is applied. namespace-map.json records names; identifiers remain stable.

Counts below are cases in this release, not upstream dataset sizes or judgment counts. A case can contain multiple judgments. Counts use the active training-manifest.json; 0 means that split is not included. MNLI validation combines validation_matched and validation_mismatched. Source links include acquisition datasets and original dataset references where recorded; they do not imply direct acquisition from each original.

Licenses below describe the original datasets. Follow the links for their full terms. Unknown means the data license has not been confirmed.

ConfigUpstream dataset / sourceTrainValidationTestLicense / usage terms
abductive_nliallenai/art166,9371000Unknown
aegis2nvidia/Aegis-AI-Content-Safety-Dataset-2.027,060100100CC BY 4.0
agnewssentence-transformers/agnews
fancyzhx/ag_news
19,99200Unknown
all_nlisentence-transformers/all-nli20,00000SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
aqua_ratdeepmind/aqua_rat97,113100100Apache-2.0
arcallenai/ai2_arc3,352100100CC BY-SA 4.0
arcttasksource/arct1,210100100Apache-2.0
argument_qualityibm-research/argument_quality_ranking_30k20,972100100CC BY 3.0 (conflicting source notices)
arxivhotchpotch/arxiv-ir-simulated-search-queries
open-index/open-arxiv
20,00000CC0 (metadata); paper licenses vary
babi_nlitasksource/babi_nli18,590100100BSD
banking77PolyAI/banking7710,0030100CC BY 4.0
bbqlighteval/bbq_helm00100CC BY 4.0
bge-hn-collieeShitao/bge-m3-data: colliee
COLIEE (original benchmark)
46300Unknown
bge-hn-hotpotqaShitao/bge-m3-data: hotpotqa
hotpotqa/hotpot_qa
84,17600CC BY-SA 4.0
bge-hn-miracl_enShitao/bge-m3-data: miracl_en
miracl/miracl
2,86100Apache-2.0; Wikipedia text: CC BY-SA / GFDL
bge-hn-mr-tydi_englishShitao/bge-m3-data: mr-tydi_english
castorini/mr-tydi
3,51300Apache-2.0; Wikipedia text: CC BY-SA / GFDL
bge-hn-msmarcoShitao/bge-m3-data: msmarco
microsoft/ms_marco
300,00000Non-commercial research only
bge-hn-nli_for_simcseShitao/bge-m3-data: nli_for_simcse
SimCSE NLI (SNLI + MultiNLI)
274,85900SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
bge-hn-nqShitao/bge-m3-data: nq
google-research-datasets/natural_questions
58,55100CC BY-SA 3.0
bge-hn-pubmed_qa_labeledShitao/bge-m3-data: pubmed_qa_labeled
qiaojin/PubMedQA
50000MIT (dataset); abstract rights vary
bge-hn-squadShitao/bge-m3-data: squad
rajpurkar/squad
87,52600CC BY-SA 4.0
bge-hn-triviaShitao/bge-m3-data: trivia
mandarjoshi/trivia_qa
60,29800Apache-2.0
bitextbitext/Bitext-customer-support-llm-chatbot-training-dataset24,63500cdla-sharing-1.0
blimpnyu-mll/blimp66,99900CC BY 4.0
boardgameqatasksource/Boardgame-QA15,000100100CC BY 4.0
boolqgoogle/boolq7,9931000CC BY-SA 3.0
canttalknvidia/CantTalkAboutThis-Topic-Control-Dataset10,8180100CC BY 4.0
circagoogle-research-datasets/circa34,01500CC BY 4.0
civil_commentsgoogle/civil_comments42,857100100CC0-1.0
cladderhttps://github.com/causalNLP/cladder00100MIT
clincclinc/clinc_oos15,189151151CC BY 3.0
coco_captionssentence-transformers/coco-captions20,00000CC BY 4.0 (captions)
codesearchnomic-ai/nomic-embed-unsupervised-data
CodeSearchNet
19,73100MIT (tools); source-code licenses vary
commonsense_qatau/commonsense_qa9,7411000MIT
commonsense_qa2tasksource/commonsense_qa_2.09,2641000CC BY 4.0
contract_nlihttps://github.com/stanfordnlp/contract-nli42261100CC BY 4.0
copaaps/super_glue3991000BSD-2-Clause (COPA)
corr2causetasksource/corr2cause274,168100100MIT
cosmos_qaSamsoup/cosmos_qa25,2011000CC BY 4.0
creakamydeng2000/CREAK10,173100100MIT (repository); data license unconfirmed
crows_pairshttps://github.com/nyu-mll/crows-pairs00100CC BY-SA 4.0
dbpediafancyzhx/dbpedia_14300,0000100CC BY-SA 3.0
defeasible_nlitasksource/defeasible-nli88,448100100Apache-2.0 / CC BY-SA 4.0
escitasksource/esci300,0000100Apache-2.0
ethicshendrycks/ethics87,143100100MIT
ethosSetFit/ethos_binary5980100GPL-3.0
feasibility_qatasksource/feasibilityQA3,23300MIT
few_nerdDFKI-SLT/few-nerd100,506100100CC BY-SA 4.0
fig_qanightingal3/fig-qa7,9621000MIT
finewebhotchpotch/fineweb-ir-simulated-search-queries
HuggingFaceFW/fineweb-edu
19,98700ODC-By 1.0 (database); source-page rights vary
fol_nlitasksource/FOL-nli82,188100100Apache-2.0
followirmteb/Core17InstructionRetrieval
mteb/News21InstructionRetrieval
mteb/Robust04InstructionRetrieval
jhu-clsp/FollowIR-train
1,72500MIT (annotations); NYT corpus access terms
MIT (annotations); Washington Post access terms
MIT (annotations); TREC research-access terms
Apache-2.0
followir_core17mteb/Core17InstructionRetrieval00100MIT (annotations); NYT corpus access terms
followir_news21mteb/News21InstructionRetrieval00100MIT (annotations); Washington Post access terms
followir_robust04mteb/Robust04InstructionRetrieval00100MIT (annotations); TREC research-access terms
go_emotionsgoogle-research-datasets/go_emotions10,714100100Apache-2.0
gooaqsentence-transformers/gooaq
allenai/gooaq
19,99300Non-commercial use only
gretel_piigretelai/gretel-pii-masking-en-v148,221100100Apache-2.0
gsm8kopenai/gsm8k7,4730100MIT
hanstasksource/hans29,851100100MIT
hatecheckPaul/hatecheck00100CC BY 4.0
helpsteernvidia/HelpSteer32,8721000CC BY 4.0
helpsteer2nvidia/HelpSteer220,3211000CC BY 4.0
helpsteer3nvidia/HelpSteer337,8441000CC BY 4.0
hh_rlhfAnthropic/hh-rlhf160,1490100MIT
hwu64FastFit/hwu_647,251100100CC BY 4.0
implihttps://github.com/UKPLab/acl2022-impli00100Apache-2.0
laya__ag_newsfancyzhx/ag_news120,0000100Unknown
laya__enron_spamSetFit/enron_spam28,1420100Unknown
laya__phishingzefang-liu/phishing-email-dataset17,53500LGPL-3.0
laya__typed_decisionsLocalLLaMA/typed-decisions1,2000100Apache-2.0
lexcomptasksource/lexcomp-nc-relation1,272100100Apache-2.0
logical_entailmenttasksource/logical-entailment99,754100100Apache-2.0
lonlitasksource/lonli00100MIT
massiveAmazonScience/massive10,463100100CC BY 4.0
measuring_hateucberkeley-dlab/measuring-hate-speech30,00000CC BY 4.0
minds14PolyAI/minds141,66300CC BY 4.0
miqatasksource/miqa00100CC BY 4.0
mnlinyu-mll/multi_nli300,0001000CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
monotonicitytasksource/monotonicity-entailment5,37900Apache-2.0
mtoptasksource/mtop15,597100100CC BY-SA 4.0
natural_logictasksource/naturallogic6,38600Apache-2.0
natural_questionssentence-transformers/natural-questions
google-research-datasets/natural_questions
19,94800CC BY-SA 3.0
nlsattasksource/nlsat160,000100100Apache-2.0
nlupphttps://github.com/PolyAI-LDN/task-specific-datasets3,04000CC BY 4.0
oasst1OpenAssistant/oasst121,8431000Apache-2.0
open_jev__amount-extraction-control-v1ZefanCai/Open-Jev32,984100100CC0-1.0
open_jev__browser-control-v1ZefanCai/Open-Jev13,814100100CC0-1.0
open_jev__citation-control-v1ZefanCai/Open-Jev2,520100100CC0-1.0
open_jev__context-retention-control-v1ZefanCai/Open-Jev6,138100100CC0-1.0
open_jev__customer-control-v1ZefanCai/Open-Jev4,206100100CC0 (conversations); question license unknown
open_jev__drone-control-v1ZefanCai/Open-Jev15,694100100CC0-1.0
open_jev__email-selection-control-v1ZefanCai/Open-Jev3,618100100CC0-1.0
open_jev__entity-alignment-control-v1ZefanCai/Open-Jev6,944100100CC0-1.0
open_jev__ir-control-v1ZefanCai/Open-Jev7,366100100CC0-1.0
open_jev__mailroom-control-v1ZefanCai/Open-Jev73,472100100CC0-1.0
open_jev__painting-geometry-v1ZefanCai/Open-Jev23,552100100CC0-1.0
open_jev__phone-extraction-control-v1ZefanCai/Open-Jev12,350100100CC0-1.0
open_jev__reasoning-control-v1ZefanCai/Open-Jev5,442100100CC0-1.0
open_jev__silent-failure-control-v1ZefanCai/Open-Jev6,432100100CC0-1.0
open_jev__snake-v1ZefanCai/Open-Jev12,508100100CC0-1.0
open_jev__sponsor-segment-control-v1ZefanCai/Open-Jev6,345100100CC0-1.0
open_jev__tic_tac_toe-v1ZefanCai/Open-Jev3,264100100CC0-1.0
open_jev__tile_platformer-v1ZefanCai/Open-Jev1,34600CC0-1.0
open_jev__trex_runner-v1ZefanCai/Open-Jev2800CC0-1.0
open_jev__vizdoom-basic-v1ZefanCai/Open-Jev6,35410066CC0 (numeric data)
open_jev__workflow-controls-v1__agent_trace_observabilityZefanCai/Open-Jev3,780100100CC0-1.0
open_jev__workflow-controls-v1__customer_serviceZefanCai/Open-Jev4,392100100CC0-1.0
open_jev__workflow-controls-v1__invoice_processingZefanCai/Open-Jev5,370100100CC0-1.0
open_jev__workflow-controls-v1__security_incidentsZefanCai/Open-Jev8,874100100CC0-1.0
openbookqaallenai/openbookqa4,870100100Apache-2.0
patent_similaritytasksource/patent-phrase-similarity36,473100100CC BY 4.0
pawsgoogle-research-datasets/paws49,334100100Google custom license; Wikipedia text: CC BY-SA / GFDL
piqalighteval/piqa15,0411000AFL-3.0
planetasksource/PLANE-ood161,0730100CC BY 2.0
poem_sentimentgoogle-research-datasets/poem_sentiment892100100CC BY 4.0
pubmedhotchpotch/pubmed-abstract-ir-simulated-search-queries
slinusc/PubMedAbstractsSubset
19,99300CC0 (metadata); abstract rights vary
qascallenai/qasc8,1291000CC BY 4.0
qasperallenai/qasper39289100CC BY 4.0
quartzallenai/quartz2,672100100CC BY 4.0
robust_lrtasksource/robustLR00100MIT
ropesallenai/ropes6,3791000CC BY 4.0
ruletakertasksource/ruletaker300,000100100Apache-2.0
s1mb-generalization-contextual-choiceAuthored evaluation source00100not separately declared
s1mb-generalization-contextual-noulAuthored evaluation source00100not separately declared
s1mb-generalization-contextual-scoreAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-choiceAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-noulAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-scoreAuthored evaluation source00100not separately declared
scicitetasksource/scicite6,099100100Unknown
scitailallenai/scitail23,563100100Apache-2.0
sconetasksource/scone4,9860100CC0-1.0
sdoh_nlitasksource/SDOH-NLI20,465100100CC BY 4.0
sgdhttps://github.com/google-research-datasets/dstc8-schema-guided-dialogue173,481100100CC BY-SA 4.0
sms_spamucirvine/sms_spam5,17100CC BY 4.0
snipshttps://github.com/sonos/nlu-benchmark13,6211000CC0-1.0
snlistanfordnlp/snli300,000100100CC BY-SA 4.0
social_iqababer/social_i_qa33,1931000CC BY 4.0
space_nlitasksource/SpaceNLI31,87700MIT
spartqatasksource/spartqa-mchoice22,472100100MIT
squad2rajpurkar/squad_v2129,6681000CC BY-SA 4.0
stepgametasksource/stepgame295,827100100MIT
strategyqatasksource/strategy-qa2,28900MIT
synthetic_relevance__agnewssentence-transformers/agnews
fancyzhx/ag_news
4,80000Unknown
synthetic_relevance__all_nlisentence-transformers/all-nli4,80000SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
synthetic_relevance__arxivhotchpotch/arxiv-ir-simulated-search-queries
open-index/open-arxiv
4,80000CC0 (metadata); paper licenses vary
synthetic_relevance__bge_m3__hotpotqaShitao/bge-m3-data: hotpotqa
hotpotqa/hotpot_qa
4,80000CC BY-SA 4.0
synthetic_relevance__bge_m3__miracl_enShitao/bge-m3-data: miracl_en
miracl/miracl
4,80000Apache-2.0; Wikipedia text: CC BY-SA / GFDL
synthetic_relevance__bge_m3__mr-tydi_englishShitao/bge-m3-data: mr-tydi_english
castorini/mr-tydi
4,80000Apache-2.0; Wikipedia text: CC BY-SA / GFDL
synthetic_relevance__bge_m3__msmarcoShitao/bge-m3-data: msmarco
microsoft/ms_marco
4,60800Non-commercial research only
synthetic_relevance__bge_m3__nqShitao/bge-m3-data: nq
google-research-datasets/natural_questions
4,80000CC BY-SA 3.0
synthetic_relevance__bge_m3__pubmed_qa_labeledShitao/bge-m3-data: pubmed_qa_labeled
qiaojin/PubMedQA
4,00000MIT (dataset); abstract rights vary
synthetic_relevance__bge_m3__squadShitao/bge-m3-data: squad
rajpurkar/squad
4,80000CC BY-SA 4.0
synthetic_relevance__bge_m3__triviaShitao/bge-m3-data: trivia
mandarjoshi/trivia_qa
4,80000Apache-2.0
synthetic_relevance__coco_captionssentence-transformers/coco-captions4,80000CC BY 4.0 (captions)
synthetic_relevance__codesearchnomic-ai/nomic-embed-unsupervised-data
CodeSearchNet
4,80000MIT (tools); source-code licenses vary
synthetic_relevance__finewebhotchpotch/fineweb-ir-simulated-search-queries
HuggingFaceFW/fineweb-edu
4,80000ODC-By 1.0 (database); source-page rights vary
synthetic_relevance__gooaqsentence-transformers/gooaq
allenai/gooaq
4,80000Non-commercial use only
synthetic_relevance__nanobeir__NanoArguAnaNanoArguAna00400CC BY 4.0
synthetic_relevance__nanobeir__NanoDBPediaNanoDBPedia00400CC BY-SA 3.0 / GFDL
synthetic_relevance__nanobeir__NanoFiQA2018NanoFiQA201800400Non-commercial use only / CC BY-SA 2.5 / CC BY-SA 3.0
synthetic_relevance__nanobeir__NanoHotpotQANanoHotpotQA00400CC BY-SA 4.0
synthetic_relevance__nanobeir__NanoMSMARCONanoMSMARCO00400Non-commercial research only
synthetic_relevance__nanobeir__NanoNFCorpusNanoNFCorpus00400Unknown
synthetic_relevance__nanobeir__NanoNQNanoNQ00400CC BY-SA 3.0
synthetic_relevance__nanobeir__NanoSCIDOCSNanoSCIDOCS00400CC BY 4.0
synthetic_relevance__nanobeir__NanoTouche2020NanoTouche202000392CC BY 4.0
synthetic_relevance__nanocoirNanoApps
NanoCodeFeedbackMT
NanoCodeFeedbackST
NanoCodeSearchNet
NanoCodeSearchNetCCR
NanoCodeTransOceanContest
NanoCodeTransOceanDL
NanoCosQA
NanoStackOverflowQA
NanoSyntheticText2SQL
00800Non-commercial research only
Apache-2.0
Non-commercial research only; source-code licenses vary
CC BY-SA 3.0
synthetic_relevance__natural_questionssentence-transformers/natural-questions
google-research-datasets/natural_questions
4,80000CC BY-SA 3.0
synthetic_relevance__pubmedhotchpotch/pubmed-abstract-ir-simulated-search-queries
slinusc/PubMedAbstractsSubset
4,80000CC0 (metadata); abstract rights vary
synthetic_relevance__trivia_qasentence-transformers/trivia-qa
mandarjoshi/trivia_qa
4,80000Apache-2.0
synthetic_relevance__wikipedia_hard_negatives_englishhotchpotch/wikipedia-english-ir-simulated-search-queries4,80000CC BY-SA 4.0 / GFDL
taskmasterhttps://github.com/google-research-datasets/Taskmaster17,29500CC BY 4.0
temporal_nlitasksource/temporal-nli72,3220100Apache-2.0
tracietasksource/tracie1,1740100Apache-2.0
trivia_qasentence-transformers/trivia-qa
mandarjoshi/trivia_qa
18,77500Apache-2.0
ud_ewtuniversal-dependencies/universal_dependencies11,663100100CC BY-SA 4.0
wikipedia_hard_negatives_englishhotchpotch/wikipedia-english-ir-simulated-search-queries300,00000CC BY-SA 4.0 / GFDL
winobiashttps://github.com/uclanlp/corefBias00100MIT
winograndeallenai/winogrande40,3921000Apache-2.0
winowhytasksource/winowhy00100MIT
wiqaallenai/wiqa29,324100100Apache-2.0

License

Each subset follows its original dataset's license and usage terms. Check the linked original sources before using or redistributing the data. This collection does not change those licenses.

Where noted, code, annotations and source documents have different terms. Detailed source records are in sources.json. An unknown license is not permission to reuse the data.

For Open-Jev, retain its acquisition attribution and third-party notices. In open_jev__customer-control-v1, the generated conversations have a CC0-1.0 declaration, but the upstream question descriptions have no verified license and are not relicensed as CC0. Authored S1MB generalization subsets retain their recorded not separately declared status. See sources.json, SOURCES.md and the retained upstream notices for scope and qualifications.

Training decontamination (2026-10-08)

Removed two complete training cases whose full inputs match active S1MB test cases after normalizing whitespace: contract_nli:default:train:366 (17 decisions) and enron_spam:default:train:18716 (1 decision, configuration laya__enron_spam). ContractNLI also has an upstream train/test target disagreement on nda-1. Test and validation membership, retained rows, candidate order and targets are unchanged. See decontamination-20261008.json for the parent revision, verification and removed IDs. This is a targeted deletion, not a claim of comprehensive semantic decontamination. Historical Arrow/source/export statistics remain in _sources/; current counts are in the active manifests and release_splits.

choice
noul
score
typed-decisions

Linked in READMEs

hotchpotch/bekko-system-one-dataset-v0

Dataset

Bekko System One dataset v0

5

12 commits

26 linked in READMEs

updated Oct 8, 2026

See the code

README

Bekko System One dataset v0

This dataset is used to train the bekko-system-one-v0 model family.

Public Hugging Face dataset distribution.

Loading and storage

from datasets import load_dataset

dataset = load_dataset("hotchpotch/bekko-system-one-dataset-v0", "laya__typed_decisions")
test = dataset["test"]

Select a configuration explicitly. Parquet uses Zstandard compression and dictionary encoding; all decoded values and row order match the local Arrow release. Only training-manifest.json defines active membership. Configurations listed in quarantine-manifest.json are retained for audit and must not enter evaluation. Source attribution and license scope are retained in sources.json, SOURCES.md, dataset_metadata.json and _sources/. README has one leading YAML block listing every config and split. Hub manifests point to Parquet files; _sources/local-arrow/ preserves the historical Arrow manifests.

Typed decisions over shared state, with Choice, Noul and numeric Score supervision.

Train uses a 300,000-judgment cap per subset with complete cases/groups. Validation is sampled from source validation. Test matches the active S1MB dataset. Use training-manifest.json for membership.

RoleSubsetsCasesJudgments
train1536,589,1888,421,771
evaluation12522,21041,938
unlabeled000

Columns

ColumnPurpose
inputShared state and typed decisions; the only inference input.
targetsAligned target distributions and annotation metadata.
splittrain, validation or test.
case_idStable case identifier.
group_idRelated-case grouping.
languageLanguage code.
input_hashStable input identity for audits.

Score values come from the explicit numeric criteria, not candidate indices. Noul retains authored false/true definitions. Preserve soft targets and candidate order. Repeated strings use physical dictionary encoding; decoded values remain strings.

Sources

See sources.json, SOURCES.md and dataset_metadata.json for upstream datasets, revisions, attribution and license scope. The laya__ namespace denotes the Laya-source conversion pipeline; it does not imply Laya authored those upstream datasets. laya__typed_decisions contains four synthetic workflows published by LocalLLaMA. No blanket relicensing is applied. namespace-map.json records names; identifiers remain stable.

Counts below are cases in this release, not upstream dataset sizes or judgment counts. A case can contain multiple judgments. Counts use the active training-manifest.json; 0 means that split is not included. MNLI validation combines validation_matched and validation_mismatched. Source links include acquisition datasets and original dataset references where recorded; they do not imply direct acquisition from each original.

Licenses below describe the original datasets. Follow the links for their full terms. Unknown means the data license has not been confirmed.

ConfigUpstream dataset / sourceTrainValidationTestLicense / usage terms
abductive_nliallenai/art166,9371000Unknown
aegis2nvidia/Aegis-AI-Content-Safety-Dataset-2.027,060100100CC BY 4.0
agnewssentence-transformers/agnews
fancyzhx/ag_news
19,99200Unknown
all_nlisentence-transformers/all-nli20,00000SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
aqua_ratdeepmind/aqua_rat97,113100100Apache-2.0
arcallenai/ai2_arc3,352100100CC BY-SA 4.0
arcttasksource/arct1,210100100Apache-2.0
argument_qualityibm-research/argument_quality_ranking_30k20,972100100CC BY 3.0 (conflicting source notices)
arxivhotchpotch/arxiv-ir-simulated-search-queries
open-index/open-arxiv
20,00000CC0 (metadata); paper licenses vary
babi_nlitasksource/babi_nli18,590100100BSD
banking77PolyAI/banking7710,0030100CC BY 4.0
bbqlighteval/bbq_helm00100CC BY 4.0
bge-hn-collieeShitao/bge-m3-data: colliee
COLIEE (original benchmark)
46300Unknown
bge-hn-hotpotqaShitao/bge-m3-data: hotpotqa
hotpotqa/hotpot_qa
84,17600CC BY-SA 4.0
bge-hn-miracl_enShitao/bge-m3-data: miracl_en
miracl/miracl
2,86100Apache-2.0; Wikipedia text: CC BY-SA / GFDL
bge-hn-mr-tydi_englishShitao/bge-m3-data: mr-tydi_english
castorini/mr-tydi
3,51300Apache-2.0; Wikipedia text: CC BY-SA / GFDL
bge-hn-msmarcoShitao/bge-m3-data: msmarco
microsoft/ms_marco
300,00000Non-commercial research only
bge-hn-nli_for_simcseShitao/bge-m3-data: nli_for_simcse
SimCSE NLI (SNLI + MultiNLI)
274,85900SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
bge-hn-nqShitao/bge-m3-data: nq
google-research-datasets/natural_questions
58,55100CC BY-SA 3.0
bge-hn-pubmed_qa_labeledShitao/bge-m3-data: pubmed_qa_labeled
qiaojin/PubMedQA
50000MIT (dataset); abstract rights vary
bge-hn-squadShitao/bge-m3-data: squad
rajpurkar/squad
87,52600CC BY-SA 4.0
bge-hn-triviaShitao/bge-m3-data: trivia
mandarjoshi/trivia_qa
60,29800Apache-2.0
bitextbitext/Bitext-customer-support-llm-chatbot-training-dataset24,63500cdla-sharing-1.0
blimpnyu-mll/blimp66,99900CC BY 4.0
boardgameqatasksource/Boardgame-QA15,000100100CC BY 4.0
boolqgoogle/boolq7,9931000CC BY-SA 3.0
canttalknvidia/CantTalkAboutThis-Topic-Control-Dataset10,8180100CC BY 4.0
circagoogle-research-datasets/circa34,01500CC BY 4.0
civil_commentsgoogle/civil_comments42,857100100CC0-1.0
cladderhttps://github.com/causalNLP/cladder00100MIT
clincclinc/clinc_oos15,189151151CC BY 3.0
coco_captionssentence-transformers/coco-captions20,00000CC BY 4.0 (captions)
codesearchnomic-ai/nomic-embed-unsupervised-data
CodeSearchNet
19,73100MIT (tools); source-code licenses vary
commonsense_qatau/commonsense_qa9,7411000MIT
commonsense_qa2tasksource/commonsense_qa_2.09,2641000CC BY 4.0
contract_nlihttps://github.com/stanfordnlp/contract-nli42261100CC BY 4.0
copaaps/super_glue3991000BSD-2-Clause (COPA)
corr2causetasksource/corr2cause274,168100100MIT
cosmos_qaSamsoup/cosmos_qa25,2011000CC BY 4.0
creakamydeng2000/CREAK10,173100100MIT (repository); data license unconfirmed
crows_pairshttps://github.com/nyu-mll/crows-pairs00100CC BY-SA 4.0
dbpediafancyzhx/dbpedia_14300,0000100CC BY-SA 3.0
defeasible_nlitasksource/defeasible-nli88,448100100Apache-2.0 / CC BY-SA 4.0
escitasksource/esci300,0000100Apache-2.0
ethicshendrycks/ethics87,143100100MIT
ethosSetFit/ethos_binary5980100GPL-3.0
feasibility_qatasksource/feasibilityQA3,23300MIT
few_nerdDFKI-SLT/few-nerd100,506100100CC BY-SA 4.0
fig_qanightingal3/fig-qa7,9621000MIT
finewebhotchpotch/fineweb-ir-simulated-search-queries
HuggingFaceFW/fineweb-edu
19,98700ODC-By 1.0 (database); source-page rights vary
fol_nlitasksource/FOL-nli82,188100100Apache-2.0
followirmteb/Core17InstructionRetrieval
mteb/News21InstructionRetrieval
mteb/Robust04InstructionRetrieval
jhu-clsp/FollowIR-train
1,72500MIT (annotations); NYT corpus access terms
MIT (annotations); Washington Post access terms
MIT (annotations); TREC research-access terms
Apache-2.0
followir_core17mteb/Core17InstructionRetrieval00100MIT (annotations); NYT corpus access terms
followir_news21mteb/News21InstructionRetrieval00100MIT (annotations); Washington Post access terms
followir_robust04mteb/Robust04InstructionRetrieval00100MIT (annotations); TREC research-access terms
go_emotionsgoogle-research-datasets/go_emotions10,714100100Apache-2.0
gooaqsentence-transformers/gooaq
allenai/gooaq
19,99300Non-commercial use only
gretel_piigretelai/gretel-pii-masking-en-v148,221100100Apache-2.0
gsm8kopenai/gsm8k7,4730100MIT
hanstasksource/hans29,851100100MIT
hatecheckPaul/hatecheck00100CC BY 4.0
helpsteernvidia/HelpSteer32,8721000CC BY 4.0
helpsteer2nvidia/HelpSteer220,3211000CC BY 4.0
helpsteer3nvidia/HelpSteer337,8441000CC BY 4.0
hh_rlhfAnthropic/hh-rlhf160,1490100MIT
hwu64FastFit/hwu_647,251100100CC BY 4.0
implihttps://github.com/UKPLab/acl2022-impli00100Apache-2.0
laya__ag_newsfancyzhx/ag_news120,0000100Unknown
laya__enron_spamSetFit/enron_spam28,1420100Unknown
laya__phishingzefang-liu/phishing-email-dataset17,53500LGPL-3.0
laya__typed_decisionsLocalLLaMA/typed-decisions1,2000100Apache-2.0
lexcomptasksource/lexcomp-nc-relation1,272100100Apache-2.0
logical_entailmenttasksource/logical-entailment99,754100100Apache-2.0
lonlitasksource/lonli00100MIT
massiveAmazonScience/massive10,463100100CC BY 4.0
measuring_hateucberkeley-dlab/measuring-hate-speech30,00000CC BY 4.0
minds14PolyAI/minds141,66300CC BY 4.0
miqatasksource/miqa00100CC BY 4.0
mnlinyu-mll/multi_nli300,0001000CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
monotonicitytasksource/monotonicity-entailment5,37900Apache-2.0
mtoptasksource/mtop15,597100100CC BY-SA 4.0
natural_logictasksource/naturallogic6,38600Apache-2.0
natural_questionssentence-transformers/natural-questions
google-research-datasets/natural_questions
19,94800CC BY-SA 3.0
nlsattasksource/nlsat160,000100100Apache-2.0
nlupphttps://github.com/PolyAI-LDN/task-specific-datasets3,04000CC BY 4.0
oasst1OpenAssistant/oasst121,8431000Apache-2.0
open_jev__amount-extraction-control-v1ZefanCai/Open-Jev32,984100100CC0-1.0
open_jev__browser-control-v1ZefanCai/Open-Jev13,814100100CC0-1.0
open_jev__citation-control-v1ZefanCai/Open-Jev2,520100100CC0-1.0
open_jev__context-retention-control-v1ZefanCai/Open-Jev6,138100100CC0-1.0
open_jev__customer-control-v1ZefanCai/Open-Jev4,206100100CC0 (conversations); question license unknown
open_jev__drone-control-v1ZefanCai/Open-Jev15,694100100CC0-1.0
open_jev__email-selection-control-v1ZefanCai/Open-Jev3,618100100CC0-1.0
open_jev__entity-alignment-control-v1ZefanCai/Open-Jev6,944100100CC0-1.0
open_jev__ir-control-v1ZefanCai/Open-Jev7,366100100CC0-1.0
open_jev__mailroom-control-v1ZefanCai/Open-Jev73,472100100CC0-1.0
open_jev__painting-geometry-v1ZefanCai/Open-Jev23,552100100CC0-1.0
open_jev__phone-extraction-control-v1ZefanCai/Open-Jev12,350100100CC0-1.0
open_jev__reasoning-control-v1ZefanCai/Open-Jev5,442100100CC0-1.0
open_jev__silent-failure-control-v1ZefanCai/Open-Jev6,432100100CC0-1.0
open_jev__snake-v1ZefanCai/Open-Jev12,508100100CC0-1.0
open_jev__sponsor-segment-control-v1ZefanCai/Open-Jev6,345100100CC0-1.0
open_jev__tic_tac_toe-v1ZefanCai/Open-Jev3,264100100CC0-1.0
open_jev__tile_platformer-v1ZefanCai/Open-Jev1,34600CC0-1.0
open_jev__trex_runner-v1ZefanCai/Open-Jev2800CC0-1.0
open_jev__vizdoom-basic-v1ZefanCai/Open-Jev6,35410066CC0 (numeric data)
open_jev__workflow-controls-v1__agent_trace_observabilityZefanCai/Open-Jev3,780100100CC0-1.0
open_jev__workflow-controls-v1__customer_serviceZefanCai/Open-Jev4,392100100CC0-1.0
open_jev__workflow-controls-v1__invoice_processingZefanCai/Open-Jev5,370100100CC0-1.0
open_jev__workflow-controls-v1__security_incidentsZefanCai/Open-Jev8,874100100CC0-1.0
openbookqaallenai/openbookqa4,870100100Apache-2.0
patent_similaritytasksource/patent-phrase-similarity36,473100100CC BY 4.0
pawsgoogle-research-datasets/paws49,334100100Google custom license; Wikipedia text: CC BY-SA / GFDL
piqalighteval/piqa15,0411000AFL-3.0
planetasksource/PLANE-ood161,0730100CC BY 2.0
poem_sentimentgoogle-research-datasets/poem_sentiment892100100CC BY 4.0
pubmedhotchpotch/pubmed-abstract-ir-simulated-search-queries
slinusc/PubMedAbstractsSubset
19,99300CC0 (metadata); abstract rights vary
qascallenai/qasc8,1291000CC BY 4.0
qasperallenai/qasper39289100CC BY 4.0
quartzallenai/quartz2,672100100CC BY 4.0
robust_lrtasksource/robustLR00100MIT
ropesallenai/ropes6,3791000CC BY 4.0
ruletakertasksource/ruletaker300,000100100Apache-2.0
s1mb-generalization-contextual-choiceAuthored evaluation source00100not separately declared
s1mb-generalization-contextual-noulAuthored evaluation source00100not separately declared
s1mb-generalization-contextual-scoreAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-choiceAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-noulAuthored evaluation source00100not separately declared
s1mb-generalization-diverse-scoreAuthored evaluation source00100not separately declared
scicitetasksource/scicite6,099100100Unknown
scitailallenai/scitail23,563100100Apache-2.0
sconetasksource/scone4,9860100CC0-1.0
sdoh_nlitasksource/SDOH-NLI20,465100100CC BY 4.0
sgdhttps://github.com/google-research-datasets/dstc8-schema-guided-dialogue173,481100100CC BY-SA 4.0
sms_spamucirvine/sms_spam5,17100CC BY 4.0
snipshttps://github.com/sonos/nlu-benchmark13,6211000CC0-1.0
snlistanfordnlp/snli300,000100100CC BY-SA 4.0
social_iqababer/social_i_qa33,1931000CC BY 4.0
space_nlitasksource/SpaceNLI31,87700MIT
spartqatasksource/spartqa-mchoice22,472100100MIT
squad2rajpurkar/squad_v2129,6681000CC BY-SA 4.0
stepgametasksource/stepgame295,827100100MIT
strategyqatasksource/strategy-qa2,28900MIT
synthetic_relevance__agnewssentence-transformers/agnews
fancyzhx/ag_news
4,80000Unknown
synthetic_relevance__all_nlisentence-transformers/all-nli4,80000SNLI: CC BY-SA 4.0; MultiNLI: CC BY 3.0 / CC BY-SA 3.0 / OANC / public domain
synthetic_relevance__arxivhotchpotch/arxiv-ir-simulated-search-queries
open-index/open-arxiv
4,80000CC0 (metadata); paper licenses vary
synthetic_relevance__bge_m3__hotpotqaShitao/bge-m3-data: hotpotqa
hotpotqa/hotpot_qa
4,80000CC BY-SA 4.0
synthetic_relevance__bge_m3__miracl_enShitao/bge-m3-data: miracl_en
miracl/miracl
4,80000Apache-2.0; Wikipedia text: CC BY-SA / GFDL
synthetic_relevance__bge_m3__mr-tydi_englishShitao/bge-m3-data: mr-tydi_english
castorini/mr-tydi
4,80000Apache-2.0; Wikipedia text: CC BY-SA / GFDL
synthetic_relevance__bge_m3__msmarcoShitao/bge-m3-data: msmarco
microsoft/ms_marco
4,60800Non-commercial research only
synthetic_relevance__bge_m3__nqShitao/bge-m3-data: nq
google-research-datasets/natural_questions
4,80000CC BY-SA 3.0
synthetic_relevance__bge_m3__pubmed_qa_labeledShitao/bge-m3-data: pubmed_qa_labeled
qiaojin/PubMedQA
4,00000MIT (dataset); abstract rights vary
synthetic_relevance__bge_m3__squadShitao/bge-m3-data: squad
rajpurkar/squad
4,80000CC BY-SA 4.0
synthetic_relevance__bge_m3__triviaShitao/bge-m3-data: trivia
mandarjoshi/trivia_qa
4,80000Apache-2.0
synthetic_relevance__coco_captionssentence-transformers/coco-captions4,80000CC BY 4.0 (captions)
synthetic_relevance__codesearchnomic-ai/nomic-embed-unsupervised-data
CodeSearchNet
4,80000MIT (tools); source-code licenses vary
synthetic_relevance__finewebhotchpotch/fineweb-ir-simulated-search-queries
HuggingFaceFW/fineweb-edu
4,80000ODC-By 1.0 (database); source-page rights vary
synthetic_relevance__gooaqsentence-transformers/gooaq
allenai/gooaq
4,80000Non-commercial use only
synthetic_relevance__nanobeir__NanoArguAnaNanoArguAna00400CC BY 4.0
synthetic_relevance__nanobeir__NanoDBPediaNanoDBPedia00400CC BY-SA 3.0 / GFDL
synthetic_relevance__nanobeir__NanoFiQA2018NanoFiQA201800400Non-commercial use only / CC BY-SA 2.5 / CC BY-SA 3.0
synthetic_relevance__nanobeir__NanoHotpotQANanoHotpotQA00400CC BY-SA 4.0
synthetic_relevance__nanobeir__NanoMSMARCONanoMSMARCO00400Non-commercial research only
synthetic_relevance__nanobeir__NanoNFCorpusNanoNFCorpus00400Unknown
synthetic_relevance__nanobeir__NanoNQNanoNQ00400CC BY-SA 3.0
synthetic_relevance__nanobeir__NanoSCIDOCSNanoSCIDOCS00400CC BY 4.0
synthetic_relevance__nanobeir__NanoTouche2020NanoTouche202000392CC BY 4.0
synthetic_relevance__nanocoirNanoApps
NanoCodeFeedbackMT
NanoCodeFeedbackST
NanoCodeSearchNet
NanoCodeSearchNetCCR
NanoCodeTransOceanContest
NanoCodeTransOceanDL
NanoCosQA
NanoStackOverflowQA
NanoSyntheticText2SQL
00800Non-commercial research only
Apache-2.0
Non-commercial research only; source-code licenses vary
CC BY-SA 3.0
synthetic_relevance__natural_questionssentence-transformers/natural-questions
google-research-datasets/natural_questions
4,80000CC BY-SA 3.0
synthetic_relevance__pubmedhotchpotch/pubmed-abstract-ir-simulated-search-queries
slinusc/PubMedAbstractsSubset
4,80000CC0 (metadata); abstract rights vary
synthetic_relevance__trivia_qasentence-transformers/trivia-qa
mandarjoshi/trivia_qa
4,80000Apache-2.0
synthetic_relevance__wikipedia_hard_negatives_englishhotchpotch/wikipedia-english-ir-simulated-search-queries4,80000CC BY-SA 4.0 / GFDL
taskmasterhttps://github.com/google-research-datasets/Taskmaster17,29500CC BY 4.0
temporal_nlitasksource/temporal-nli72,3220100Apache-2.0
tracietasksource/tracie1,1740100Apache-2.0
trivia_qasentence-transformers/trivia-qa
mandarjoshi/trivia_qa
18,77500Apache-2.0
ud_ewtuniversal-dependencies/universal_dependencies11,663100100CC BY-SA 4.0
wikipedia_hard_negatives_englishhotchpotch/wikipedia-english-ir-simulated-search-queries300,00000CC BY-SA 4.0 / GFDL
winobiashttps://github.com/uclanlp/corefBias00100MIT
winograndeallenai/winogrande40,3921000Apache-2.0
winowhytasksource/winowhy00100MIT
wiqaallenai/wiqa29,324100100Apache-2.0

License

Each subset follows its original dataset's license and usage terms. Check the linked original sources before using or redistributing the data. This collection does not change those licenses.

Where noted, code, annotations and source documents have different terms. Detailed source records are in sources.json. An unknown license is not permission to reuse the data.

For Open-Jev, retain its acquisition attribution and third-party notices. In open_jev__customer-control-v1, the generated conversations have a CC0-1.0 declaration, but the upstream question descriptions have no verified license and are not relicensed as CC0. Authored S1MB generalization subsets retain their recorded not separately declared status. See sources.json, SOURCES.md and the retained upstream notices for scope and qualifications.

Training decontamination (2026-10-08)

Removed two complete training cases whose full inputs match active S1MB test cases after normalizing whitespace: contract_nli:default:train:366 (17 decisions) and enron_spam:default:train:18716 (1 decision, configuration laya__enron_spam). ContractNLI also has an upstream train/test target disagreement on nda-1. Test and validation membership, retained rows, candidate order and targets are unchanged. See decontamination-20261008.json for the parent revision, verification and removed IDs. This is a targeted deletion, not a claim of comprehensive semantic decontamination. Historical Arrow/source/export statistics remain in _sources/; current counts are in the active manifests and release_splits.

choice
noul
score
typed-decisions

Linked in READMEs