MUDDLE - source questions and hard-negative metadata
0
26 commits
1 linked in READMEs
updated Aug 4, 2026
README
MUDDLE - source questions and hard-negative metadata
The 270 human-annotated questions behind MUDDLE, derived from MMLongBench-Doc, plus the
curated hard-negative pool (hard_negatives.json) and the source / hard-negative PDFs.
Each question is tied to a single source document and is instantiated in five context conditions in the eval bundles: source alone, source + 2 or 4 topically similar hard negatives, and source + 2 or 4 length-matched random distractors.
Related
Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).
| Piece | Where |
|---|---|
| Code | https://github.com/luoojason/muddle |
| Questions + hard-negative metadata | https://huggingface.co/datasets/luoojason/mmlongbench-text-only |
| Eval cells, PDF modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle |
| Eval cells, markdown modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md |
| Eval cells, page-image modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img |
| Paper | https://openreview.net/forum?id=9YS7tSPM9c |
Contributors
26 commits
Similar repos
MUDDLE - source questions and hard-negative metadata
0
26 commits
1 linked in READMEs
updated Aug 4, 2026
README
MUDDLE - source questions and hard-negative metadata
The 270 human-annotated questions behind MUDDLE, derived from MMLongBench-Doc, plus the
curated hard-negative pool (hard_negatives.json) and the source / hard-negative PDFs.
Each question is tied to a single source document and is instantiated in five context conditions in the eval bundles: source alone, source + 2 or 4 topically similar hard negatives, and source + 2 or 4 length-matched random distractors.
Related
Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).
| Piece | Where |
|---|---|
| Code | https://github.com/luoojason/muddle |
| Questions + hard-negative metadata | https://huggingface.co/datasets/luoojason/mmlongbench-text-only |
| Eval cells, PDF modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle |
| Eval cells, markdown modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md |
| Eval cells, page-image modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img |
| Paper | https://openreview.net/forum?id=9YS7tSPM9c |
Similar repos
Linked in READMEs
Contributors
26 commits