README
MUDDLE eval bundle - MARKDOWN modality
Each document is rendered once to markdown and referenced by every cell that uses it. Same cell ids, same seeded distractor selection, and the same source-first ordering as the PDF and page-image bundles; only the input rendering differs.
This is the modality the distractor sweep reported in the paper is run in, because a source document plus its distractors exceeds current image and PDF input limits.
Related
Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).
| Piece | Where |
|---|---|
| Code | https://github.com/luoojason/muddle |
| Questions + hard-negative metadata | https://huggingface.co/datasets/luoojason/mmlongbench-text-only |
| Eval cells, PDF modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle |
| Eval cells, markdown modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md |
| Eval cells, page-image modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img |
| Paper | https://openreview.net/forum?id=9YS7tSPM9c |
Contributors
20 commits
Similar repos
README
MUDDLE eval bundle - MARKDOWN modality
Each document is rendered once to markdown and referenced by every cell that uses it. Same cell ids, same seeded distractor selection, and the same source-first ordering as the PDF and page-image bundles; only the input rendering differs.
This is the modality the distractor sweep reported in the paper is run in, because a source document plus its distractors exceeds current image and PDF input limits.
Related
Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).
| Piece | Where |
|---|---|
| Code | https://github.com/luoojason/muddle |
| Questions + hard-negative metadata | https://huggingface.co/datasets/luoojason/mmlongbench-text-only |
| Eval cells, PDF modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle |
| Eval cells, markdown modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md |
| Eval cells, page-image modality | https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img |
| Paper | https://openreview.net/forum?id=9YS7tSPM9c |
Similar repos
Linked in READMEs
Contributors
20 commits