luoojason/muddle-eval-bundle-img

Dataset

MUDDLE eval bundle - IMAGE modality

52 commits

1 linked in READMEs

updated Aug 4, 2026

See the code

README

MUDDLE eval bundle - IMAGE modality

Each page rendered as one 150-DPI PNG. Page images are stored ONCE per distinct document under images/<doc>/pNNNN.png and referenced by each cell question.json (image_docs, in load order: source first, then distractors). 1350 cells, all hard negatives in [10,40] pages.

Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).

Piece	Where
Code	https://github.com/luoojason/muddle
Questions + hard-negative metadata	https://huggingface.co/datasets/luoojason/mmlongbench-text-only
Eval cells, PDF modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle
Eval cells, markdown modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md
Eval cells, page-image modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img
Paper	https://openreview.net/forum?id=9YS7tSPM9c

context-degradation

image

long-context

multi-document

Contributors

luoojason

52 commits

Similar repos

luoojason/muddle-eval-bundle-md

Dataset

MUDDLE eval bundle - MARKDOWN modality

luoojason/muddle-eval-bundle

Dataset

MUDDLE eval bundle - PDF modality

luoojason/mmlongbench-text-only

Dataset

MUDDLE - source questions and hard-negative metadata

luoojason/muddle

MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 CBW) -…

lmms-lab/HallusionBench

Dataset

Large-scale Multi-modality Models Evaluation Suite

lmms-lab/MMMU

Dataset

This is a merged version of [MMMU/MMMU](https://huggingface.co/datasets/MMMU/MMMU) with all subsets…

MUIRBENCH/MUIRBENCH

Dataset

MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding

lmms-lab/DocVQA

Dataset

Large-scale Multi-modality Models Evaluation Suite

Linked in READMEs

luoojason/muddle

MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 CBW) -…

README badge

luoojason/muddle-eval-bundle-img

Dataset

MUDDLE eval bundle - IMAGE modality

52 commits

1 linked in READMEs

updated Aug 4, 2026

See the code

README

MUDDLE eval bundle - IMAGE modality

Part of MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects (COLM 2026 Workshop on Context Beyond the Window).

Piece	Where
Code	https://github.com/luoojason/muddle
Questions + hard-negative metadata	https://huggingface.co/datasets/luoojason/mmlongbench-text-only
Eval cells, PDF modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle
Eval cells, markdown modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle-md
Eval cells, page-image modality	https://huggingface.co/datasets/luoojason/muddle-eval-bundle-img
Paper	https://openreview.net/forum?id=9YS7tSPM9c