MBench Data

This repository contains the evaluation data released with MBench.

• MBench-A: 547 action-conditioned evaluation samples.
• MBench-T: 493 base cases, expanded into 684 condition-level evaluation items.

Structure

Each benchmark follows the directory protocol expected by the MBench evaluator:

MBench-{A|T}/
├── dataset.yaml
├── samples/{subset}/{sample_id}/sample.json
└── models/{model_id}/
    ├── model.yaml
    ├── samples.jsonl
    └── outputs/{subset}/{sample_id}/{condition_id}/video.mp4

The exact evaluation items available for each model are listed in models/{model_id}/samples.jsonl.

In MBench-T, the 493 base cases consist of 293 non-causal samples and 200 causal samples. The causal samples are expanded into 200 camera-motion, 100 occlusion, and 91 dimming conditions, producing 684 condition-level items in total.