README
WBench Dataset
A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation
TL;DR — WBench evaluates 20 video world models across 5 dimensions and 22 metrics.
Overview
WBench is a comprehensive multi-turn benchmark for interactive video world model evaluation. It contains 289 multi-turn interaction cases with 1,058 interaction turns for evaluating models across 22 metrics and 5 dimensions:
- Video Quality
- Setting Adherence
- Interaction Adherence
- Consistency
- Physics Compliance
The benchmark covers diverse scenes, styles, subjects, and both first- and third-person perspectives, with four interaction types: navigation, subject action, event editing, and perspective switching.
Usage
Download Dataset
You can download the dataset using the Hugging Face CLI:
pip install huggingface_hub
huggingface-cli download meituan-longcat/WBench --repo-type dataset --local-dir data/ --exclude "splits/*"
Framework Usage
To generate multi-turn videos from a case using the WBench framework (as implemented in the GitHub repository):
from src.models import get_model
# Available: wan, kling, seedance (or register your own)
model = get_model("wan")
# Generate multi-turn video from a case
# case_dict should be loaded from the 'cases/' directory
result = model.generate_multi_turn(
case=case_dict,
output_path="work_dirs/wan/videos/case_1_combined.mp4",
data_root="data/",
)
Structure
├── cases/ # 289 case JSONs (prompts + interactions + annotations)
├── images/ # 289 first-frame images
├── masks/ # 188 subject segmentation masks
🏆 Leaderboard
20 Models — Navigation Split (5 Dimensions, sorted by average)
| # | Model | Average | Quality | Setting | Interaction | Consistency | Physical |
|---|---|---|---|---|---|---|---|
| 1 | Kling 3.0 | 79.2 🥇 | 83.0 🥈 | 91.0 🥈 | 70.3 | 82.5 | 69.3 🥉 |
| 2 | LingBot-World | 78.8 🥈 | 81.5 | 72.6 | 79.8 | 88.9 🥇 | 71.2 🥈 |
| 3 | Wan 2.7 | 78.5 🥉 | 82.6 🥉 | 91.4 🥇 | 66.0 | 80.5 | 71.8 🥇 |
| 4 | HY-World 1.5 | 78.4 | 80.2 | 72.2 | 87.5 🥇 | 86.0 | 66.3 |
| 5 | HY-Video 1.5 | 78.2 | 79.7 | 85.6 🥉 | 71.8 | 86.7 🥉 | 67.4 |
Detailed leaderboard results and metric breakdowns are available on the Project Page.
Citation
@article{ying2026wbenchcomprehensivemultiturnbenchmark,
title={WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation},
author={Ying, Kaining and Hu, Hengrui and Ren, Siyu and Li, Jiamu and Chen, Fengjiao and Wang, Ziwen and Cao, Xuezhi and Cai, Xunliang and Ding, Henghui},
journal={arXiv preprint arXiv:2605.25874},
year={2026}
}
Contributors
Similar repos
README
WBench Dataset
A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation
TL;DR — WBench evaluates 20 video world models across 5 dimensions and 22 metrics.
Overview
WBench is a comprehensive multi-turn benchmark for interactive video world model evaluation. It contains 289 multi-turn interaction cases with 1,058 interaction turns for evaluating models across 22 metrics and 5 dimensions:
- Video Quality
- Setting Adherence
- Interaction Adherence
- Consistency
- Physics Compliance
The benchmark covers diverse scenes, styles, subjects, and both first- and third-person perspectives, with four interaction types: navigation, subject action, event editing, and perspective switching.
Usage
Download Dataset
You can download the dataset using the Hugging Face CLI:
pip install huggingface_hub
huggingface-cli download meituan-longcat/WBench --repo-type dataset --local-dir data/ --exclude "splits/*"
Framework Usage
To generate multi-turn videos from a case using the WBench framework (as implemented in the GitHub repository):
from src.models import get_model
# Available: wan, kling, seedance (or register your own)
model = get_model("wan")
# Generate multi-turn video from a case
# case_dict should be loaded from the 'cases/' directory
result = model.generate_multi_turn(
case=case_dict,
output_path="work_dirs/wan/videos/case_1_combined.mp4",
data_root="data/",
)
Structure
├── cases/ # 289 case JSONs (prompts + interactions + annotations)
├── images/ # 289 first-frame images
├── masks/ # 188 subject segmentation masks
🏆 Leaderboard
20 Models — Navigation Split (5 Dimensions, sorted by average)
| # | Model | Average | Quality | Setting | Interaction | Consistency | Physical |
|---|---|---|---|---|---|---|---|
| 1 | Kling 3.0 | 79.2 🥇 | 83.0 🥈 | 91.0 🥈 | 70.3 | 82.5 | 69.3 🥉 |
| 2 | LingBot-World | 78.8 🥈 | 81.5 | 72.6 | 79.8 | 88.9 🥇 | 71.2 🥈 |
| 3 | Wan 2.7 | 78.5 🥉 | 82.6 🥉 | 91.4 🥇 | 66.0 | 80.5 | 71.8 🥇 |
| 4 | HY-World 1.5 | 78.4 | 80.2 | 72.2 | 87.5 🥇 | 86.0 | 66.3 |
| 5 | HY-Video 1.5 | 78.2 | 79.7 | 85.6 🥉 | 71.8 | 86.7 🥉 | 67.4 |
Detailed leaderboard results and metric breakdowns are available on the Project Page.
Citation
@article{ying2026wbenchcomprehensivemultiturnbenchmark,
title={WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation},
author={Ying, Kaining and Hu, Hengrui and Ren, Siyu and Li, Jiamu and Chen, Fengjiao and Wang, Ziwen and Cao, Xuezhi and Cai, Xunliang and Ding, Henghui},
journal={arXiv preprint arXiv:2605.25874},
year={2026}
}