Data for RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation.
RotBench is a benchmark for evaluating whether multimodal large language models (MLLMs) can identify image orientation. It contains 350 manually filtered images. The dataset includes two subsets:
All images were drawn from the Spatial-MM dataset and passed a two-stage human verification process to ensure rotations are distinguishable.
from datasets import load_dataset
dataset = load_dataset("tianyin/RotBench")
data = dataset['large'] # or dataset['small']
for i, sample in enumerate(data):
image = sample['image'] # PIL Image object
image_name = sample['image_name']
If you find our data useful in your research, please cite the following paper:
@misc{niu2025rotbenchevaluatingmultimodallarge,
title={RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation},
author={Tianyi Niu and Jaemin Cho and Elias Stengel-Eskin and Mohit Bansal},
year={2025},
eprint={2508.13968},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2508.13968},
}
Data for RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation.
RotBench is a benchmark for evaluating whether multimodal large language models (MLLMs) can identify image orientation. It contains 350 manually filtered images. The dataset includes two subsets:
All images were drawn from the Spatial-MM dataset and passed a two-stage human verification process to ensure rotations are distinguishable.
from datasets import load_dataset
dataset = load_dataset("tianyin/RotBench")
data = dataset['large'] # or dataset['small']
for i, sample in enumerate(data):
image = sample['image'] # PIL Image object
image_name = sample['image_name']
If you find our data useful in your research, please cite the following paper:
@misc{niu2025rotbenchevaluatingmultimodallarge,
title={RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation},
author={Tianyi Niu and Jaemin Cho and Elias Stengel-Eskin and Mohit Bansal},
year={2025},
eprint={2508.13968},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2508.13968},
}