Awesome MLLM Benchmarks – Sample Data

🌐 Interactive Dashboard · 💻 GitHub

This dataset hosts the sample data (images, questions, answers, metadata) used by the Awesome MLLM Benchmarks interactive dashboard. It provides curated preview samples from 130+ multimodal LLM benchmarks across 20+ categories.

Overview

Each benchmark folder contains ~30 representative samples with images and a data.json metadata file, enabling quick browsing without downloading the full benchmark datasets.

Categories

Dataset Structure

samples/
├── <BenchmarkName>/
│   ├── <subtask>/
│   │   ├── data.json          # Sample metadata (questions, answers, options, etc.)
│   │   ├── 0_image.jpg        # Sample images
│   │   ├── 1_image.jpg
│   │   └── ...
│   └── <subtask>/
│       └── ...
└── ...

data.json Format

Each data.json is a JSON array of sample entries. The exact fields vary by benchmark, but common fields include:

Some benchmarks include additional fields like category, difficulty, subject, knowledge, etc.

Usage

With the Interactive Dashboard (Recommended)

Visit lchen1019.github.io/awesome-mllm-benchmarks to browse samples interactively with full rendering support (LaTeX, images, multi-view, etc.).

Clone Locally

# Clone the main project
git clone https://github.com/lchen1019/awesome-mllm-benchmarks.git
cd awesome-mllm-benchmarks

# Download sample data from Hugging Face
# Option 1: Using huggingface_hub
python -c "
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id='lchen1019/awesome-mllm-benchmarks-samples',
    repo_type='dataset',
    local_dir='samples'
)
"

# Option 2: Using git
git clone https://huggingface.co/datasets/lchen1019/awesome-mllm-benchmarks-samples samples

# Start the local server
python serve.py 8080

Load with Python

import json
from pathlib import Path

# Load samples for a specific benchmark
benchmark = "MathVista"
subtask = "default"
data = json.loads(Path(f"samples/{benchmark}/{subtask}/data.json").read_text())

for sample in data[:3]:
    print(f"Q: {sample['question'][:100]}...")
    print(f"A: {sample['answer']}")
    print()

Related Resources

• Interactive Dashboard: lchen1019.github.io/awesome-mllm-benchmarks
• GitHub Repository: github.com/lchen1019/awesome-mllm-benchmarks

Citation

@misc{awesome-mllm-benchmarks,
  title        = {Awesome MLLM Benchmarks: An Interactive Explorer for MLLM Benchmarks},
  author       = {Chen, Lin},
  year         = {2026},
  howpublished = {\url{https://github.com/lchen1019/awesome-mllm-benchmarks}},
  note         = {Accessed: 2026}
}

License

This dataset collection is released under the Apache 2.0 License. Individual benchmark samples may be subject to their original licenses. Please refer to each benchmark's source for specific licensing terms.