ViMDoc: Visually-rich Long Multi-Document Retrieval Benchmark

ViMDoc is a benchmark for evaluating visual document retrieval in both multi-document and long document settings, proposed in our paper Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy (ACL 2026 Findings).

🚀 Quick Start

(1) Load Queries

from datasets import load_dataset
dataset = load_dataset("kaistdata/ViMDoc", split="ViMDoc")

(2) Download Document Pages (Images)

# Option 1 — wget
wget https://huggingface.co/datasets/kaistdata/ViMDoc/resolve/main/ViMDoc_pages.tar.gz
tar -xzf ViMDoc_pages.tar.gz

# Option 2 — HuggingFace CLI
huggingface-cli download kaistdata/ViMDoc ViMDoc_pages.tar.gz \
    --repo-type dataset \
    --local-dir ./
tar -xzf ViMDoc_pages.tar.gz

Data Structure

Each query follows this format:

{
    "id": "<query_id>",
    "query": "<query_text>",
    "doc_ids": ["<document_id>"]
}

Statistics

Citation

If you use ViMDoc in your research, please cite our paper:

@article{kim2025hybrid,
  title={Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy},
  author={Kim, Juyeon and Lee, Geon and Choi, Dongwon and Kim, Taeuk and Shin, Kijung},
  journal={arXiv preprint arXiv:2510.22215},
  year={2025}
}