Dataset Card for OpenDocVQA

This is a training and evaluation QA data file for VDocRAG, a new RAG framework that can directly understand diverse real-world documents purely from visual features.

Dataset Description

OpenDocVQA is the first unified collection of open-domain document visual question answering datasets, encompassing diverse document types and formats.

Supported Tasks and Leaderboards

Given a large collection of document images and a question, the goal of the OpenDocVQA task is to output an answer by finding the relevant images. We decompose the task into two stages.

• Visual document retrieval: Given a question and a corpus of document images, the model retrieves the relevant images from which to derive the answer.
• DocumentVQA: The model takes a question and the retrieved images as input to generate an answer.

OpenDocVQA covers multiple open-domain DocumentVQA datasets with diverse document types. To reflect real-world scenarios, we evaluate models with both single-pool and all-pool settings. In the single-pool setting, retrieval is performed from a specific pool of documents provided by each original dataset. The all-pool setting requires retrieving from the entire candidate pool, which includes documents from a wide range of domains.

Languages

English

Dataset Structure

Data Instances

{
  'query_id': "chartqa-test_6",
  'query': 'Instruct: Given a user query, retrieve a chart image that answers the query.\nQuery: How many more Hispanics younger than 18 tend to be Mexican than Spanish?',
  'answers': ['65'],
  'relevant_doc_ids': ['chartqa/9001.png'],
  'dataset_names': ['chartqa']
}

Data Fields

An example of a sample looks as follows:

{
  'query_id': Query ID,
  'query': Query,
  'answers': List of answers,
  'relevant_doc_ids': List of relevant document IDs,
  'dataset_names': List of source dataset names
}

Stats about the datasets in The OpenDocVQA

Additional Information

License

Each of the publicly available sub-datasets present in the OpenDocVQA is governed by specific licensing conditions. Therefore, when making use of them, you must take into consideration each of the licenses governing each dataset.

The QA pairs of MHDocVQA, VisualMRC, and SlideVQA datasets in this repo are released under the NTT License.

Citation

@inproceedings{tanaka2025vdocrag,
  author    = {Ryota Tanaka and
               Taichi Iki and
               Taku Hasegawa and
               Kyosuke Nishida and
               Kuniko Saito and
               Jun Suzuki},
  title     = {VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents},
  booktitle = {CVPR},
  year      = {2025}
}