Dataset Card for Spatial Mosaic VQA

This repository contains Spatial Mosaic VQA annotations for evaluating and training vision-language models on multi-view spatial reasoning tasks.

Dataset Details

Dataset Description

Spatial Mosaic VQA is a visual question answering dataset focused on spatial understanding across multiple image frames. The dataset includes VQA tasks covering object existence, object counting, relative spatial relations, object localization, distance estimation, object size, object attribute, and best-view selection.

The released archive contains annotation JSON files only. It does not redistribute original ScanNet++, Waymo, image. Some entries reference ScanNet++ and Waymo scene identifiers, and users must obtain any required source data from the official dataset providers under their respective terms.

• Curated by: Anonymous submission authors
• Language(s) (NLP): English
• License: CC BY-NC 4.0

Uses

Direct Use

This dataset is intended for non-commercial research on visual question answering, spatial reasoning, multiview understanding, and evaluation of vision-language models. It can be used to train or evaluate models that answer questions from multiple frames and reason about object visibility, occlusion, position, count, distance, and size.

Out-of-Scope Use

This dataset should not be used for commercial purposes under the released license. It should not be used to redistribute or reconstruct restricted source datasets. Users should not treat the annotations as a substitute for the original ScanNet++ or Waymo datasets, and must comply with the original terms of those datasets when using referenced source data.

Dataset Structure

The archive contains four JSON files:

spatial_mosaic_vqa/
├── train/
│   ├── spatialmosaic_indoor_train.json
│   └── spatialmosaic_outdoor_train.json
└── test/
    ├── spatialmosaic_indoor_test.json
    └── spatialmosaic_outdoor_test.json

Dataset sizes:

Total entries: 403,307.

Training files use a conversation-style format with fields such as id, data_source, scene_name, question_type, frames, and conversations.

Test files use an evaluation-style multiple-choice format with fields such as dataset, scene_name, question_type, frames, question, options, mc_answer, and task-specific metadata such as visibility, occlusion, overlap, and bounding-box information when available.

Archive checksum:

Dataset Creation

Curation Rationale

Spatial Mosaic VQA was created to support research on multi-view spatial reasoning in vision-language models. The dataset emphasizes questions that require models to compare visual evidence across multiple frames rather than relying on a single image.

Source Data

The annotations reference indoor scenes from ScanNet++ and outdoor scenes from Waymo. The released files contain only VQA annotations and metadata references. Original source data must be obtained separately from the official dataset providers.

Data Collection and Processing

Questions were created for multi-view visual reasoning tasks using scene-level and frame-level metadata. The dataset includes training annotations in a dialogue format and test annotations in a multiple-choice format for evaluation.

Who are the source data producers?

The source visual data referenced by the annotations originates from ScanNet++ and Waymo. This repository does not redistribute the original source data.

Personal and Sensitive Information

The released archive was checked for direct personal identifiers, local filesystem paths, account names, emails, URLs, API keys, and related metadata. The tar archive was also anonymized so that local owner/group metadata is not included. The repository does not intentionally contain personal or sensitive information.

Bias, Risks, and Limitations

The dataset inherits limitations from the source datasets and from the annotation-generation process. Scene coverage, object categories, camera viewpoints, and environmental conditions may be unevenly distributed. Model performance on this dataset may not generalize to all real-world spatial reasoning settings.

The annotations reference source dataset scene identifiers. Users are responsible for ensuring that their use of any corresponding original data complies with the ScanNet++ and Waymo terms of use.

Recommendations

Users should evaluate models across indoor and outdoor splits separately and report any preprocessing assumptions. Users should also verify compliance with the original dataset licenses before combining these annotations with source images.

Dataset Card Contact

Anonymous submission authors.