README
JoyAI-Image-OpenSpatial
Spatial understanding dataset built on OpenSpatial, used in JoyAI-Image.
The full dataset contains about ~6M multi-turn visual-spatial QA samples across 7 open-source datasets and web data. The open-source datasets contain ARKitScenes, ScanNet, ScanNet++, HyperSim, Matterport3D, WildRGB-D, and Ego-Exo4D. Tasks cover a wide range of spatial understanding capabilities including 3D object grounding, depth ordering, spatial relation reasoning, distance estimation, and more.
We release the dataset as two subsets:
| Config | Samples | Source | Location |
|---|---|---|---|
default | ~2.34M | 7 open-source scene datasets (ARKitScenes, ScanNet, ScanNet++, HyperSim, Matterport3D, WildRGB-D, Ego-Exo4D) | data/*.parquet |
web | ~3.63M | Web images with 3D lifting annotations for spatial reasoning | web_data/*.parquet |
Quick Start
from datasets import load_dataset
# Default config: 2.3M samples from open-source scene datasets
ds = load_dataset("jdopensource/JoyAI-Image-OpenSpatial", split="train", streaming=True)
for sample in ds:
print(sample["conversations"])
break
# Web (3D lifting) config: 3.6M samples from web imagery
ds_web = load_dataset("jdopensource/JoyAI-Image-OpenSpatial", "web", split="train", streaming=True)
for sample in ds_web:
print(sample["conversations"])
break
Data Format
Both configs share the same schema. Each parquet file contains the following columns:
| Column | Type | Description |
|---|---|---|
conversations | list[{from, value}] | Multi-turn conversation pairs (human / gpt). The human turn provides camera parameters and a spatial reasoning question; the gpt turn provides structured spatial annotations (e.g., 3D bounding boxes, depth ordering, spatial relations). |
id | string | Unique sample identifier |
data_source | string | Source dataset (e.g., arkitscenes, scannet, scannetpp, hypersim, matterport3d, wildrgbd, Ego-Exo4D, webdata) |
images | list[{bytes, path}] | Embedded image data (PNG bytes) |
type | string | Data type label |
meta_info | string | JSON string with image dimensions (width, height, resized_width, resized_height) |
Contributors
500 commits
Similar repos
README
JoyAI-Image-OpenSpatial
Spatial understanding dataset built on OpenSpatial, used in JoyAI-Image.
The full dataset contains about ~6M multi-turn visual-spatial QA samples across 7 open-source datasets and web data. The open-source datasets contain ARKitScenes, ScanNet, ScanNet++, HyperSim, Matterport3D, WildRGB-D, and Ego-Exo4D. Tasks cover a wide range of spatial understanding capabilities including 3D object grounding, depth ordering, spatial relation reasoning, distance estimation, and more.
We release the dataset as two subsets:
| Config | Samples | Source | Location |
|---|---|---|---|
default | ~2.34M | 7 open-source scene datasets (ARKitScenes, ScanNet, ScanNet++, HyperSim, Matterport3D, WildRGB-D, Ego-Exo4D) | data/*.parquet |
web | ~3.63M | Web images with 3D lifting annotations for spatial reasoning | web_data/*.parquet |
Quick Start
from datasets import load_dataset
# Default config: 2.3M samples from open-source scene datasets
ds = load_dataset("jdopensource/JoyAI-Image-OpenSpatial", split="train", streaming=True)
for sample in ds:
print(sample["conversations"])
break
# Web (3D lifting) config: 3.6M samples from web imagery
ds_web = load_dataset("jdopensource/JoyAI-Image-OpenSpatial", "web", split="train", streaming=True)
for sample in ds_web:
print(sample["conversations"])
break
Data Format
Both configs share the same schema. Each parquet file contains the following columns:
| Column | Type | Description |
|---|---|---|
conversations | list[{from, value}] | Multi-turn conversation pairs (human / gpt). The human turn provides camera parameters and a spatial reasoning question; the gpt turn provides structured spatial annotations (e.g., 3D bounding boxes, depth ordering, spatial relations). |
id | string | Unique sample identifier |
data_source | string | Source dataset (e.g., arkitscenes, scannet, scannetpp, hypersim, matterport3d, wildrgbd, Ego-Exo4D, webdata) |
images | list[{bytes, path}] | Embedded image data (PNG bytes) |
type | string | Data type label |
meta_info | string | JSON string with image dimensions (width, height, resized_width, resized_height) |
Similar repos
Linked in READMEs
Contributors
500 commits