OpenSpaces

The OpenSpaces dataset is created using VQASynth to synthesize spatialVQA data using images from the first 30K rows of the localized narratives split of the cauldron.

Compared to the related dataset used to train SpaceLLaVA, the OpenSpaces emphasizes greater diversity in the image distribution instead of focusing on warehouse scenes. The following chart shows the distribution of images over tags labeled by CLIP embedding similarity:

The OpenSpaces dataset also includes relatively more of the qualitative distance estimation queries than the quanlitative queries about the relative position of objects in a scene. This histogram shows the distribution of estimated distances between objects in the image scenes

Citation

@article{chen2024spatialvlm,
  title = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
  author = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brian and Driess, Danny and Florence, Pete and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
  journal = {arXiv preprint arXiv:2401.12168},
  year = {2024},
  url = {https://arxiv.org/abs/2401.12168},
}


@misc{VQASynth,
        author = {remyxai},
        title = {VQASynth},
        year = {2024},
        note = {GitHub repository},
        url = {https://github.com/remyxai/VQASynth/tree/main}
      }