README
ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
Dataset Description
We introduce ViewSpatial-Bench, a comprehensive benchmark with over 5,700 question-answer pairs across 1,000+ 3D scenes from ScanNet and MS-COCO validation sets. This benchmark evaluates VLMs' spatial localization capabilities from multiple perspectives, specifically testing both egocentric (camera) and allocentric (human subject) viewpoints across five distinct task types.
ViewSpatial-Bench addresses a critical gap: while VLMs excel at spatial reasoning from their own perspective, they struggle with perspective-taking—adopting another entity's spatial frame of reference—which is essential for embodied interaction and multi-agent collaboration. The figure below shows the construction pipeline and example demonstrations of our benchmark.
The dataset contains the following fields:
| Field Name | Description |
|---|---|
question_type | Type of spatial reasoning task, includes 5 distinct categories for evaluating different spatial capabilities |
image_path | Path to the source image, includes data from two sources: scannetv2_val (ScanNet validation set) and val2017 (MS-COCO validation set) |
question | The spatial reasoning question posed to the model |
answer | The correct answer to the question |
choices | Multiple choice options available for the question |
- Language(s) (NLP): en
- License: apache-2.0
Uses
With HuggingFace datasets library.
from datasets import load_dataset
ds = load_dataset("lidingm/ViewSpatial-Bench")
Benchmark
We provide benchmark results for various models on our benchmark. More model evaluations will be added.
| Model | Camera-based Tasks | Person-based Tasks | Overall | |||||
|---|---|---|---|---|---|---|---|---|
| Rel. Dir. | Obj. Ori. | Avg. | Obj. Ori. | Rel. Dir. | Sce. Sim. | Avg. | ||
| Proprietary Models | ||||||||
| GPT-4o | 41.46 | 19.58 | 33.57 | 42.97 | 40.86 | 26.79 | 36.29 | 34.98 |
| Gemini-2.0-Flash | 45.29 | 12.95 | 33.66 | 41.16 | 32.78 | 21.90 | 31.53 | 32.56 |
| GPT-5-mini | 56.97 | 27.41 | 46.34 | 43.98 | 49.29 | 26.06 | 38.77 | 42.44 |
| Gemini-2.5-Flash | 52.62 | 23.09 | 42.00 | 42.97 | 42.16 | 20.27 | 34.22 | 37.99 |
| Gemini-2.5-Pro | 58.71 | 32.73 | 49.37 | 48.59 | 45.84 | 25.79 | 39.24 | 44.15 |
| Gemini-3.0-Flash | 62.94 | 35.54 | 53.08 | 44.88 | 60.69 | 26.24 | 42.40 | 47.58 |
| GLM-4.6v | 56.35 | 36.35 | 49.16 | 48.90 | 47.39 | 23.44 | 38.91 | 43.87 |
| Doubao-Seed-1.8 | 62.10 | 45.28 | 56.05 | 44.98 | 62.47 | 33.67 | 45.74 | 50.74 |
| Doubao-Seed-2.0 | 65.60 | 44.78 | 58.11 | 47.19 | 72.09 | 33.57 | 49.20 | 53.52 |
| Open-Source General Models | ||||||||
| InternVL2.5 (2B) | 38.52 | 22.59 | 32.79 | 47.09 | 40.02 | 25.70 | 37.04 | 34.98 |
| Qwen3-VL (4B) | 46.98 | 28.01 | 40.16 | 45.68 | 29.22 | 17.74 | 30.48 | 35.17 |
| Qwen2.5-VL (7B) | 46.64 | 29.72 | 40.56 | 37.05 | 35.04 | 28.78 | 33.37 | 36.85 |
| LLaVA-NeXT-Video (7B) | 26.34 | 19.28 | 23.80 | 44.68 | 38.60 | 29.05 | 37.07 | 30.64 |
| LLaVA-OneVision (7B) | 29.84 | 26.10 | 28.49 | 22.39 | 31.00 | 26.88 | 26.54 | 27.49 |
| InternVL2.5 (8B) | 49.41 | 41.27 | 46.48 | 46.79 | 42.04 | 32.85 | 40.20 | 43.24 |
| Qwen3-VL (8B) | 54.60 | 30.32 | 45.87 | 45.28 | 35.75 | 26.79 | 35.61 | 40.58 |
| Llama-3.2-Vision (11B) | 25.27 | 20.98 | 23.73 | 51.20 | 32.19 | 18.82 | 33.61 | 28.82 |
| InternVL3 (14B) | 54.65 | 33.63 | 47.09 | 33.43 | 37.05 | 31.86 | 33.88 | 40.28 |
| Kimi-VL-Instruct (16B) | 26.85 | 22.09 | 25.14 | 63.05 | 43.94 | 20.27 | 41.52 | 33.58 |
| Qwen2.5-VL (32B) | 39.03 | 29.92 | 35.75 | 36.45 | 34.68 | 21.09 | 30.18 | 32.88 |
| Qwen2.5-VL (72B) | 50.65 | 26.71 | 42.04 | 42.17 | 42.76 | 24.80 | 35.82 | 38.83 |
| Qwen3-VL-Thinking (235B) | 59.73 | 36.95 | 51.54 | 43.67 | 48.93 | 31.67 | 40.67 | 45.94 |
| Qwen3.5-Plus (397B) | 62.21 | 38.65 | 53.74 | 50.20 | 68.17 | 38.37 | 50.90 | 52.28 |
| Multi-View Spatial Fine-Tuning | ||||||||
| Qwen2.5-VL (3B) | 43.43 | 33.33 | 39.80 | 39.16 | 28.62 | 28.51 | 32.14 | 35.85 |
| +SFT | 83.59 | 87.65 | 85.05 | 90.16 | 71.14 | 75.75 | 79.31 | 82.09 |
| Improvement over backbone | +40.16 | +54.32 | +45.25 | +51.00 | +42.52 | +47.24 | +47.17 | +46.24 |
| Random Baseline | 25.16 | 26.10 | 25.50 | 24.60 | 31.12 | 26.33 | 27.12 | 26.33 |
Contributors
27 commits
README
ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
Dataset Description
We introduce ViewSpatial-Bench, a comprehensive benchmark with over 5,700 question-answer pairs across 1,000+ 3D scenes from ScanNet and MS-COCO validation sets. This benchmark evaluates VLMs' spatial localization capabilities from multiple perspectives, specifically testing both egocentric (camera) and allocentric (human subject) viewpoints across five distinct task types.
ViewSpatial-Bench addresses a critical gap: while VLMs excel at spatial reasoning from their own perspective, they struggle with perspective-taking—adopting another entity's spatial frame of reference—which is essential for embodied interaction and multi-agent collaboration. The figure below shows the construction pipeline and example demonstrations of our benchmark.
The dataset contains the following fields:
| Field Name | Description |
|---|---|
question_type | Type of spatial reasoning task, includes 5 distinct categories for evaluating different spatial capabilities |
image_path | Path to the source image, includes data from two sources: scannetv2_val (ScanNet validation set) and val2017 (MS-COCO validation set) |
question | The spatial reasoning question posed to the model |
answer | The correct answer to the question |
choices | Multiple choice options available for the question |
- Language(s) (NLP): en
- License: apache-2.0
Uses
With HuggingFace datasets library.
from datasets import load_dataset
ds = load_dataset("lidingm/ViewSpatial-Bench")
Benchmark
We provide benchmark results for various models on our benchmark. More model evaluations will be added.
| Model | Camera-based Tasks | Person-based Tasks | Overall | |||||
|---|---|---|---|---|---|---|---|---|
| Rel. Dir. | Obj. Ori. | Avg. | Obj. Ori. | Rel. Dir. | Sce. Sim. | Avg. | ||
| Proprietary Models | ||||||||
| GPT-4o | 41.46 | 19.58 | 33.57 | 42.97 | 40.86 | 26.79 | 36.29 | 34.98 |
| Gemini-2.0-Flash | 45.29 | 12.95 | 33.66 | 41.16 | 32.78 | 21.90 | 31.53 | 32.56 |
| GPT-5-mini | 56.97 | 27.41 | 46.34 | 43.98 | 49.29 | 26.06 | 38.77 | 42.44 |
| Gemini-2.5-Flash | 52.62 | 23.09 | 42.00 | 42.97 | 42.16 | 20.27 | 34.22 | 37.99 |
| Gemini-2.5-Pro | 58.71 | 32.73 | 49.37 | 48.59 | 45.84 | 25.79 | 39.24 | 44.15 |
| Gemini-3.0-Flash | 62.94 | 35.54 | 53.08 | 44.88 | 60.69 | 26.24 | 42.40 | 47.58 |
| GLM-4.6v | 56.35 | 36.35 | 49.16 | 48.90 | 47.39 | 23.44 | 38.91 | 43.87 |
| Doubao-Seed-1.8 | 62.10 | 45.28 | 56.05 | 44.98 | 62.47 | 33.67 | 45.74 | 50.74 |
| Doubao-Seed-2.0 | 65.60 | 44.78 | 58.11 | 47.19 | 72.09 | 33.57 | 49.20 | 53.52 |
| Open-Source General Models | ||||||||
| InternVL2.5 (2B) | 38.52 | 22.59 | 32.79 | 47.09 | 40.02 | 25.70 | 37.04 | 34.98 |
| Qwen3-VL (4B) | 46.98 | 28.01 | 40.16 | 45.68 | 29.22 | 17.74 | 30.48 | 35.17 |
| Qwen2.5-VL (7B) | 46.64 | 29.72 | 40.56 | 37.05 | 35.04 | 28.78 | 33.37 | 36.85 |
| LLaVA-NeXT-Video (7B) | 26.34 | 19.28 | 23.80 | 44.68 | 38.60 | 29.05 | 37.07 | 30.64 |
| LLaVA-OneVision (7B) | 29.84 | 26.10 | 28.49 | 22.39 | 31.00 | 26.88 | 26.54 | 27.49 |
| InternVL2.5 (8B) | 49.41 | 41.27 | 46.48 | 46.79 | 42.04 | 32.85 | 40.20 | 43.24 |
| Qwen3-VL (8B) | 54.60 | 30.32 | 45.87 | 45.28 | 35.75 | 26.79 | 35.61 | 40.58 |
| Llama-3.2-Vision (11B) | 25.27 | 20.98 | 23.73 | 51.20 | 32.19 | 18.82 | 33.61 | 28.82 |
| InternVL3 (14B) | 54.65 | 33.63 | 47.09 | 33.43 | 37.05 | 31.86 | 33.88 | 40.28 |
| Kimi-VL-Instruct (16B) | 26.85 | 22.09 | 25.14 | 63.05 | 43.94 | 20.27 | 41.52 | 33.58 |
| Qwen2.5-VL (32B) | 39.03 | 29.92 | 35.75 | 36.45 | 34.68 | 21.09 | 30.18 | 32.88 |
| Qwen2.5-VL (72B) | 50.65 | 26.71 | 42.04 | 42.17 | 42.76 | 24.80 | 35.82 | 38.83 |
| Qwen3-VL-Thinking (235B) | 59.73 | 36.95 | 51.54 | 43.67 | 48.93 | 31.67 | 40.67 | 45.94 |
| Qwen3.5-Plus (397B) | 62.21 | 38.65 | 53.74 | 50.20 | 68.17 | 38.37 | 50.90 | 52.28 |
| Multi-View Spatial Fine-Tuning | ||||||||
| Qwen2.5-VL (3B) | 43.43 | 33.33 | 39.80 | 39.16 | 28.62 | 28.51 | 32.14 | 35.85 |
| +SFT | 83.59 | 87.65 | 85.05 | 90.16 | 71.14 | 75.75 | 79.31 | 82.09 |
| Improvement over backbone | +40.16 | +54.32 | +45.25 | +51.00 | +42.52 | +47.24 | +47.17 | +46.24 |
| Random Baseline | 25.16 | 26.10 | 25.50 | 24.60 | 31.12 | 26.33 | 27.12 | 26.33 |
Similar repos
Linked in READMEs
Contributors
27 commits