ViewBench

ViewBench is a dataset for camera-conditioned long-horizon video world models. It is designed to evaluate view consistency and loop closure: when a camera returns to a previously observed viewpoint, the generated observation should preserve stable scene structure and appearance.

This dataset accompanies the paper Consistent Video World Model With Geometry-Aware Rotary Position Embedding.

Dataset mirrors:

• ModelScope: https://modelscope.cn/datasets/JEdward/viewbench-dataset
• Hugging Face: https://huggingface.co/datasets/JEdward/viewbench-dataset

Highlights

• Complete yaw, pitch, and roll coverage for controlled camera motion.
• Round-trip loop-closure trajectories where the camera returns to previously visited viewpoints.
• 10 photorealistic UE5 environments spanning indoor, outdoor, urban, industrial, historical, and suburban scenes.
• Per-frame SE(3) camera-to-world poses and depth-based geometric overlap annotations.

Dataset Contents

This v1 release contains the public ViewBench training split described in the paper: 1,059 UE5-rendered video sequences, about 500k frames at 30 fps, across 10 photorealistic environments.

The release is organized into two trajectory groups:

• pure_rotation: stationary-camera rotate-away-rotate-back trajectories for loop closure.
• rotation_translation: compact exploration trajectories with both rotation and translation.

The original internal directories were STAGE1 and STAGE3. In this public release they are renamed to pure_rotation and rotation_translation.

The paper's held-out evaluation set is separately collected and is not included in this training release unless explicitly provided in a later update.

Files

The dataset is distributed as tar.zst shards plus manifest.json:

• pure_rotation_0000.tar.zst ... pure_rotation_0011.tar.zst (600 sequences)
• rotation_translation_0000.tar.zst ... rotation_translation_0009.tar.zst (459 sequences)
• manifest.json

manifest.json records the shard membership, sequence IDs, original-to-public stage mapping, and archive contents.

Each archive extracts into:

ViewBench4Training/
  pure_rotation/
    frames/{sequence_id}/
    jsons/{sequence_id}.json
    metadata/{sequence_id}/
  rotation_translation/
    frames/{sequence_id}/
    jsons/{sequence_id}.json
    metadata/{sequence_id}/

Data Format

Each sequence contains:

• EXR frames with RGB/depth information.
• Per-frame camera poses in jsons/{sequence_id}.json.
• Raw metadata in metadata/{sequence_id}/tickStatus.jsonl where available.
• Depth-based frame overlap labels in metadata/{sequence_id}/overlap.json where available.

Camera convention:

• UE left-handed coordinates: X=forward, Y=right, Z=up.
• Position is measured in centimeters.
• Rotation is [pitch, roll, yaw] in degrees.
• c2w is a 4x4 camera-to-world SE(3) matrix.
• Rotation convention: R = Rz(yaw) * Ry(pitch) * Rx(roll).

Usage

Download from ModelScope:

modelscope download --dataset JEdward/viewbench-dataset --local_dir ViewBench-v1

After all shards are downloaded, extract them into a single directory:

mkdir -p ViewBench4Training
for shard in ViewBench-v1/pure_rotation_*.tar.zst ViewBench-v1/rotation_translation_*.tar.zst; do
  tar --zstd -xf "$shard" -C ViewBench4Training
done

Repeat extraction for all shards listed in manifest.json.

License

This dataset is released for non-commercial research use under CC BY-NC 4.0-style terms.

Users may use, copy, and redistribute the dataset for academic and non-commercial research purposes, provided that they give appropriate attribution and cite the accompanying paper.

Commercial use, resale, or redistribution as part of a commercial dataset or product is not permitted without prior written permission from the authors.

The dataset contains UE5-rendered outputs from third-party scene assets. The release does not include raw UE assets, source asset files, or engine content. Users are responsible for ensuring that their downstream use complies with applicable third-party asset terms.

Citation

@inproceedings{
xiang2026consistent,
title={Consistent Video World Model With Geometry-Aware Rotary Position Embedding},
author={Chendong Xiang and Jiajun Liu and Jintao Zhang and Xiao Yang and Zhengwei Fang and Shizun Wang and Zijun Wang and Yingtian Zou and Hang Su and Jun Zhu},
booktitle={ICLR 2026 the 2nd Workshop on World Models: Understanding, Modelling and Scaling},
year={2026},
url={https://openreview.net/forum?id=eXgmwOOvlR}
}