ABot World Explorer 4D

ABot World Explorer 4D is a depth-enabled sample of the action-conditioned video data infrastructure described in ABot-World-0. Its source manifest references 20 episodes and 181,561 EXR depth objects; the release preserves their bytes.

Dataset facts

Exact duration and timestamp-level RGB/action/pose/depth alignment are not claimed by this payload-only publication. Twenty episodes must not be confused with twenty depth frames.

Layout

meta/abot-world-explorer-poster.png
metadata.jsonl
data/<prefix>/<sample_id>/video.mp4
data/<prefix>/<sample_id>/annotations.tar
data/<prefix>/<sample_id>/depth/shard-*.tar
LICENSE
README.md

sample_id is an anonymous, dataset-namespaced HMAC identifier. Source keys and OSS locations are not released.

Preview

The Hugging Face Dataset Viewer is backed by metadata.jsonl. It covers all 20 episodes and contains only each anonymous sample_id, a typed Video descriptor with an immutable Hub URI for the existing video.mp4, and an immutable, commit-pinned link to the corresponding annotations.tar. The Dataset Server turns the typed descriptor into the playable HTTPS source used by the paginated Viewer; it does not copy or rewrite the media payload. Depth TAR archives remain ordinary repository payload and are intentionally not decoded by the Viewer.

Data file formats

video.mp4 and annotations.tar

video.mp4 preserves the source MP4 bytes without release-time transcoding. annotations.tar is an uncompressed POSIX USTAR archive containing action.json, caption.json, and the three files under sparse/0/. Treat these five members as one sample-level annotation package and validate their paths and hashes before extracting them.

action.json

action.json is a UTF-8 JSON object containing sequence metadata and a frames array. The released records use the following structure; the available controls and dataset-native numeric values may vary by sequence.

Each frames[] object contains frame_id (string), timestamp (number), a keys mapping from control names to booleans, and four length-3 numeric vectors: delta_translation_cam, delta_translation_cam_smooth, delta_euler_deg, and delta_euler_deg_smooth. Rotation deltas are named in degrees; do not infer the translation or timestamp units, thresholds, or exact cross-modal alignment when this card marks them as unknown. Some source variants add per-frame translation/rotation threshold and release fields. Readers should accept additional fields and must not assume a fixed set of keys.

caption.json

caption.json is a UTF-8 JSON object with perspective, scene_static, and narrative strings plus a dense_temporal array. dense_temporal may be empty; all current audit canaries contain an empty array, so its item schema is not claimed here. Consumers must not assume at least one segment. String values are preserved. The release scan found no complete-token private source path identifier requiring replacement (redacted_units=0), and JSON keys are never rewritten.

COLMAP pose files

The three files under sparse/0/ form one COLMAP text sparse model and must be interpreted together. Lines beginning with # are comments.

• cameras.txt: one camera per line as CAMERA_ID MODEL WIDTH HEIGHT PARAMS[]. The parameter list depends on the camera model.
• images.txt: two lines per image. The first is IMAGE_ID QW QX QY QZ TX TY TZ CAMERA_ID NAME; the second is a repeated list of (X, Y, POINT3D_ID) observations and may be empty. The pose maps world coordinates to camera coordinates using a Hamilton quaternion. COLMAP camera axes are +X right, +Y down, +Z forward, and the camera center is -R^T T.
• points3D.txt: one sparse point per line as POINT3D_ID X Y Z R G B ERROR TRACK[]; each track item is (IMAGE_ID, POINT2D_IDX). A present but empty file is valid and means that the model contains no sparse 3D points.

IDs are not guaranteed to be contiguous. See the official specifications for cameras.txt, images.txt, and points3D.txt.

For all 20 4D episodes, the audited model is pose-only: cameras.txt contains one 1920 x 1080 PINHOLE camera, every images.txt observation line is empty, and points3D.txt has no point records. Image names run from frame_000001.jpg in order, and the image/depth counts match within each episode. This is a COLMAP-compatible camera-pose model, not a sparse point cloud; matching counts alone do not prove timestamp-level cross-modal alignment.

OpenEXR depth files and organization

Each depth/<frame_key>.exr member is an OpenEXR binary image. OpenEXR stores named channels as HALF (16-bit floating point), FLOAT (32-bit floating point), or UINT (32-bit unsigned integer) samples; the header's dataWindow defines the valid pixel region. A file may contain one or more parts and each part has its own header and channels. See the official channel description and file layout.

A stratified audit of the first, middle, and last EXR from each of the 20 episodes found one consistent canary profile:

Six decoded pixel canaries had identical values in B, G, and R, indicating that these files carry the same depth plane in all three channels. Readers may use R only after validating channel equality for the files they consume. PXR24 reduces FLOAT data to 24-bit precision before compression and is therefore lossy relative to the original 32-bit values; see the official compression specification.

The release nevertheless preserves every existing EXR byte: it does not perform additional normalization, clipping, quantization, conversion, or re-encoding. Absolute metric depth in metres with scale_to_m=1.0 is the dataset semantic contract supplied by the authors; it is not encoded as custom EXR header metadata. Whether the measurement is camera-z depth or Euclidean range, and the meaning of zero or another invalid sentinel, remain explicitly unknown.

EXRs are grouped per sample in deterministic, uncompressed data/<prefix>/<sample_id>/depth/shard-*.tar archives. Members are ordered by six-digit frame key (depth/000001.exr, depth/000002.exr, ...), a member is never split, and a shard never spans samples. The published payload does not include a separate frame/member Parquet index: inspect each TAR member list in archive order and use its six-digit frame key. Do not infer timestamp-level alignment or global temporal order from shard names alone.

To inspect the original parts and channels with the official Python bindings:

import OpenEXR
import numpy as np

with OpenEXR.File("frame.exr", separate_channels=True) as exr:
    channels = exr.channels()
    depth_m = channels["R"].pixels
    if not all(np.array_equal(depth_m, channels[name].pixels) for name in "GB"):
        raise ValueError("B/G/R depth channels differ")

Selective download

Use a full commit ID when reproducibility matters. The Preview index provides the anonymous sample_id; the corresponding payload paths are derived from that ID without consulting source keys or OSS locations.

from huggingface_hub import snapshot_download

REPO_ID = "acvlab/ABot-World-Explorer-4D"
REVISION = "<full Hugging Face commit ID>"

sample_id = "<sample_id from metadata.jsonl>"
prefix = sample_id[:2]
snapshot = snapshot_download(
    repo_id=REPO_ID,
    repo_type="dataset",
    revision=REVISION,
    allow_patterns=[
        "LICENSE",
        "README.md",
        "metadata.jsonl",
        f"data/{prefix}/{sample_id}/video.mp4",
        f"data/{prefix}/{sample_id}/annotations.tar",
        f"data/{prefix}/{sample_id}/depth/*.tar",
    ],
)
print(snapshot)

Absolute source timestamps and a frame/member index are not exposed. Stream each selected depth shard and validate exact regular-file member names before reading members; never call unchecked extractall().

Intended use and limitations

Intended for metric-depth-aware world-model, multimodal alignment, action-conditioned video, controllable generation, and agent-learning research. It is not a calibrated sensor benchmark or a guarantee of physical/causal correctness. Pose units, axis conventions, invalid-depth sentinels, and exact cross-modal joins are defined only where the released audit supports them.

Additional data access

For additional data access or customized dataset requirements, please contact the AMAP Data Department directly at phys_ai_data@service.alibaba.com

Citation

@misc{jiang2026abotworld0infiniteinteractiveworld,
      title={ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU},
      author={Fan Jiang and Zhaoxu Sun and Mengchao Wang and Ziyu Zhu and Chiyu Wang and Yunpeng Zhang and Wenlin Liu and Yun Wang and Xue Zheng and Rui Sun and Junfeng Ni and Hongyu Pan and Zhongxu Sun and Fei Yu and Zengye Ge and Mengmeng Du and Nianfei Fan and Mingchao Sun and Yu Liu and Yongchang and Yanqing Zhu and Jiahang Wang and Ning Ying and Yuze Xuan and Di Yang and Zhicheng Liu and Zhe Gao and Tingbing Xu and Jiacheng Sui and Wenjin Yang and Junnan Lai and Shufeng Liu and Yuan Liu and Zheng Zhou and Yingliang Peng and Dawei Cao and Kaifeng Sheng and Yuxiang Cai and Fei Lu and Mu Xu and Ning Guo},
      year={2026},
      eprint={2607.19191},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.19191},
}