Object Permanence

The training corpus of WROP (World Reasoning with Object Permanence): 1.5M Blender-rendered video-continuation samples across 150 hand-designed cognitive tasks, one tar per task.

Abstract

Object permanence is the hallmark of human cognitive priors. Recent studies show that video models, also often referred as world models, have also shown emerged reasoning abilities, making them ideal candidates for building human-like multimodal intelligence. Do video models have emerged object permanence in them? If not, could we train them with a core-cognition inspired dataset? We introduce WROP (World Reasoning with Object Permanence), a data infrastructure of 150 hand-designed cognitive science inspired tasks, divided into six cognitive categories. We build Blender generators that randomize speed, lighting, camera angle, and other nuisance parameters while preserving each task's cognitive structure, yielding 10,000+ samples per task. We release a 1.5M-sample training corpus and a 300-question exam. On this exam we evaluate 14 video models, 4 continuation, 3 reference-to-video, 7 edit, and PWM-WROP, our 16B world model. In a blind pairwise Elo study, PWM-WROP ranks third overall, behind a statistical tie between two commercial reference-to-video systems, and first among continuation models. We release the data, exam, model answers, scores, weights, and PWM, our training stack, a native-PyTorch implementation of the model on AWS Trainium2.

At a glance

Layout

.
├── LICENSE
└── train/
    ├── checksums.txt                  # per archive: sha256, file name, member count (50,000 each)
    ├── accordion_fold_screen_task.tar
    ├── ball_behind_box_stack_task.tar
    └── …                              # 150 archives, one per task

Each archive keeps the original directory layout. train/<task>_task.tar extracts to:

train/
└── shard1/                            # render partition: shard1 = G01–G30 … shard5 = G121–G150
    └── cabinet_task/
        ├── cabinet_0000/
        │   ├── input_video.mp4        # 60 frames, 1280 × 720, 24 fps — the context
        │   ├── target_video.mp4       # 60 frames, same shape — the continuation to predict
        │   ├── prompt.txt             # scene description, prefixed "Continue this scene as a short video."
        │   ├── trajectory.npz         # ground-truth body poses for all 120 source frames
        │   └── metadata.json          # parameters, provenance, video_split
        ├── cabinet_0001/ … cabinet_9999/
        └── …

License

CC BY-NC 4.0 — attribution: Hokin Deng. Non-commercial use only. For commercial licensing contact hokinxqdeng@gmail.com.

Citation

Training Object Permanence in World Models (arXiv link to follow).

@article{zhang2026training,
  title   = {Training Object Permanence in World Models},
  author  = {Zhang, Haotian and others},
  year    = {2026},
  url     = {https://object-permanence.world}
}