Causal World Modeling for Robot Control

LingBot-VA is an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously, introduced in the paper Causal World Modeling for Robot Control.

It focuses on:

• Autoregressive Video-Action World Modeling: Architecturally unifies visual dynamics prediction and action inference within a single interleaved sequence while maintaining their conceptual distinction.
• High-efficiency Execution: A dual-stream mixture-of-transformers (MoT) architecture with Asynchronous Execution and KV Cache.
• Long-Horizon Performance and Generalization: High improvements in sample efficiency, long-horizon success rates, and generalization to novel scenes.

---

Model Sources

• Repository: https://github.com/Robbyant/lingbot-va
• Paper: Causal World Modeling for Robot Control
• Project Page: https://technology.robbyant.com/lingbot-va

---

📦 Model Download

• Pretrained Checkpoints for Post-Training

---

🛠️ Quick Start

Installation

Requirements • Python == 3.10.16 • Pytorch == 2.9.0 • CUDA 12.6

pip install torch==2.9.0 torchvision==0.24.0 torchaudio==2.9.0 --index-url https://download.pytorch.org/whl/cu126
pip install websockets einops diffusers==0.36.0 transformers==5.0.0 accelerate msgpack opencv-python matplotlib ftfy easydict
pip install flash-attn --no-build-isolation

Run Image to Video-Action Generation

We provide a script for image to video-action generation:

NGPU=1 CONFIG_NAME='robotwin_i2av' bash script/run_launch_va_server_sync.sh 

---

📊 Performance

We evaluate our model on both simulation benchmarks and real-world scenarios, achieving state-of-the-art performance.

Simulation Evaluation (Success Rate %)

---

📚 Citation

@article{lingbot-va2026,
  title={Causal World Modeling for Robot Control},
  author={Li, Lin and Zhang, Qihang and Luo, Yiming and Yang, Shuai and Wang, Ruilin and Han, Fei and Yu, Mingrui and Gao, Zelin and Xue, Nan and Zhu, Xing and Shen, Yujun and Xu, Yinghao},
  journal={arXiv preprint arXiv:2601.21998},
  year={2026}
}

🪪 License

This project is released under the Apache License 2.0. See LICENSE file for details.

🧩 Acknowledgments

This work builds upon several excellent open-source projects:

• Wan-Video - Vision transformer backbone
• MoT - Mixture-of-Transformers architecture