Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability

https://github.com/rerun-io/hf-example-vista/assets/9785832/0b9a01ca-90a2-4b36-98fc-a7a7b378fd54

Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, Hongyang Li

This is a demo of the Vista model, a driving world model that can be used to simulate a variety of driving scenarios. This demo uses Rerun's custom gradio component to livestream the model's output and show intermediate results.

📜technical report, 🎬video demos, 🤗model weights

Please refer to the original repository for the original code base and README.

You can try the example on Rerun's HuggingFace space here.

Run the example locally

To run this example locally use the following command (you need a GPU with at least 20GB of memory, tested with an RTX 4090):

pixi run example

You can specify the first image, the number of generated segments, and the number of diffusion steps per segment:

pixi run example --img-path "example_images/streetview.png" --num-segments 10 --num-steps 100

To see other all options, use the following:

pixi run example --help