[!IMPORTANT]

🚀 Cosmos 3 Has Arrived

Cosmos 3 is NVIDIA's next-generation foundation model platform for Physical AI. Compared with Cosmos-Predict1, Cosmos 3 delivers significantly stronger world prediction capabilities, producing more accurate, coherent, and physically grounded future-state predictions across a wide range of environments and embodiments.

Beyond improving prediction quality, Cosmos 3 unifies capabilities that previously required multiple specialized models. A single Cosmos 3 model can reason, predict future world states, transfer across domains and modalities, and generate actions and policies for embodied agents within one unified architecture.

This repository is no longer under active development and will receive only limited maintenance updates. Future model releases, features, documentation, and community support will be focused on Cosmos 3.

👉 Visit the new Cosmos home: https://github.com/NVIDIA/Cosmos

There you will find the latest Cosmos 3 models, technical reports, tutorials, benchmarks, and ecosystem updates.

Thank you for your support of Cosmos-Predict1. We encourage all users to migrate to Cosmos 3 for the latest state-of-the-art Physical AI capabilities.

Product Website | Hugging Face | Paper | Paper Website

Cosmos-Predict1 is a key branch of Cosmos World Foundation Models (WFMs) specialized for future state prediction, often referred to as world models. The tree main branches of Cosmos WFMs are cosmos-predict, cosmos-transfer, and cosmos-reason. We visualize the architecture of Cosmos-Predict1 in the following figure.

Cosmos-Predict1 includes the following:

• Diffusion-based world foundation models for Text2World and Video2World generation, where a user can generate visual simulation based on text prompts and video prompts.
• Autoregressive-based world foundation models for Video2World generation, where a user can generate visual simulation based on video prompts and optional text prompts.
• Image and video tokenizers for tokenizing videos into continuous tokens (latent vectors) and discrete tokens (integers) efficiently and effectively.
• Post-training scripts for helping Physical AI builders post-train pre-trained Cosmos-Predict1 for their applications.

News

• [2025/05] Cosmos AV Single2MultiView is available! Now you can create dynamic, multi-view clips from just one video. Try it out and tell us what you think!
  • Inference guide
  • PyTorch post-training
  • Hugging Face model

Example Model Behavior

Cosmos-Predict Text2World

Your browser does not support the video tag.

Cosmos-Predict Video2World

Your browser does not support the video tag.

Getting Started

We provide a comphrehensive set of examples to illustrate how to perform inference, post-training, etc, with Cosmos-Predict1. Click a relevant example below and start your Cosmos journey.

Installation

Please refer to INSTALL.md for general instructions on environment setup.

Inference with pre-trained Cosmos-Predict1 models

• Inference with diffusion-based Text2World models [with multi-GPU support]
• Inference with diffusion-based Video2World models [with multi-GPU support]
• Inference with diffusion-based WorldInterpolator models [with multi-GPU support]
• Inference with diffusion-based Single2MultiView AV models [with multi-GPU support]
• Inference with autoregressive-based base models [with multi-GPU support]
• Inference with autoregressive-based Video2World models [with multi-GPU support]
• Inference with tokenizer models

Post-train pre-trained Cosmos-Predict1 models

• Post-train diffusion-based Text2World models using custom datasets [with multi-node support]
• Post-train diffusion-based Video2World models using custom datasets [with multi-node support]
• Post-train diffusion-based Video2World models with action control using custom datasets [with multi-node support]
• Post-train diffusion-based WorldInterpolator models using custom datasets [with multi-node support]
• Post-train diffusion-based Text2World models using custom multi-view datasets [with multi-node support]
• Post-train diffusion-based Video2World models using custom multi-view datasets) [with multi-node support]
• Post-train diffusion-based Single2MultiView models using custom multi-view datasets [with multi-node support]
• Post-train autoregressive-based base models using custom datasets [with multi-node support]
• Post-train tokenizers using custom datasets [with multi-node support]

Inference with post-trained models:

• Inference with post-trained multi-view diffusion-based Text2World models) [with multi-GPU support]
• Inference with post-trained multi-view diffusion-based Video2World models) [with multi-GPU support]

Cosmos-Predict1 Models

Cosmos-Predict1 include the following models

Diffusion models

• Cosmos-Predict1-7B-Text2World: Text to visual world generation
• Cosmos-Predict1-14B-Text2World: Text to visual world generation
• Cosmos-Predict1-7B-Video2World: Video + Text based future visual world generation
• Cosmos-Predict1-14B-Video2World: Video + Text based future visual world generation
• Cosmos-Predict1-7B-WorldInterpolator: Generates a smooth, higher-FPS video segment.

Autoregressive models

• Cosmos-Predict1-4B: Future visual world generation
• Cosmos-Predict1-12B: Future visual world generation
• Cosmos-Predict1-5B-Video2World: Video + Text based future visual world generation
• Cosmos-Predict1-13B-Video2World: Video + Text based future visual world generation

Tokenizers

• Cosmos-Tokenize1-CV8×8×8-720p: Continuous Video Tokenizer with 8x8x8 spatio-temporal compression with, 121 frames context
• Cosmos-Tokenize1-DV8×16×16-720p: Discrete Video Tokenizer with 8x16x16 spatio-temporal compression, and 49 frames context
• Cosmos-Tokenize1-CI8×8-360p: Continuous Image Tokenizer with 8x8 spatial compression with low-resolution support
• Cosmos-Tokenize1-CI16x16-360p: Continuous Image Tokenizer with 16x16 spatial compression with low-resolution support
• Cosmos-Tokenize1-CV4×8×8-360p: Continuous Video Tokenizer with 4x8x8 spatio-temporal compression with low-resolution support
• Cosmos-Tokenize1-DI8×8-360p: Discrete Image Tokenizer with 8x8 spatial compression with low-resolution support
• Cosmos-Tokenize1-DI16x16-360p: Discrete Image Tokenizer with 16x16 spatial compression with low-resolution support
• Cosmos-Tokenize1-DV4×8×8-360p: Discrete Video Tokenizer with 4x8x8 spatio-temporal compression with low-resolution support

License and Contact

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

This model includes safety and content moderation features powered by Llama Guard 3. Llama Guard 3 is used solely as a content input filter and is subject to its own license.

NVIDIA Cosmos source code is released under the Apache 2 License.

NVIDIA Cosmos models are released under the NVIDIA Open Model License. For a custom license, please contact cosmos-license@nvidia.com.