Text-to-Video, Image-to-Video, Start-End Frames, Video Completion, Video Extension, Video Transition, and more.... Below are some showcases for Pusa-Wan2.2-V1. Please refer to Pusa V1.0 README for detailed info.
Example 1: Image-to-Video in 4 Steps
|
noise: 0.2, high_lora_alpha 1.5 |
noise: 0.3, high_lora_alpha 1.4 |
|
noise: 0.2, high_lora_alpha 1.5 |
noise: 0.2, high_lora_alpha 1.5 |
Example 2: Video Extension in 4 Steps
|
noise: [0.0, 0.3, 0.5, 0.7], high_lora_alpha 1.5 |
noise: [0.2, 0.4, 0.4, 0.4], high_lora_alpha 1.4 |
Example 3: Start-End Frames in 4 Steps
|
noise: [0.2, 0.5], high_lora_alpha 1.5 |
noise: [0.0, 0.4], high_lora_alpha 1.5 |
Example 4: Text-to-Video with in 4 Steps
--lightx2v acceleration flagPusa (pu: 'sA:, from "Thousand-Hand Guanyin" in Chinese) introduces a paradigm shift in video diffusion modeling through frame-level noise control with vectorized timesteps, departing from conventional scalar timestep approaches. This shift was first presented in our FVDM paper.
Pusa V1.0 is based on the SOTA Wan-T2V-14B model and enhances it with our unique vectorized timestep adaptations (VTA), a non-destructive adaptation that fully preserves the capabilities of the base model. With the new Wan2.2 support, Pusa now leverages MoE DiT architecture for improved quality and efficiency.
The codebase has been integrated into the PusaV1 directory, based on DiffSynth-Studio.
Pusa V1.0 not only sets a new standard for image-to-video generation but also unlocks many other zero-shot multi-task capabilities such as start-end frames and video extension, all without task-specific training while preserving the base model's T2V capabilities.
For detailed usage and examples for Pusa V1.0, please see the Pusa V1.0 README.
Pusa V0.5 leverages this architecture, and it is based on Mochi1-Preview. We are open-sourcing this work to foster community collaboration, enhance methodologies, and expand capabilities.
Comprehensive Multi-task Support:
Unprecedented Efficiency:
Advanced Architecture Options:
Complete Open-Source Release:
Novel Diffusion Paradigm: Implements frame-level noise control with vectorized timesteps, originally introduced in the FVDM paper, enabling unprecedented flexibility and scalability.
MoE DiT Support: Wan2.2 models feature separate high-noise and low-noise DiT components, allowing for better noise handling and improved generation quality.
Lightning Acceleration: LightX2V integration provides 4-step inference with maintained quality, dramatically reducing generation time.
Non-destructive Modification: Our adaptations to the base model preserve its original Text-to-Video generation capabilities. After this adaptation, we only need a slight fine-tuning.
Universal Applicability: The methodology can be readily applied to other leading video diffusion models including Hunyuan Video, Wan2.1, and others. Collaborations enthusiastically welcomed!
Pusa V1.0 leverages the powerful Wan-Video models and enhances them with our custom LoRA models and training scripts. Now featuring both Wan2.1 and Wan2.2 architecture support with LightX2V acceleration capabilities.
Model Options:
Key Features:
cfg_scale=1 for LightX2V, cfg_scale=3 for standard inferenceFor detailed instructions on installation, model preparation, usage examples, and training, please refer to the Pusa V1.0 README.
You may install using uv:
git clone https://github.com/genmoai/models
cd models
pip install uv
uv venv .venv
source .venv/bin/activate
uv pip install setuptools
uv pip install -e . --no-build-isolation
If you want to install flash attention, you can use:
uv pip install -e .[flash] --no-build-isolation
Option 1: Use the Hugging Face CLI:
pip install huggingface_hub
huggingface-cli download RaphaelLiu/Pusa-V0.5 --local-dir <path_to_downloaded_directory>
Option 2: Download directly from Hugging Face to your local machine.
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Your_prompt_here" \
--image_dir "/path/to/input/image.jpg" \
--cond_position 0 \
--num_steps 30 \
--noise_multiplier 0
Note: We suggest you try different con_position here, and you may also modify the level of noise added to the condition image. You'd be likely to get some surprises.
Take ./demos/example.jpg as an example and run with 4 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3 python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "The camera remains still, the man is surfing on a wave with his surfboard." \
--image_dir "./demos/example.jpg" \
--cond_position 0 \
--num_steps 30 \
--noise_multiplier 0.4
You can get this result:
You may refer to the baselines' results from the VideoGen-Eval benchmark for comparison:
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--image_dir "/path/to/image/directory" \
--prompt_dir "/path/to/prompt/directory" \
--cond_position 1 \
--num_steps 30
For group processing, each image should have a corresponding text file with the same name in the prompt directory.
We also provide a shell script for convenience:
# Edit cli_test_ti2v_release.sh to set your paths
# Then run:
bash ./demos/cli_test_ti2v_release.sh
Pusa supports generating videos from multiple keyframes (2 or more) placed at specific positions in the sequence. This is useful for both start-end frame generation and multi-keyframe interpolation.
python ./demos/cli_test_multi_frames_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Drone view of waves crashing against the rugged cliffs along Big Surβs garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliffβs edge. The steep drop from the road down to the beach is a dramatic feat, with the cliffβs edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway." \
--multi_cond '{"0": ["./demos/example3.jpg", 0.3], "20": ["./demos/example5.jpg", 0.7]}' \
--num_steps 30
The multi_cond parameter specifies frame condition positions and their corresponding image paths and noise multipliers. In this example, the first frame (position 0) uses ./demos/example3.jpg with noise multiplier 0.3, and frame 20 uses ./demos/example5.jpg with noise multiplier 0.5.
Alternatively, use the provided shell script:
# Edit parameters in cli_test_multi_frames_release.sh first
bash ./demos/cli_test_multi_frames_release.sh
To generate videos with more than two keyframes (e.g., start, middle, and end):
python ./demos/cli_test_multi_frames_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Drone view of waves crashing against the rugged cliffs along Big Surβs garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliffβs edge. The steep drop from the road down to the beach is a dramatic feat, with the cliffβs edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway." \
--multi_cond '{"0": ["./demos/example3.jpg", 0.3], "13": ["./demos/example4.jpg", 0.7], "27": ["./demos/example5.jpg", 0.7]}' \
--num_steps 30
Create smooth transitions between two videos:
python ./demos/cli_test_transition_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A fluffy Cockapoo, perched atop a vibrant pink flamingo jumps into a crystal-clear pool." \
--video_start_dir "./demos/example1.mp4" \
--video_end_dir "./demos/example2.mp4" \
--cond_position_start "[0]" \
--cond_position_end "[-3,-2,-1]" \
--noise_multiplier "[0.3,0.8,0.8,0.8]" \
--num_steps 30
Parameters:
cond_position_start: Frame indices from the start video to use as conditioningcond_position_end: Frame indices from the end video to use as conditioningnoise_multiplier: Noise level multipliers for each conditioning frameAlternatively, use the provided shell script:
# Edit parameters in cli_test_transition_release.sh first
bash ./demos/cli_test_transition_release.sh
Extend existing videos with generated content:
python ./demos/cli_test_extension_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A cinematic shot captures a fluffy Cockapoo, perched atop a vibrant pink flamingo float, in a sun-drenched Los Angeles swimming pool. The crystal-clear water sparkles under the bright California sun, reflecting the playful scene." \
--video_dir "./demos/example1.mp4" \
--cond_position "[0,1,2,3]" \
--noise_multiplier "[0.1,0.2,0.3,0.4]" \
--num_steps 30
Parameters:
cond_position: Frame indices from the input video to use as conditioningnoise_multiplier: Noise level multipliers for each conditioning frameAlternatively, use the provided shell script:
# Edit parameters in cli_test_v2v_release.sh first
bash ./demos/cli_test_v2v_release.sh
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A man is playing basketball" \
--num_steps 30
For Pusa V1.0, please find the training details in the Pusa V1.0 README.
For Pusa V0.5, you can find our training code and details here, which also supports training for the original Mochi model.
Pusa currently has several known limitations:
v1.0.1 (September 1, 2025)
wan22_* inference scripts with MoE LoRA supportwan_* scripts with --lightx2v acceleration flagv1.0 (July 15, 2025)
/PusaV1.PusaV1/.v0.5 (June 3, 2025)
v0.5 (April 10, 2025)
If you use this work in your project, please cite the following references.
@article{liu2025pusa,
title={PUSA V1. 0: Surpassing Wan-I2V with $500 Training Cost by Vectorized Timestep Adaptation},
author={Liu, Yaofang and Ren, Yumeng and Artola, Aitor and Hu, Yuxuan and Cun, Xiaodong and Zhao, Xiaotong and Zhao, Alan and Chan, Raymond H and Zhang, Suiyun and Liu, Rui and others},
journal={arXiv preprint arXiv:2507.16116},
year={2025}
}
@misc{Liu2025pusa,
title={Pusa: Thousands Timesteps Video Diffusion Model},
author={Yaofang Liu and Rui Liu},
year={2025},
url={https://github.com/Yaofang-Liu/Pusa-VidGen},
}
@article{liu2024redefining,
Β title={Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach},
Β author={Liu, Yaofang and Ren, Yumeng and Cun, Xiaodong and Artola, Aitor and Liu, Yang and Zeng, Tieyong and Chan, Raymond H and Morel, Jean-michel},
Β journal={arXiv preprint arXiv:2410.03160},
Β year={2024}
}
Python
99.9%
Text-to-Video, Image-to-Video, Start-End Frames, Video Completion, Video Extension, Video Transition, and more.... Below are some showcases for Pusa-Wan2.2-V1. Please refer to Pusa V1.0 README for detailed info.
Example 1: Image-to-Video in 4 Steps
|
noise: 0.2, high_lora_alpha 1.5 |
noise: 0.3, high_lora_alpha 1.4 |
|
noise: 0.2, high_lora_alpha 1.5 |
noise: 0.2, high_lora_alpha 1.5 |
Example 2: Video Extension in 4 Steps
|
noise: [0.0, 0.3, 0.5, 0.7], high_lora_alpha 1.5 |
noise: [0.2, 0.4, 0.4, 0.4], high_lora_alpha 1.4 |
Example 3: Start-End Frames in 4 Steps
|
noise: [0.2, 0.5], high_lora_alpha 1.5 |
noise: [0.0, 0.4], high_lora_alpha 1.5 |
Example 4: Text-to-Video with in 4 Steps
--lightx2v acceleration flagPusa (pu: 'sA:, from "Thousand-Hand Guanyin" in Chinese) introduces a paradigm shift in video diffusion modeling through frame-level noise control with vectorized timesteps, departing from conventional scalar timestep approaches. This shift was first presented in our FVDM paper.
Pusa V1.0 is based on the SOTA Wan-T2V-14B model and enhances it with our unique vectorized timestep adaptations (VTA), a non-destructive adaptation that fully preserves the capabilities of the base model. With the new Wan2.2 support, Pusa now leverages MoE DiT architecture for improved quality and efficiency.
The codebase has been integrated into the PusaV1 directory, based on DiffSynth-Studio.
Pusa V1.0 not only sets a new standard for image-to-video generation but also unlocks many other zero-shot multi-task capabilities such as start-end frames and video extension, all without task-specific training while preserving the base model's T2V capabilities.
For detailed usage and examples for Pusa V1.0, please see the Pusa V1.0 README.
Pusa V0.5 leverages this architecture, and it is based on Mochi1-Preview. We are open-sourcing this work to foster community collaboration, enhance methodologies, and expand capabilities.
Comprehensive Multi-task Support:
Unprecedented Efficiency:
Advanced Architecture Options:
Complete Open-Source Release:
Novel Diffusion Paradigm: Implements frame-level noise control with vectorized timesteps, originally introduced in the FVDM paper, enabling unprecedented flexibility and scalability.
MoE DiT Support: Wan2.2 models feature separate high-noise and low-noise DiT components, allowing for better noise handling and improved generation quality.
Lightning Acceleration: LightX2V integration provides 4-step inference with maintained quality, dramatically reducing generation time.
Non-destructive Modification: Our adaptations to the base model preserve its original Text-to-Video generation capabilities. After this adaptation, we only need a slight fine-tuning.
Universal Applicability: The methodology can be readily applied to other leading video diffusion models including Hunyuan Video, Wan2.1, and others. Collaborations enthusiastically welcomed!
Pusa V1.0 leverages the powerful Wan-Video models and enhances them with our custom LoRA models and training scripts. Now featuring both Wan2.1 and Wan2.2 architecture support with LightX2V acceleration capabilities.
Model Options:
Key Features:
cfg_scale=1 for LightX2V, cfg_scale=3 for standard inferenceFor detailed instructions on installation, model preparation, usage examples, and training, please refer to the Pusa V1.0 README.
You may install using uv:
git clone https://github.com/genmoai/models
cd models
pip install uv
uv venv .venv
source .venv/bin/activate
uv pip install setuptools
uv pip install -e . --no-build-isolation
If you want to install flash attention, you can use:
uv pip install -e .[flash] --no-build-isolation
Option 1: Use the Hugging Face CLI:
pip install huggingface_hub
huggingface-cli download RaphaelLiu/Pusa-V0.5 --local-dir <path_to_downloaded_directory>
Option 2: Download directly from Hugging Face to your local machine.
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Your_prompt_here" \
--image_dir "/path/to/input/image.jpg" \
--cond_position 0 \
--num_steps 30 \
--noise_multiplier 0
Note: We suggest you try different con_position here, and you may also modify the level of noise added to the condition image. You'd be likely to get some surprises.
Take ./demos/example.jpg as an example and run with 4 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3 python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "The camera remains still, the man is surfing on a wave with his surfboard." \
--image_dir "./demos/example.jpg" \
--cond_position 0 \
--num_steps 30 \
--noise_multiplier 0.4
You can get this result:
You may refer to the baselines' results from the VideoGen-Eval benchmark for comparison:
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--image_dir "/path/to/image/directory" \
--prompt_dir "/path/to/prompt/directory" \
--cond_position 1 \
--num_steps 30
For group processing, each image should have a corresponding text file with the same name in the prompt directory.
We also provide a shell script for convenience:
# Edit cli_test_ti2v_release.sh to set your paths
# Then run:
bash ./demos/cli_test_ti2v_release.sh
Pusa supports generating videos from multiple keyframes (2 or more) placed at specific positions in the sequence. This is useful for both start-end frame generation and multi-keyframe interpolation.
python ./demos/cli_test_multi_frames_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Drone view of waves crashing against the rugged cliffs along Big Surβs garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliffβs edge. The steep drop from the road down to the beach is a dramatic feat, with the cliffβs edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway." \
--multi_cond '{"0": ["./demos/example3.jpg", 0.3], "20": ["./demos/example5.jpg", 0.7]}' \
--num_steps 30
The multi_cond parameter specifies frame condition positions and their corresponding image paths and noise multipliers. In this example, the first frame (position 0) uses ./demos/example3.jpg with noise multiplier 0.3, and frame 20 uses ./demos/example5.jpg with noise multiplier 0.5.
Alternatively, use the provided shell script:
# Edit parameters in cli_test_multi_frames_release.sh first
bash ./demos/cli_test_multi_frames_release.sh
To generate videos with more than two keyframes (e.g., start, middle, and end):
python ./demos/cli_test_multi_frames_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "Drone view of waves crashing against the rugged cliffs along Big Surβs garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliffβs edge. The steep drop from the road down to the beach is a dramatic feat, with the cliffβs edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway." \
--multi_cond '{"0": ["./demos/example3.jpg", 0.3], "13": ["./demos/example4.jpg", 0.7], "27": ["./demos/example5.jpg", 0.7]}' \
--num_steps 30
Create smooth transitions between two videos:
python ./demos/cli_test_transition_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A fluffy Cockapoo, perched atop a vibrant pink flamingo jumps into a crystal-clear pool." \
--video_start_dir "./demos/example1.mp4" \
--video_end_dir "./demos/example2.mp4" \
--cond_position_start "[0]" \
--cond_position_end "[-3,-2,-1]" \
--noise_multiplier "[0.3,0.8,0.8,0.8]" \
--num_steps 30
Parameters:
cond_position_start: Frame indices from the start video to use as conditioningcond_position_end: Frame indices from the end video to use as conditioningnoise_multiplier: Noise level multipliers for each conditioning frameAlternatively, use the provided shell script:
# Edit parameters in cli_test_transition_release.sh first
bash ./demos/cli_test_transition_release.sh
Extend existing videos with generated content:
python ./demos/cli_test_extension_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A cinematic shot captures a fluffy Cockapoo, perched atop a vibrant pink flamingo float, in a sun-drenched Los Angeles swimming pool. The crystal-clear water sparkles under the bright California sun, reflecting the playful scene." \
--video_dir "./demos/example1.mp4" \
--cond_position "[0,1,2,3]" \
--noise_multiplier "[0.1,0.2,0.3,0.4]" \
--num_steps 30
Parameters:
cond_position: Frame indices from the input video to use as conditioningnoise_multiplier: Noise level multipliers for each conditioning frameAlternatively, use the provided shell script:
# Edit parameters in cli_test_v2v_release.sh first
bash ./demos/cli_test_v2v_release.sh
python ./demos/cli_test_ti2v_release.py \
--model_dir "/path/to/Pusa-V0.5" \
--dit_path "/path/to/Pusa-V0.5/pusa_v0_dit.safetensors" \
--prompt "A man is playing basketball" \
--num_steps 30
For Pusa V1.0, please find the training details in the Pusa V1.0 README.
For Pusa V0.5, you can find our training code and details here, which also supports training for the original Mochi model.
Pusa currently has several known limitations:
v1.0.1 (September 1, 2025)
wan22_* inference scripts with MoE LoRA supportwan_* scripts with --lightx2v acceleration flagv1.0 (July 15, 2025)
/PusaV1.PusaV1/.v0.5 (June 3, 2025)
v0.5 (April 10, 2025)
If you use this work in your project, please cite the following references.
@article{liu2025pusa,
title={PUSA V1. 0: Surpassing Wan-I2V with $500 Training Cost by Vectorized Timestep Adaptation},
author={Liu, Yaofang and Ren, Yumeng and Artola, Aitor and Hu, Yuxuan and Cun, Xiaodong and Zhao, Xiaotong and Zhao, Alan and Chan, Raymond H and Zhang, Suiyun and Liu, Rui and others},
journal={arXiv preprint arXiv:2507.16116},
year={2025}
}
@misc{Liu2025pusa,
title={Pusa: Thousands Timesteps Video Diffusion Model},
author={Yaofang Liu and Rui Liu},
year={2025},
url={https://github.com/Yaofang-Liu/Pusa-VidGen},
}
@article{liu2024redefining,
Β title={Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach},
Β author={Liu, Yaofang and Ren, Yumeng and Cun, Xiaodong and Artola, Aitor and Liu, Yang and Zeng, Tieyong and Chan, Raymond H and Morel, Jean-michel},
Β journal={arXiv preprint arXiv:2410.03160},
Β year={2024}
}
Python
99.9%