README
Introduction
DiffSynth Studio is a diffusion engine. We have restructured architectures, including Text Encoder, UNet, and VAE, among others, maintaining compatibility with models from the open-source community while enhancing computational performance. We offer many interesting features. Enjoy the magic of diffusion models!
Installation
git clone https://github.com/Agoooy/AI-Video
cd DiffSynth-Studio
pip install -e .
Usage (in Python code)
The Python examples are in examples. We provide an overview here.
Long Video Synthesis
We trained an extended video synthesis model, which can generate 128 frames. examples/ExVideo
https://github.com/modelscope/DiffSynth-Studio/assets/35051019/d97f6aa9-8064-4b5b-9d49-ed6001bb9acc
Image Synthesis
Generate high-resolution images, by breaking the limitation of diffusion models! examples/image_synthesis
| 512*512 | 1024*1024 | 2048*2048 | 4096*4096 |
|---|---|---|---|
| 1024*1024 | 2048*2048 |
|---|---|
Toon Shading
Render realistic videos in a flatten style and enable video editing features. examples/Diffutoon
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/b54c05c5-d747-4709-be5e-b39af82404dd
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/20528af5-5100-474a-8cdc-440b9efdd86c
Video Stylization
Video stylization without video models. examples/diffsynth
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-4481-b79f-0c3a7361a1ea
Chinese Models
Use Hunyuan-DiT to generate images with Chinese prompts. We also support LoRA fine-tuning of this model. examples/hunyuan_dit
Prompt: 少女手捧鲜花，坐在公园的长椅上，夕阳的余晖洒在少女的脸庞，整个画面充满诗意的美感
| 1024x1024 | 2048x2048 (highres-fix) |
|---|---|
Prompt: 一只小狗蹦蹦跳跳，周围是姹紫嫣红的鲜花，远处是山脉
| Without LoRA | With LoRA |
|---|---|
Usage (in WebUI)
python -m streamlit run DiffSynth_Studio.py
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/93085557-73f3-4eee-a205-9829591ef954
Similar repos
Languages
Python
100.0%
README
Introduction
DiffSynth Studio is a diffusion engine. We have restructured architectures, including Text Encoder, UNet, and VAE, among others, maintaining compatibility with models from the open-source community while enhancing computational performance. We offer many interesting features. Enjoy the magic of diffusion models!
Installation
git clone https://github.com/Agoooy/AI-Video
cd DiffSynth-Studio
pip install -e .
Usage (in Python code)
The Python examples are in examples. We provide an overview here.
Long Video Synthesis
We trained an extended video synthesis model, which can generate 128 frames. examples/ExVideo
https://github.com/modelscope/DiffSynth-Studio/assets/35051019/d97f6aa9-8064-4b5b-9d49-ed6001bb9acc
Image Synthesis
Generate high-resolution images, by breaking the limitation of diffusion models! examples/image_synthesis
| 512*512 | 1024*1024 | 2048*2048 | 4096*4096 |
|---|---|---|---|
| 1024*1024 | 2048*2048 |
|---|---|
Toon Shading
Render realistic videos in a flatten style and enable video editing features. examples/Diffutoon
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/b54c05c5-d747-4709-be5e-b39af82404dd
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/20528af5-5100-474a-8cdc-440b9efdd86c
Video Stylization
Video stylization without video models. examples/diffsynth
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/59fb2f7b-8de0-4481-b79f-0c3a7361a1ea
Chinese Models
Use Hunyuan-DiT to generate images with Chinese prompts. We also support LoRA fine-tuning of this model. examples/hunyuan_dit
Prompt: 少女手捧鲜花，坐在公园的长椅上，夕阳的余晖洒在少女的脸庞，整个画面充满诗意的美感
| 1024x1024 | 2048x2048 (highres-fix) |
|---|---|
Prompt: 一只小狗蹦蹦跳跳，周围是姹紫嫣红的鲜花，远处是山脉
| Without LoRA | With LoRA |
|---|---|
Usage (in WebUI)
python -m streamlit run DiffSynth_Studio.py
https://github.com/Artiprocher/DiffSynth-Studio/assets/35051019/93085557-73f3-4eee-a205-9829591ef954
Similar repos
Languages
Python
100.0%