DiffusionDemo

This repository provides an in-depth exploration of stable diffusion models, walking readers through the inner workings in a step-by-step manner.

Live Demo

Try the application online: diffusiondemo.org

Getting Started

To run the demo, click "Open in Colab". Follow the instructions to execute the code and visualize diffusion in action.

Demo Tabs Overview

1. Latent Space:

   • Denoising: Observe the intermediate images during denoising.
   • Seeds: Understand how different starting points in latent space can lead to different images.
   • Perturbations: Explore different perturbations from a point in latent space.
   • Circular: Generate a circular path in latent space and observe how the images vary along the path.
   • Poke: Perturb a region in the image and observe the effect.
   • Guidance: Observe the effect of different guidance scales.
   • Inpainting: Inpaint the image based on the prompt.

2. Clip Space:

   • Embeddings: Visualize text embedding space in 3D with input texts and output images based on the chosen axis.
   • Interpolate: Interpolate between the first and the second prompt, and observe how the output changes.
   • Negative: Observe the effect of negative prompts.

References

Here are some additional references you might find interesting:

• Cats in Latent Space
• Diffusion Explainer: Stable Diffusion Explained with Visualization
• The Illustrated Stable Diffusion

Collaborators

Author: Adithya Kameswara Rao, Carnegie Mellon University.
Advisor: David S. Touretzky, Carnegie Mellon University.

Funding

This work was funded by a grant from NEOM Company, and by National Science Foundation award IIS-2112633.

Contributing

If you encounter any issues, please don't hesitate to open an issue in this repository.

Happy exploring!