This repository contains a Cog implementation of ThinkSound, a video-to-audio generation model that thinks through what sounds should happen in your videos. This Cog package makes it easy to run ThinkSound locally or deploy it to Replicate.
⚠️ For research and educational use only. No commercial use.
ThinkSound doesn't just match sounds to objects like other models. Instead, it thinks through what sounds should happen and when, creating natural audio that fits the mood, timing, and context of your video. It's like having an AI sound designer that watches your video and creates a complete audio track that fits perfectly.
Original research:
Clone this repository:
git clone https://github.com/zsxkib/cog-thinksound.git
cd cog-thinksound
Run predictions:
Basic usage:
# Generate audio from video
cog predict -i video=@your_video.mp4
# Add a caption for better results
cog predict -i [email protected] -i caption="Cooking pasta in a kitchen"
Advanced usage with step-by-step reasoning:
# Detailed audio description for precise control
cog predict \
-i [email protected] \
-i caption="Rain on window" \
-i cot="Begin with gentle raindrops hitting glass, gradually building to steady rainfall. Add subtle ambient sounds of water flowing and distant thunder."
Professional controls:
# High quality generation
cog predict \
-i [email protected] \
-i caption="Forest wildlife" \
-i cot="Create layered nature sounds with bird calls, rustling leaves, and distant animal sounds" \
-i num_inference_steps=50 \
-i cfg_scale=7.0
# Fast generation
cog predict \
-i [email protected] \
-i caption="Quick test" \
-i num_inference_steps=15
# Reproducible results
cog predict \
-i [email protected] \
-i caption="Demo video" \
-i seed=42
Push your own version to Replicate:
cog login
cog push r8.im/your-username/thinksound
This Cog package includes optimizations that make it faster and easier to use:
The implementation processes video at two frame rates:
Text processing uses multiple encoders:
Audio generation uses a process with configurable steps and guidance strength.
Important: This model is for research and educational purposes only.
Commercial use is NOT permitted without explicit licensing from the original authors.
This Cog implementation follows the original ThinkSound project's Apache 2.0 license for the code, but the model weights and research are subject to non-commercial restrictions.
For commercial licensing, contact the original research team.
⭐ Star this repo if you find it useful!
🐦 Follow @zsakib_ for updates
This repository contains a Cog implementation of ThinkSound, a video-to-audio generation model that thinks through what sounds should happen in your videos. This Cog package makes it easy to run ThinkSound locally or deploy it to Replicate.
⚠️ For research and educational use only. No commercial use.
ThinkSound doesn't just match sounds to objects like other models. Instead, it thinks through what sounds should happen and when, creating natural audio that fits the mood, timing, and context of your video. It's like having an AI sound designer that watches your video and creates a complete audio track that fits perfectly.
Original research:
Clone this repository:
git clone https://github.com/zsxkib/cog-thinksound.git
cd cog-thinksound
Run predictions:
Basic usage:
# Generate audio from video
cog predict -i video=@your_video.mp4
# Add a caption for better results
cog predict -i [email protected] -i caption="Cooking pasta in a kitchen"
Advanced usage with step-by-step reasoning:
# Detailed audio description for precise control
cog predict \
-i [email protected] \
-i caption="Rain on window" \
-i cot="Begin with gentle raindrops hitting glass, gradually building to steady rainfall. Add subtle ambient sounds of water flowing and distant thunder."
Professional controls:
# High quality generation
cog predict \
-i [email protected] \
-i caption="Forest wildlife" \
-i cot="Create layered nature sounds with bird calls, rustling leaves, and distant animal sounds" \
-i num_inference_steps=50 \
-i cfg_scale=7.0
# Fast generation
cog predict \
-i [email protected] \
-i caption="Quick test" \
-i num_inference_steps=15
# Reproducible results
cog predict \
-i [email protected] \
-i caption="Demo video" \
-i seed=42
Push your own version to Replicate:
cog login
cog push r8.im/your-username/thinksound
This Cog package includes optimizations that make it faster and easier to use:
The implementation processes video at two frame rates:
Text processing uses multiple encoders:
Audio generation uses a process with configurable steps and guidance strength.
Important: This model is for research and educational purposes only.
Commercial use is NOT permitted without explicit licensing from the original authors.
This Cog implementation follows the original ThinkSound project's Apache 2.0 license for the code, but the model weights and research are subject to non-commercial restrictions.
For commercial licensing, contact the original research team.
⭐ Star this repo if you find it useful!
🐦 Follow @zsakib_ for updates