ControlFoley — Unified and Controllable Video-to-Audio Generation
5
13 commits
1 linked in READMEs
updated Aug 27, 2026
README
ControlFoley — Unified and Controllable Video-to-Audio Generation
Demo of ControlFoley, accepted at ACM MM 2026: a unified model for controllable sound-effect and video-to-audio generation.
Supported tasks (selected automatically from the inputs you provide):
| Task | Video | Reference audio | Prompt |
|---|---|---|---|
| TV2A / TC-V2A | ✅ | – | ✅ |
| AC-V2A (reference-audio control) | ✅ | ✅ | optional |
| V2A | ✅ | – | – |
| T2A | – | – | ✅ |
Weights are downloaded at runtime from
YJX-Xiaomi/ControlFoley.
The five interactive examples use the official full-task inputs from the
YJX-Xiaomi/ControlFoley examples.
Demo media credits: audio from Pixabay; video from Pexels and Jimeng AI-generated
content.
Contributors
11 commits
2 commits
Similar repos
ControlFoley — Unified and Controllable Video-to-Audio Generation
5
13 commits
1 linked in READMEs
updated Aug 27, 2026
README
ControlFoley — Unified and Controllable Video-to-Audio Generation
Demo of ControlFoley, accepted at ACM MM 2026: a unified model for controllable sound-effect and video-to-audio generation.
Supported tasks (selected automatically from the inputs you provide):
| Task | Video | Reference audio | Prompt |
|---|---|---|---|
| TV2A / TC-V2A | ✅ | – | ✅ |
| AC-V2A (reference-audio control) | ✅ | ✅ | optional |
| V2A | ✅ | – | – |
| T2A | – | – | ✅ |
Weights are downloaded at runtime from
YJX-Xiaomi/ControlFoley.
The five interactive examples use the official full-task inputs from the
YJX-Xiaomi/ControlFoley examples.
Demo media credits: audio from Pixabay; video from Pexels and Jimeng AI-generated
content.
Similar repos
Linked in READMEs
Contributors
11 commits
2 commits