🎤 Hum to Song

Hum a melody for 10–30 seconds, add a style line and lyrics, and get back a produced song that keeps your tune, builds a structure around it, and carries on long after the hum stops.

This Space runs Mothersuperior/YuE2-hum-to-song on top of m-a-p/YuE2-3B, following the adapter's reference scripts (hum_continue.py, infer_hum.py):

1. Score continuation. SheetSage2 transcribes the hum to ABC in YuE2's two-voice score format. The prompt is built as [EOD] instructions+style+lyrics [ABC_START] hum score and deliberately left open (no [ABC_END]), so the planner treats the hummed bars as the opening of a song and keeps writing — new sections, transitions, an ending — before emitting semantic tokens for the whole song.
2. Prosody adapter. The hum is reduced to a pitch carrier (pYIN f0 → sine wave, amplitude = the rectified hum low-passed with a 4th-order 30 Hz and a 2nd-order 80 Hz Butterworth), VAE-encoded to 25 Hz latents, and four Linear(64 → 2048) projections add it into the NAR hidden state at layers 0 / 7 / 14 / 21 while the rank-96 LoRA-adapted flow-matching decoder renders the audio. You can hear exactly what the decoder is given in the Pitch carrier player.

Three melody modes: Continue from hum (the above), Hum only (close the score right after the hum, so the song is exactly your melody), and Ignore hum melody (the planner writes its own tune and the adapter only shapes phrasing).

Notes

• The transcriber needs something voice-like — a real hum works, a synthesised tone does not.
• The adapter is subtle by design; it is best judged over the first hummed section.
• The hum sets the melody and the opening, not the length: the planner decides where the song ends, and Max song length is only a cap.
• Weights derive from YuE2-3B and are CC BY-NC 4.0 — non-commercial use only.
• The example hum comes from the CHAD hummings subset (Amatov et al., A Semi-Supervised Deep Learning Approach to Dataset Collection for Query-by-Humming Task, ISMIR 2023), CC BY-NC 4.0.