SongFormDB 🎵

[English ｜ 中文]

A Large-Scale Multilingual Music Structure Analysis Dataset for Training SongFormer 🚀

---

🌟 What is SongFormDB?

SongFormDB is a comprehensive, large-scale, multilingual dataset designed to revolutionize Music Structure Analysis (MSA). This dataset serves as the training foundation for our state-of-the-art SongFormer model, providing unprecedented scale and diversity for MSA research.

---

✨ Key Highlights

🎯 Four Powerful Subsets

🎸 SongFormDB-HX (HX) - Precision & Quality

• ✅ Rule-corrected HarmonixSet with improved annotation accuracy
• 📊 Unified train/validation/test splits for consistent evaluation

🎸 SongFormDB-Ext (E) - Precision & Quality

• 🀄 Designed for Chinese generalization
• 🎶 4,314 songs for robust training diversity

🎸 SongFormDB-Hook (H) - Scale & Diversity

• 🎼 5,933 songs with precise structural annotations
• 🌍 Helps improve the model's generalization ability

🎸 SongFormDB-Gem (G) - Global Coverage

• 🌐 47 different languages for true multilingual coverage
• 🎶 Diverse BPMs and musical styles for comprehensive training
• 🤖 Gemini-annotated with strong performance on ACC and HR3F metrics
• 🎯 4,387 high-quality songs with music structure analysis

---

📊 Dataset Composition

🎵 SongFormDB-HX (HX) - 712 Songs

Enhanced HarmonixSet with rule-based corrections and unified evaluation protocol.

Data Location: data/HX/SongFormDB-HX.jsonl

🎵 SongFormDB-Ext (E) - 4,314 Songs

Data Location: data/Ext/SongFormDB-Ext.jsonl

🎵 SongFormDB-Hook (H) - 5,933 Songs

Large-scale dataset with precise structural annotations for enhanced generalization.

Data Location: data/Hook/SongFormDB-Hook.jsonl

⚠️ Important Notes:

• Each row corresponds to a structurally annotated segment
• One song may have multiple annotation rows
• Labels are provided as lists (multi-label support)

🎵 SongFormDB-Gem (G) - 4,387 Songs

Globally diverse dataset with Gemini-powered annotations across 47 languages.

Data Location: data/Gem/SongFormDB-Gem.jsonl

⚠️ Important Notes:

• Some YouTube links might be inactive, so the actual number of available samples is slightly reduced.
• Format similar to SongFormDB-HX
• YouTube URLs correspond to actual used data
• Gaps between segments labeled as NO_LABEL due to Gemini's time resolution limitations

---

🚀 Quick Start

Getting the Audio Files

The dataset contains annotations only. To get the actual audio files, follow these instructions based on the dataset version:

SongFormDB-HX

You have two options:

Option 1 (Recommended): Audio Reconstruction

• Use the mel-spectrograms provided in the official HarmonixSet dataset, which are also included in this repository.
• Follow the Audio Reconstruction steps described later in this document

Option 2: YouTube Download

• Download songs from YouTube using this list
• Important: Pay attention to the notes in brackets after each link
• YouTube versions may differ from the original HarmonixSet
• If needed, you can align the audio using: Reference code and mel-spectrograms from the HarmonixSet README
• Note: Alignment may cause audio discontinuities, so Option 1 is preferred

SongFormDB-Ext (E), SongFormDB-Hook (H) and SongFormDB-Gem (G)

Choose either method:

• Direct download from YouTube (better quality)
• Use a vocoder to reconstruct from mel-spectrograms (may have lower quality)

---

🎼 Audio Reconstruction

If YouTube sources become unavailable, reconstruct audio using mel spectrograms:

For SongFormDB-HX:

# Clone BigVGAN repository
git clone https://github.com/NVIDIA/BigVGAN.git
cd utils/HarmonixSet
# Update BIGVGAN_REPO_DIR in inference_e2e.sh
bash inference_e2e.sh

For SongFormDB-Ext (E), SongFormDB-Hook (H) and SongFormDB-Gem (G):

Use bigvgan_v2_44khz_128band_256x:

# Add BigVGAN to PYTHONPATH, then:
# See implementation in utils/CN/infer.py

---

📈 Impact & Applications

• 🎯 Enhanced MSA Performance: Train more robust and accurate music structure analysis models
• 🌍 Cross-lingual Music Understanding: Enable comprehensive multilingual music analysis capabilities that transcend language barriers
• 🎵 Genre Adaptability: Strengthen model generalization across diverse musical styles and genres for broader applicability

---

🤝 Citation

@misc{hao2026songformerscalingmusicstructure,
      title={SongFormer: Scaling Music Structure Analysis with Heterogeneous Supervision}, 
      author={Chunbo Hao and Ruibin Yuan and Jixun Yao and Qixin Deng and Xinyi Bai and Yanbo Wang and Wei Xue and Lei Xie},
      year={2026},
      eprint={2510.02797},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2510.02797}, 
}

---

📧 Contact & Support

🐛 Issues? Open an issue on our GitHub repository
📧 Collaboration? Contact us through GitHub