HIFI-BEGANSing: Korean Singing Voice Synthesis with BigVGAN Vocoder

Korean singing voice synthesis system using BEGAN (Boundary Equilibrium GAN) with BigVGAN v2 vocoder for high-quality audio synthesis.

Based on Korean Singing Voice Synthesis based on Auto-regressive Boundary Equilibrium GAN.

Features

• Mel Spectrogram Output: 80-dim mel spectrogram (HiFi-GAN/BigVGAN compatible)
• BigVGAN v2 Vocoder: State-of-the-art neural vocoder for high-fidelity audio synthesis
• End-to-End: Text + MIDI → Singing Voice

Architecture

Text + MIDI → Embedding → Prenet → UNet → Mel Spectrogram (80) → BigVGAN → Audio

Installation

# Install PyTorch (https://pytorch.org/)
pip install torch torchaudio

# Install dependencies
pip install -r requirements.txt

# Install BigVGAN vocoder
pip install bigvgan

Vocoder

This project uses BigVGAN v2 from NVIDIA for audio synthesis. The model is automatically downloaded from Hugging Face on first inference.

• Model: nvidia/bigvgan_v2_22khz_80band_fmax8k_256x
• Sample rate: 22050 Hz
• Mel bands: 80
• fmax: 8000 Hz

No manual download required - the vocoder weights are fetched automatically.

Dataset

Dataset requirements:

• Text file (.txt): Korean lyrics (same syllable count as MIDI notes)
• MIDI file (.mid): Temporally aligned with audio
• Audio file (.wav): 22050 Hz sample rate

Structure (file_structure: 2):

dataset/
├── txt/
├── mid/
└── wav/

Usage

1. Preprocessing

python preprocess.py -c config/default_train.yml

2. Training

python train.py -c config/default_train.yml --device 0 --batch_size 32

TensorBoard monitoring:

tensorboard --logdir=checkpoint/default

3. Inference

python infer.py -c config/default_infer.yml

Output: .wav file synthesized with BigVGAN v2 vocoder.

Configuration

default_train.yml

default_infer.yml

License

• g2p/korean_g2p.py: KoG2P
• utils/midi_utils.py: Madmom
• BigVGAN: MIT License

References

@inproceedings{choi2020korean,
  title={Korean Singing Voice Synthesis Based on Auto-Regressive Boundary Equilibrium GAN},
  author={Choi, Soonbeom and Kim, Wonil and Park, Saebyul and Yong, Sangeon and Nam, Juhan},
  booktitle={ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={7234--7238},
  year={2020},
  organization={IEEE}
}

@inproceedings{lee2023bigvgan,
  title={BigVGAN: A Universal Neural Vocoder with Large-Scale Training},
  author={Lee, Sang-gil and Ping, Wei and Ginsburg, Boris and Catanzaro, Bryan and Yoon, Sungroh},
  booktitle={ICLR},
  year={2023}
}