MaskGCT-Windows

MaskGCT-Windows is a Windows-compatible implementation of MaskGCT: a state-of-the-art, zero-shot, non-autoregressive Text-to-Speech (TTS) model that eliminates the need for explicit text-speech alignment and duration prediction. This implementation allows users to leverage the model's powerful capabilities in a Windows environment.

---

Overview

MaskGCT is a fully non-autoregressive TTS model using a two-stage approach:

1. Predicting semantic tokens from text using a self-supervised learning (SSL) model.
2. Generating acoustic tokens conditioned on the semantic tokens using a mask-and-predict learning paradigm.

MaskGCT generates high-quality, intelligible, and similar speech in a parallel manner. It outperforms other zero-shot TTS models when trained on large-scale, diverse datasets. You can find more details and audio samples on the demo page.

---

Prerequisites

• Python 3.10: Install Anaconda Python or regular Python 3.10.
• Git: Install Git for Windows (see instructions below).
• eSpeak-NG: A text-to-phoneme converter used in MaskGCT.
• FFmpeg: Required for audio processing.

---

Installation Steps

1. Installing Git

1. Download Git for Windows from the official Git website.
2. Run the installer and follow the on-screen instructions.
3. After installation, you should be able to use Git commands in your terminal.

---

2. Installing eSpeak-NG

1. Download espeak-ng-X64.msi from this link and run the installer.
2. Add the following environment variable:
   • Variable Name: PHONEMIZER_ESPEAK_LIBRARY
   • Value: C:\Program Files\eSpeak NG\libespeak-ng.dll
3. Note: Remember to restart your terminal to apply the changes.

---

3. Installing FFmpeg

1. Download and install FFmpeg from the official FFmpeg website.
2. Add the path to ffmpeg.exe to your system's environment variables.

---

4. Setting Up the Python Environment

1. Clone the repository:

   git clone https://github.com/justinjohn0306/MaskGCT-Windows.git
   cd MaskGCT-Windows
   

2. Create a Conda environment and install dependencies:

   conda create -n maskgct python=3.10
   conda activate maskgct
   pip install -r requirements.txt
   

---

5. Installing PyTorch

To install PyTorch, follow these steps based on your system's configuration:

1. Visit the official PyTorch website to get the latest installation command tailored to your system.
2. Use the following command for a standard installation with CUDA (GPU support):

   # If you have an NVIDIA GPU and want to use CUDA
   conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia
   

3. Or, if you want to use the CPU version:

   conda install pytorch torchvision torchaudio cpuonly -c pytorch
   

Note: Ensure that the PyTorch version you are installing matches the CUDA version installed on your system. This is crucial for the proper installation and functioning of onnxruntime. Additionally, if you don't already have CUDA and cuDNN installed on your system, download and install them from the NVIDIA CUDA Toolkit.

---

6. Downloading the Models

1. Download the pre-trained models from this link.
2. Unzip the downloaded file and place the contents inside the repository.

---

Usage

You can run MaskGCT using the Gradio Playground with the following command:

Running Gradio App

python app.py

---

Citations

If you use MaskGCT in your research, please cite the following papers:

@article{wang2024maskgct,
  title={MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer},
  author={Wang, Yuancheng and others},
  journal={arXiv preprint arXiv:2409.00750},
  year={2024}
}
@inproceedings{amphion,
  author={Zhang, Xueyao and others},
  title={Amphion: An Open-Source Audio, Music and Speech Generation Toolkit},
  booktitle={{IEEE} SLT Workshop},
  year={2024}
}