[Pytorch repo] [Samples] [huggingface | online demo] [Flutter app] [arXiv]

Arabic TTS models (FastPitch, MixerTTS) from the tts-arabic-pytorch repo in the ONNX format — usable as a Python package for offline speech synthesis.

Audio samples can be found here.

Install with

pip install git+https://github.com/nipponjo/tts_arabic.git

Examples

# %%
from tts_arabic import tts

# %%
text = "السَّلامُ عَلَيكُم يَا صَدِيقِي."
wave = tts(text, speaker=2, pace=0.9, play=True)

# %% Buckwalter transliteration
text = ">ls~alAmu Ealaykum yA Sadiyqiy."
wave = tts(text, speaker=0, play=True)

# %% Unvocalized input
text_unvoc = "القهوة مشروب يعد من بذور البن المحمصة"
wave = tts(text_unvoc, play=True, vowelizer='shakkelha')

Pretrained models

The sequence of transformations is as follows:

Text → Phonemizer → Phonemes → Tokenizer → Token Ids → Text->Mel model → Mel spectrogram → Vocoder model → Wave

The Text->Mel models map token ids to mel frames. All models use the 80 bin configuration proposed by HiFi-GAN. This mel spectrogram contains frequencies up to 8kHz. The vocoder models map the mel spectrogram to a waveform. The vocoders with vocoder_id hifigan and vocos artificially extend the bandwidth to 11025Hz, and vocos44 to 22050Hz. Samples for comparing the models can be found here.

Manuscript

More information about how the models were trained can be found in the manuscript Arabic TTS with FastPitch: Reproducible Baselines, Adversarial Training, and Oversmoothing Analysis (arXiv | ResearchGate).

TTS options

from tts_arabic import tts

text = "السَّلامُ عَلَيكُم يَا صَدِيقِي."
wave = tts(
    text, # input text
    speaker = 1, # speaker id; choose between 0,1,2,3
    pace = 1, # speaker pace
    denoise = 0.005, # vocoder denoiser strength
    volume = 0.9, # Max amplitude (between 0 and 1)
    play = True, # play audio?
    pitch_mul = 1, # pitch multiplier
    pitch_add = 0, # pitch offset
    vowelizer = None, # vowelizer model
    model_id = 'fastpitch', # Model ID for Text->Mel model
    vocoder_id = 'hifigan', # Model ID for vocoder model
    cuda = None, # Optional; CUDA device index
    save_to = './test.wav', # Optionally; save audio WAV file
    bits_per_sample = 32, # when save_to is specified (8, 16, 24 or 32 bits)
    )

Vowelizer models

References

The vocoder vocos44 was converted from (patriotyk/vocos-mel-hifigan-compat-44100khz).

The vowelizer catt_eo was converted from https://github.com/abjadai/catt/releases/tag/v2 best_eo_mlm_ns_epoch_193.pt (License: Apache-2.0)