🦜 VieNeu-TTS

VieNeu-TTS-v2 is the next generation of on-device Vietnamese TTS, featuring 10,000+ hours of bilingual training, instant voice cloning, and a dedicated Podcast/Conversation mode.

[!NOTE] 🆕 VieNeu-TTS v3 Turbo (early access) is out for preview! A brand-new architecture designed and trained from scratch by Phạm Nguyễn Ngọc Bảo (codec: MOSS-Audio-Tokenizer-Nano; phonemizer: sea-g2p):

• 48 kHz high-fidelity audio (up from 24 kHz).
• Built-in default voices via dedicated speaker tokens — stable, consistent, no reference clip needed.
• Emotion / non-verbal cues (experimental): drop [cười], [thở dài], [hắng giọng] straight into the text.
• Batched generation (batch size up to 32), including a multi-speaker Conversation mode that batches the whole script regardless of speaker.
• Instant voice cloning from 3–5s of audio.

Try it in the Web UI (backbone "VieNeu-TTS-v3-Turbo (Thử nghiệm)") or the SDK (Vieneu(mode="v3turbo")). The full v3 release is coming in the next few weeks.

[!IMPORTANT] 🚀 VieNeu-TTS-v2 is here! The full high-fidelity bilingual architecture is now available with:

• 10,000+ Hours of Data: Unmatched naturalness in both English and Vietnamese.
• Podcast & Dialogue Mode: Multi-speaker support with emotional nuances.
• Zero-shot Cloning: Clone any voice in 3-5 seconds across all v2 variants.

✨ Key Features

• 10,000+ Hours Training: Trained on a massive English-Vietnamese dataset for human-like prosody.
• Bilingual (En-Vi) Code-switching: Powered by sea-g2p for high-fidelity pronunciation and seamless transitions between Vietnamese and English.
• Podcast & Conversation Mode: Multi-speaker dialogue support with automatic character detection.
• Instant Voice Cloning: Clone any voice with just 3-5 seconds of reference audio.
• Ultra-Fast Performance: Optimized for GPU (LMDeploy) and CPU (GGUF/ONNX).
• Production-Ready: High-quality 24 kHz waveform generation, fully offline.

📌 Table of Contents

1. 🦜 Installation & Web UI
2. 📦 Using the Python SDK
3. 🐳 High-Quality Server (Standard Mode)
4. 🔬 Model Overview
5. 🚀 Roadmap
6. 🤝 Support & Contact
7. 📑 Citation

---

🦜 1. Installation & Web UI

Setup with uv (Recommended)

uv is the fastest way to manage dependencies.

# Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Linux/macOS:
curl -LsSf https://astral.sh/uv/install.sh | sh

1. Clone the Repo:

   git clone https://github.com/pnnbao97/VieNeu-TTS.git
   cd VieNeu-TTS
   

2. Install Dependencies:

   • Option 1: CPU & macOS (minimal, torch-free) — recommended for maximum speed — runs v3 Turbo via ONNX

     💡 No GPU required. Installs only the lightweight ONNX stack; v3 Turbo runs on CPU (48 kHz) with default voices, voice cloning and emotion cues. PyTorch is never installed.

     ⚡ For the fastest CPU inference, install with uv sync — not pip install. uv sync reproduces the locked environment that pins the optimized ONNX Runtime build, so you get maximum speed out of the box.

     🍎 macOS users: use this option too. For v3 Turbo the torch-free ONNX path on the CPU is faster than the MPS/PyTorch build (--group gpu), so prefer uv sync for top speed on Apple Silicon.

     uv sync
     

   • Option 2: GPU — v3 Turbo (PyTorch) + VieNeu-TTS v2 (GPU)

     💡 Requires a CUDA NVIDIA GPU (CUDA ≥ 12.8) or Apple Silicon MPS. NVIDIA Toolkit recommended. Adds the PyTorch stack so v3 Turbo runs on GPU and the v1 / v2 (GPU) models become available.

     uv sync --group gpu
     

3. Start the Web UI:

   uv run vieneu-web
   

   Access the UI at http://127.0.0.1:7860.

---

📦 2. Using the Python SDK (vieneu)

The vieneu SDK defaults to VieNeu-TTS v3 Turbo (48 kHz). The minimal install is torch-free: on CPU everything runs on ONNX Runtime (PyTorch is never imported), and on a CUDA machine it auto-switches to the PyTorch engine. Older models (v1/v2) are available via the [gpu] extra.

Quick Start

# Minimal, TORCH-FREE install — runs v3 Turbo on CPU via ONNX Runtime
pip install vieneu

from vieneu import Vieneu
from time import time

# Default = v3 Turbo. CPU → ONNX (torch-free); GPU → PyTorch (auto-detected).
tts = Vieneu()

text = f"""[cười] Trời ơi, cái giọng nó tự nhiên mà nó mượt mà dã man, nghe không khác gì người thật luôn. Giờ thì tha hồ mà quẩy content với cả kho giọng nói đa dạng, đủ mọi sắc thái biểu cảm. Mọi người bật loa lên rồi cùng trải nghiệm thử với mình nhé!"""

start_time = time()
# 1. Default voice (Bình An) — 48 kHz, no reference needed
audio = tts.infer(text)
tts.save(audio, "output.wav")
end_time = time()
print(f"Time taken: {end_time - start_time} seconds")
# 2. Built-in voices by name
for label, voice_id in tts.list_preset_voices():
    print(label, voice_id)
audio = tts.infer("Mình là Xuân Vĩnh nè!", voice="Xuân Vĩnh")
tts.save(audio, "output_Xuân Vĩnh.wav")
# # 3. Emotion / non-verbal cues — EXPERIMENTAL: [cười] [thở dài] [hắng giọng]
# audio = tts.infer("Nghe hay quá đi [cười]. Để mình nói tiếp [hắng giọng].", voice="Ngọc Linh")

# # 4. Instant voice cloning from a 3–5s reference clip
# audio = tts.infer("Đây là giọng được nhân bản tức thì.", ref_audio="my_voice.wav")

---

🐳 3. High-Quality Server (Standard Mode)

Deploy VieNeu-TTS as a high-performance API Server (powered by LMDeploy) with a single command.

1. Run with Docker (Recommended)

Requirement: NVIDIA Container Toolkit is required for GPU support.

Start the Server with a Public Tunnel (No port forwarding needed):

docker run --gpus all -p 23333:23333 -v huggingface_cache:/root/.cache/huggingface pnnbao/vieneu-tts:latest --tunnel

• Default: The server loads the VieNeu-TTS-v2 model for maximum quality.
• Tunneling: The Docker image includes a built-in bore tunnel. Check the container logs to find your public address (e.g., bore.pub:31631).

2. Using the SDK (Remote Mode)

Once the server is running, you can connect from anywhere (Colab, Web Apps, etc.) without loading heavy models locally.

Installation:

pip install "vieneu[gpu]"

Usage:

from vieneu import Vieneu
import os

# Configuration
REMOTE_API_BASE = 'http://your-server-ip:23333/v1'  # Or bore tunnel URL
REMOTE_MODEL_ID = "pnnbao-ump/VieNeu-TTS-v2"

# Initialization (LIGHTWEIGHT - only loads small codec locally)
# Default emotion is "natural" (conversational) - set emotion="storytelling" for storytelling mode
tts = Vieneu(mode='remote', api_base=REMOTE_API_BASE, model_name=REMOTE_MODEL_ID, emotion="natural")
os.makedirs("outputs", exist_ok=True)

# List remote voices
available_voices = tts.list_preset_voices()
for desc, name in available_voices:
    print(f"   - {desc} (ID: {name})")

# Use specific voice (dynamically select second voice)
if available_voices:
    _, my_voice_id = available_voices[1]
    voice_data = tts.get_preset_voice(my_voice_id)
    audio_spec = tts.infer(text="Chào bạn, tôi đang nói bằng giọng của bác sĩ Tuyên.", voice=voice_data)
    tts.save(audio_spec, f"outputs/remote_{my_voice_id}.wav")
    print(f"💾 Saved synthesis to: outputs/remote_{my_voice_id}.wav")

# Standard synthesis (uses default voice)
text_input = "Chế độ remote giúp tích hợp VieNeu vào ứng dụng Web hoặc App cực nhanh mà không cần GPU tại máy khách."
audio = tts.infer(text=text_input)
tts.save(audio, "outputs/remote_output.wav")
print("💾 Saved remote synthesis to: outputs/remote_output.wav")

# Zero-shot voice cloning (encodes audio locally, sends codes to server)
if os.path.exists("examples/audio_ref/example_ngoc_huyen.wav"):
    cloned_audio = tts.infer(
        text="Đây là giọng nói được clone và xử lý thông qua VieNeu Server.",
        ref_audio="examples/audio_ref/example_ngoc_huyen.wav",
        ref_text="Tác phẩm dự thi bảo đảm tính khoa học, tính đảng, tính chiến đấu, tính định hướng."
    )
    tts.save(cloned_audio, "outputs/remote_cloned_output.wav")
    print("💾 Saved remote cloned voice to: outputs/remote_cloned_output.wav")

For full implementation details, see: examples/main_remote.py

Voice Preset Specification (v1.0)

VieNeu-TTS uses the official vieneu.voice.presets specification to define reusable voice assets. Only voices.json files following this spec are guaranteed to be compatible with VieNeu-TTS SDK ≥ v1.x.

3. Advanced Configuration

Customize the server to run specific versions or your own fine-tuned models.

Run the 0.3B Model (Faster):

docker run --gpus all pnnbao/vieneu-tts:serve --model pnnbao-ump/VieNeu-TTS-0.3B --tunnel

Serve a Local Fine-tuned Model: If you have merged a LoRA adapter, mount your output directory to the container:

# Linux / macOS
docker run --gpus all \
  -v $(pwd)/finetune/output:/workspace/models \
  pnnbao/vieneu-tts:serve \
  --model /workspace/models/merged_model --tunnel

---

🔬 4. Model Overview

[!TIP] Use Turbo v2 for AI assistants, chatbots, and real-time edge applications where speed is critical. Note: It may have stability issues with very short phrases (< 5 words). Use GPU/Standard (VieNeu-TTS v1/v2) for maximum audio quality and high-fidelity voice cloning.

---

🚀 5. Roadmap

• VieNeu-TTS-v2: Full high-fidelity bilingual architecture with Podcast Mode and Voice Cloning.
• VieNeu-Codec: Optimized neural codec for Vietnamese (ONNX).
• Turbo Voice Cloning: Bringing instant cloning to the lightweight Turbo engine.
• VieNeu-TTS v3 Turbo (early access): New from-scratch 48 kHz architecture — built-in default voices (speaker tokens), experimental emotion cues, batched generation & multi-speaker conversation.
• VieNeu-TTS v3 (full release): Complete v3 with finalized quality, stable emotion control, more default voices & streaming server.
• Mobile SDK: Official support for Android/iOS deployment.

---

🤝 6. Support & Contact

• Hugging Face: pnnbao-ump
• Discord: Join our community
• Facebook: Pham Nguyen Ngoc Bao
• License: Apache 2.0 (Free to use).

---

📑 7. Citation

@misc{vieneutts2026,
  title        = {VieNeu-TTS-v2: Advanced Vietnamese Text-to-Speech with Podcast and Code-Switching Support},
  author       = {Pham Nguyen Ngoc Bao},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pnnbao-ump/VieNeu-TTS}}
}

---

🌟 Star History

---

🤝 Contributors

Thanks to all the amazing people who have contributed to this project!

---

🙏 Acknowledgements

This project uses neucodec (v1/v2) and MOSS-Audio-Tokenizer-Nano (v3 Turbo) for audio coding, and sea-g2p for text normalization and phonemization.

Made with ❤️ for the Vietnamese TTS community