Modern TTS

A unified, extensible, and future-proof Python toolkit for locally running state-of-the-art LLM-based Text-to-Speech (TTS) synthesis models.

---

✨ Features

• 🧩 25+ Models — MeloTTS, ChatTTS, CosyVoice, Fish Speech, Parler-TTS, XTTS, GPT-SoVITS, F5-TTS, Qwen3-TTS, GLM-TTS, Index-TTS, MaskGCT, and more
• 🔌 Plugin Architecture — Add new models with @register_model decorator
• 🚀 Hot-Swap — Switch models at runtime without restarting
• 🌍 Multi-Language — Chinese, English, Japanese, Korean, and more
• 🎯 Multi-Task — Speech synthesis, voice cloning, emotion control, style transfer, streaming
• 💻 Local-First — All inference on-device. No APIs. No data leaves your machine.
• 🐍 Modern Python — uv-native packaging, Pydantic configs, rich CLI
• 📦 Zero-Config for select models — GLM-TTS and Index-TTS automatically download their official code repositories on first use

---

📦 Installation

# Clone the repository
git clone https://github.com/vra/modern-tts.git
cd modern-tts

# Sync all dependencies (recommended)
uv sync --all-extras

# Or install specific extras only
uv sync --extra melotts --extra chattts --extra glm --extra index

# Or just core dependencies
uv sync

Python 3.10+ recommended. Some models (e.g. Index-TTS) require specific PyTorch / transformers versions—see per-model notes below.

---

🚀 Quick Start

from modern_tts import TTSPipeline

# Synthesize with MeloTTS
pipe = TTSPipeline("melotts-zh")
result = pipe("你好世界，这是语音合成测试。")
result.save("output.wav")

# Switch to ChatTTS for emotional speech
pipe.switch_model("chattts")
result = pipe("这是一个带有情感的语音合成。")
result.save("output_emotion.wav")

# Voice cloning with CosyVoice
pipe.switch_model("cosyvoice-300m")
result = pipe("这是克隆的声音。", task="clone", reference_audio="reference.wav")
result.save("cloned.wav")

# Zero-config voice cloning with GLM-TTS (auto-downloads code)
pipe.switch_model("glm-tts")
result = pipe("你好，这是 GLM-TTS 的语音克隆测试。", task="clone", reference_audio="ref.wav")
result.save("glm_cloned.wav")

# Zero-config voice cloning with Index-TTS (auto-downloads code)
pipe.switch_model("index-tts")
result = pipe("你好，这是 Index-TTS 的语音克隆测试。", task="clone", reference_audio="ref.wav")
result.save("index_cloned.wav")

---

🎙️ Supported Models

✅ Ready to use (loadable out-of-the-box)

ZS-VC = Zero-Shot Voice Cloning (requires a reference_audio sample).

⚠️ Requires manual setup

These models need you to manually clone their official repositories and/or download weights before use. Calling load() will raise a RuntimeError with setup instructions.

❌ Temporarily unavailable

---

📋 Changelog & API Changes

Latest

New Models

• glm-tts — LLM + Flow Matching zero-shot TTS (Zhipu AI). Merged previous glm-tts-nano-2512 and glm-tts-2512 into a single glm-tts model ID.
• index-tts — Industrial-level multilingual zero-shot voice cloning (IndexTeam).

Zero-Config Auto-Download

• GLM-TTS and Index-TTS no longer require manual environment variables (GLM_TTS_REPO_PATH, INDEX_TTS_REPO_PATH) or PYTHONPATH manipulation.
• On first use, the framework automatically:
  1. Clones the official repository to ~/.cache/modern-tts/repos/
  2. Injects the path into sys.path
  3. Proceeds with model loading
• You can still override the auto-download path via config.extra["glm_tts_repo_path"] / config.extra["index_tts_repo_path"] or the corresponding environment variables.

New Infrastructure Modules

• modern_tts.core.hf_hub — HuggingFace Hub download helpers (download_hf_model, get_hf_model_path) so custom-code adapters don't re-implement caching logic.
• modern_tts.core.repo_manager — Generic git repository auto-downloader (ensure_repo, inject_repo_path) used by adapters that depend on upstream code not on PyPI.

Base Class Improvements

• TextToAudioLLMModel.load() now raises a clear NotImplementedError when a subclass has not set PROCESSOR_CLS / MODEL_CLS, signaling that the subclass must override load() for custom loading logic.

Model ID Changes

---

🏗️ Architecture

Modern TTS is built on three layers:

1. TTSPipeline — Unified user API. Handles text normalization, task dispatch, model lifecycle.
2. TTSModel / TextToAudioLLMModel — Adapter layer. New models often need only 8 lines of config via TextToAudioLLMModel.
3. Backends — Transformers, vLLM, ONNX Runtime.

Adding a New Model

from modern_tts.core.audio_llm import TextToAudioLLMModel
from modern_tts.core.registry import register_model

@register_model("my-tts-1b")
class MyTTS1B(TextToAudioLLMModel):
    HF_PATH = "org/MyTTS-1B"
    PROCESSOR_CLS = "transformers.AutoTokenizer"
    MODEL_CLS = "transformers.AutoModelForTextToWaveform"
    SUPPORTED_LANGUAGES = {"zh", "en"}
    DEFAULT_SAMPLE_RATE = 24000

    @property
    def model_id(self) -> str:
        return "my-tts-1b"

That's it. The registry auto-discovers it at runtime.

---

🤝 Contributing

See Contributing Guide for development setup, code style, and PR checklist.

---

📄 License

Apache-2.0