MOSS-TTS OpenAI Compatible API

This repository provides an OpenAI-compatible Text-to-Speech (TTS) API for the MOSS-TTS 1.7B Local Transformer model. It includes optimizations for Windows and NVIDIA GPUs, featuring Flash Attention 2 support.

🚀 Performance & Benchmarks

The following benchmarks were conducted on a high-end laptop to establish a baseline for real-world usage.

Hardware Profile

• GPU: NVIDIA GeForce RTX 4070 Laptop GPU (8GB VRAM)
• VRAM Usage: ~4-5 GB (BF16)
• CPU: Intel Core i9 (13th Gen)
• OS: Windows 11
• Environment: Python 3.13.2 (Portable), CUDA 13.0

SDPA vs. Flash Attention 2

MOSS-TTS originally uses PyTorch's SDPA (Scaled Dot-Product Attention). By upgrading to Flash Attention 2, we achieved significant throughput improvements, especially on longer text strings.

[!TIP] 90-Second Budget: For real-time applications, we recommend keeping individual requests under 300 characters to stay within a 90-second response window.

🛠 Installation & Setup

Architecture Note

This implementation uses the MossTTSLocal architecture for the 1.7B model. It is significantly faster than the 8B "Delay" architecture while maintaining high emotional range.

Flash Attention 2 (Windows)

To achieve the performance metrics above, we utilized a specific pre-compiled wheel for Windows/Python 3.13:

• Wheel: flash_attn-2.8.3+cu130torch2.9.1cxx11abiTRUE-cp313-cp313-win_amd64.whl
• Source: Compatible with PyTorch 2.9.1 and CUDA 13.0.

Prerequisites

1. Python 3.13 (Required for the specific FA2 wheel).
2. CUDA 13.0+ Toolkit installed at the system level.

Quick Start

# Install dependencies
pip install -r requirements.txt

# Run the server
python server.py --port 8090

🎙 API Usage

The server adheres to the OpenAI /v1/audio/speech specification.

Example Request (PowerShell)

$body = @{
    model = "moss-tts"
    input = "Hello, I am a demon of the Fifth Circle."
    voice = "default"
} | ConvertTo-Json

Invoke-RestMethod -Uri "http://localhost:8090/v1/audio/speech" -Method Post -Body $body -ContentType "application/json" -OutFile "output.ogg"

📜 Voice Characteristics

• Default Voice: A high-quality female voice (similar to AWS Polly's Ivy).
• Zero-Shot Cloning: Support for 3-10 second reference clips is implemented in the underlying model logic and can be exposed via the voice parameter mapping.

---

Maintained by OpenMOSS community contributors.