qwen3-tts-candle

Pure Rust inference implementation for Qwen3-TTS (Alibaba's text-to-speech model), powered by Candle ML framework.

Features

• Pure Rust — no Python runtime or C++ dependencies required
• Multi-platform GPU — Apple Silicon (Metal), NVIDIA CUDA, Flash Attention 2
• CPU fallback — with optional MKL / Accelerate BLAS acceleration
• Voice cloning — ECAPA-TDNN speaker encoder extracts voice embeddings from reference audio
• Streaming synthesis — binary frame protocol delivers ~0.8s audio chunks incrementally
• HTTP server — production-ready Axum server with health checks, prompt caching, and request tracing
• Voice Recorder UI — React + Bun full-stack app for recording, managing speaker profiles, and voice clone jobs
• Model auto-detection — automatically identifies model variant (0.6B / 1.7B, Base / CustomVoice / VoiceDesign) from config.json
• HuggingFace Hub — downloads model weights automatically if not found locally

Architecture

Three-stage TTS pipeline:

Text ──→ [TalkerModel] ──→ semantic tokens (1 per frame)
              │
              ▼
         [CodePredictor] ──→ 15 acoustic codes per semantic token
              │
              ▼
         [Decoder12Hz] ──→ 24kHz mono WAV audio

Supported Models

Quick Start

Prerequisites

• Rust (1.75+) — rustup.rs
• Bun (optional, for Voice Recorder UI) — bun.sh

One-command startup (server + UI)

git clone https://github.com/HeiSir2014/qwen3-tts-candle.git
cd qwen3-tts-candle
bash scripts/start.sh

This will:

1. Build the Rust TTS server in release mode
2. Install Bun dependencies for the Voice Recorder UI
3. Start the Bun server (auto-launches the Rust TTS server)

Once running:

• TTS Server: http://localhost:5001
• Voice Recorder UI: https://localhost:3000

Model weights are downloaded automatically from HuggingFace Hub on first run.

Run TTS server only

# Build
cargo build --release -p tts-server

# Run (model auto-downloads from HuggingFace Hub)
./target/release/tts-server

# Or specify options
./target/release/tts-server \
  --port 5001 \
  --model-path Qwen/Qwen3-TTS-12Hz-0.6B-Base \
  --device auto

CLI flags:

Use as a Rust library

Add to your Cargo.toml:

[dependencies]
qwen3-tts = { path = "crates/qwen3-tts", features = ["metal"] }

use qwen3_tts::{Qwen3TTS, SynthesisOptions, auto_device};

// Load model (auto-detected from config.json)
let device = auto_device()?;
let model = Qwen3TTS::from_pretrained("path/to/model", device)?;

// Synthesize speech
let audio = model.synthesize("Hello, world!", None)?;
audio.save("output.wav")?;

// With custom options
let options = SynthesisOptions {
    temperature: 0.8,
    top_k: 30,
    ..Default::default()
};
let audio = model.synthesize("Custom settings!", Some(options))?;

API Endpoints

Streaming protocol

/clone-stream returns chunked binary frames (big-endian u32):

[chunkIndex: u32] [marker: u32] [sampleRate: u32] [dataLen: u32] [data bytes]

Platform Support

Build with a specific backend:

# Apple Silicon (default for tts-server)
cargo build --release -p tts-server

# NVIDIA GPU
cargo build --release -p tts-server --no-default-features --features cuda

# CPU only
cargo build --release -p tts-server --no-default-features --features cpu

Project Structure

qwen3-tts-candle/
├── crates/qwen3-tts/       # Core inference library
│   └── src/
│       ├── lib.rs           # Public API: Qwen3TTS, StreamingSession
│       ├── models/          # TalkerModel, CodePredictor, Decoder12Hz, SpeakerEncoder
│       ├── generation/      # Sampling (top-k/p, temperature, repetition penalty)
│       ├── audio/           # WAV I/O, mel spectrogram, resampling
│       └── tokenizer/       # Text tokenizer (Qwen2 vocab, 151K tokens)
├── tts-server/              # Axum HTTP server
│   └── src/
│       ├── main.rs          # CLI entry point
│       ├── server.rs        # Router + middleware
│       ├── routes.rs        # API handlers
│       ├── streaming.rs     # Binary frame streaming protocol
│       └── embedding.rs     # Speaker embedding serialization
├── voice-recorder/          # Bun + React voice clone studio
│   ├── src/                 # Bun backend (HTTPS, SSE, Rust bridge)
│   └── ui/                  # React SPA (profiles, recording, clone lab)
└── scripts/
    ├── start.sh             # One-command build + run
    └── convert_to_f16.py    # BF16 → F16 weight conversion

BF16 → F16 Weight Conversion

To halve model disk size / VRAM usage:

pip install torch safetensors huggingface_hub
python scripts/convert_to_f16.py --keep-decoder-f32

License

This project provides a Rust inference implementation for the Qwen3-TTS model. Please refer to Qwen3-TTS for model license terms.