Voxtral TTS Q4 GGUF

Q4_0 quantized weights for Voxtral 4B TTS in GGUF format. For use with voxtral-mini-realtime-rs.

Try the browser demo — runs entirely client-side via WASM + WebGPU.

Files

Model Details

• Base model: mistralai/Voxtral-4B-TTS-2603
• Quantization: Q4_0 (4-bit, 18 bytes per 32 elements)
• File size: 2.67 GB (vs ~8 GB BF16 original)
• Format: GGUF v3 (381 tensors)
• Inference: Burn ML framework with custom WGSL compute shaders

What is Quantized

Codec weights stored as F32 with pre-fused weight normalization.

Benchmarks

NVIDIA DGX Spark (GB10, LPDDR5x), "The quick brown fox jumps over the lazy dog":

Optimizations: batched CFG, fused QKV+gate/up projections, pre-allocated KV cache.

Usage

Native CLI

# Download
uv run --with huggingface_hub \
  hf download TrevorJS/voxtral-tts-q4-gguf voxtral-tts-q4.gguf --local-dir models

# Synthesize (unified voxtral CLI)
cargo run --release --features "wgpu,cli,hub" --bin voxtral -- \
  speak --text "Hello world" --voice casual_female --gguf models/voxtral-tts-q4.gguf

# Real-time with 3 Euler steps
cargo run --release --features "wgpu,cli,hub" --bin voxtral -- \
  speak --text "Hello world" --gguf models/voxtral-tts-q4.gguf --euler-steps 3

# List voices
cargo run --release --features "wgpu,cli,hub" --bin voxtral -- speak --list-voices

Browser (WASM + WebGPU)

Shards are pre-split for browser loading. The TTS demo loads them automatically.

For local dev:

wasm-pack build --target web --no-default-features --features wasm
bun serve.mjs  # serves shards from models/voxtral-tts-q4-shards/

Available Voices

20 presets across 9 languages:

Quantization Script

uv run --with safetensors --with torch --with numpy --with packaging \
  scripts/quantize_tts_gguf.py models/voxtral-tts/ -o voxtral-tts-q4.gguf

Source: scripts/quantize_tts_gguf.py

Related

• Code: TrevorS/voxtral-mini-realtime-rs
• ASR Model: TrevorJS/voxtral-mini-realtime-gguf
• ASR Demo: TrevorJS/voxtral-mini-realtime
• TTS Demo: TrevorJS/voxtral-4b-tts