Bench for local text-to-speech (TTS) models. Three lenses, on whatever hardware you put it on:

• Speed — cold + warm TTFA (time to first audio), RTFx (realtime speed; higher = faster than realtime), memory, on CPU / CUDA / Apple Silicon
• Listen — every model on every prompt, default voice + voice cloning, with inline audio players, so you can pick a model by ear
• Scores — objective metrics per model: UTMOS (naturalness), WER (intelligibility), SIM (cloning fidelity), scored over the bench prompts via seed-tts-eval-style ASR + speaker-verification. Sortable, with a Default/Cloning toggle.

An objective quality score (NAQ) was prototyped but isn't part of the bench — the v2 features didn't track subjective ranking closely enough to publish, so it was pulled and is being redesigned separately. The bench measures speed; quality is by-ear via the Listen lens.

---

▶ Demos

5uck1ess.github.io/tts-bench — listen to every model, no install. Three lenses:

• Listen — one consolidated gallery with an inline <audio> player for every model on every prompt, in default voice and voice cloning (each clone sits next to the reference it's imitating). Browse by prompt (compare all models on one sentence) or by model (audition one model across prompts); only one clip plays at a time. Audio is rig-independent, so each sample is sourced once from the highest-fidelity rig and tagged with where it came from. Quality, prosody, and artifacts are obvious in 5 seconds — benchmark tables can't show that.
• Speed — per-rig leaderboards (Ryzen 9 9950X3D + RTX 5090, Apple M4, Ryzen + RTX 3090) with cold/warm TTFA, RTFx, and memory, sortable. Pick the box you actually own.
• Scores — objective metrics per model (UTMOS naturalness, WER intelligibility, SIM cloning fidelity), with an interactive top-15 chart, sortable tables, and a Default/Cloning toggle. Human votes remain the preference ground truth; these are objective backstops.

Full per-rig reports (every model × prompt × device, plus by-prompt samples) are linked from the Archive.

---

🗳 Vote

Quality is subjective, so the ground truth is your ears. The companion TTS Voting Arena is a public, blind A/B listening test — two clips, no model names shown, pick the one that sounds better. No login, ~5 seconds a vote.

• Default voice — which model sounds more natural?
• Cloning — which clone better matches the reference voice?

Votes feed a live human-preference Elo leaderboard right there on the arena. This is where the "best sounding" and cloning calls above come from — every vote sharpens the ranking.

→ Vote now at the TTS Arena

---

Quick start

Requires uv and Python 3.11. ~10-15 min install. Disk for the full set is large: ~39 GB of per-model venvs in the repo, plus ~125 GB of model weights downloaded to your Hugging Face cache (~/.cache/huggingface, not the repo) — ~165 GB all-in. Individual models are far smaller, so installing a subset costs a fraction of that.

# Windows — everything, or just the models you want
.\install.ps1
.\install.ps1 kokoro,piper,miso
python bench.py

# macOS / Linux — everything, or just the models you want
./install.sh
./install.sh kokoro piper miso
python bench.py

Pass model names to install only those (names = the venvs/<name> slugs, which match the tables below — lowercase, e.g. kokoro, f5tts, chatterbox, miso). A few share one install: neutts covers NeuTTS Air + Nano, chatterbox both ChatterBox variants, vibevoice the 0.5B/1.5B, moss_tts both MOSS checkpoints, inflect both Inflect v2 sizes, qwentts the 1.7B Base + both 0.6B checkpoints, fish is Fish Speech 1.5. Add scoring (plus scoring_sim on Linux) for the objective-metrics venv. bench.py only runs models whose venv exists, so a partial install benches cleanly — install more models later by re-running with new names.

Interactive feel-test: python speak.py kokoro. One-shot A/B comparison: python compare.py "your phrase". See docs/architecture.md for the runner protocol and how to add a model.

---

TLDR (June 2026)

Fastest:

• CPU (Ryzen 9 9950X3D, Windows): Piper — 107ms warm TTFA, 59× RTFx
• CUDA (RTX 5090): Kokoro — 67ms warm TTFA, 104× RTFx
• CPU + MPS (Apple M4, 16 GB): Piper — 208ms warm TTFA, 32× RTFx

Best sounding: No objective ranking right now — the NAQ score is paused pending redesign. Open the Demos site and use the Listen lens.

Best cloning — blind A/B votes (these measure voice-match preference, not intelligibility):

• 1. OmniVoice — top on voice/accent match (24-1-3), but it can garble or drop words; a timbre-focused A/B vote doesn't penalize that, so read this as "best voice match," not "best overall clone." Audition it first — objective WER (the new Scores lens) is meant to catch exactly this gap.
• 2. Echo-TTS — near-tied #1 (21-1-6), clean 44.1 kHz
• 3. IndexTTS-2 — third (16-2-5), accent held

→ full per-rig results · → full cloning ranking

---

Models tracked (71)

Predefined voices

Zero-shot cloning

Expressive column — what explicit emotion/delivery control the model offers: tags = inline cues in the text itself ((laughs), [sigh], <laugh>); desc = natural-language style/emotion instructions; knob = numeric or preset parameter (exaggeration, style enum, pitch/speed); emo-ref = emotion conditioned on a separate reference clip or emotion vector; — = none (for cloning models, expression simply follows the reference clip). * = caveat applies. Exact syntax, sources, and caveats per model: docs/expressive-control.md. Note the bench feeds every model the same plain prompts for fairness, so these features are not exercised in any score.

Full per-model gotchas + license details: docs/known-issues.md. Models considered but excluded: docs/considered.md.

Predefined vs Cloning. Predefined models have fixed/selectable speaker voices baked into the weights — they speak with no reference needed. Cloning (zero-shot) models have no voice of their own: they synthesize whatever voice you hand them as a reference clip at inference. Given no reference, a pure zero-shot model falls back to a bundled sample (this bench uses chris_hemsworth_15s.wav), so its "default voice" is just a clone of that clip. A few models do both (e.g. Voxtral has 20 presets and cloning).

Newly added, not yet benched. qwentts_06b_custom (Qwen3-TTS 0.6B CustomVoice, 9 preset timbres) is registered and smoke-tested but carries no speed or objective numbers yet — the bench + UTMOS/WER/SIM scoring pass runs on the Linux rig. The 0.6B Base checkpoint (qwentts_06b) is registered too, but like both 1.7B Base rows it is reference-only — Base has no model-native preset voice, so its no-reference run clones the house chris_hemsworth_15s.wav and it appears under Cloning, never Default. Its cloning lens is disabled by an intermittent decode runaway (identical input yielded 12.9 s of audio on one call and 655.3 s on the next) — see docs/known-issues.md.

Rig availability: Voxtral is Mac (MLX, preset-voice only) + Linux (vLLM, cloning); Fish S2-Pro / MetaVoice / Step-Audio-EditX / Higgs Audio v3 / dots.tts / Zonos2 / Orpheus / CosyVoice 3 are Linux-only (CUDA) — Higgs v3 is the one server-backed model (it runs via a Docker sgl-omni HTTP server, not an in-process load), dots.tts and Zonos2 require Linux-only dependencies/compiled CUDA kernels, and Orpheus (vLLM) + CosyVoice 3 (cu121 / torch 2.3.1) have no Blackwell-compatible Windows path; Echo-TTS and DramaBox are Windows + Linux (CUDA-only, no CPU/MPS; DramaBox needs ~18 GB VRAM). The rest run on Windows + Linux CUDA, most on CPU/MPS too. Per-rig speed + samples on the Demos site.

---

Voice cloning

48 of the 71 tracked models can clone a voice from a reference clip. Three reference formats supported (wav only / wav + transcript / HF-gated wav). Drop a reference into reference/, then python bench.py --reference reference/myvoice.wav.

Reference-format docs + the blind-vote cloning ranking (28 cloning models, human-preference A/B, frozen at 397 votes; the live arena board now has 738 cloning votes): docs/cloning.md.

---

Test hardware

If you reproduce on different hardware, file an issue or PR with your results and we'll add a column.

---

Docs

• Full results tables — per-rig, per-prompt, per-model
• Cloning ranking — reference formats + blind-vote ranking (28 cloning models, human-preference A/B)
• Architecture — bench design, runner protocol, adding a model
• Expressive control — which models take emotion tags / style prompts / knobs, with exact syntax
• Known issues — per-model gotchas + per-license table
• Considered but skipped — models evaluated and excluded
• Tasks & pending work — open issues, planned features
• Methodology — what's measured, why cold + warm, why reproducible

---

License

MIT for the bench code in this repo. Each TTS model has its own license — see docs/known-issues.md for the full per-model table.

---

Support

If this bench saved you a weekend of writing your own: