TTS-STT Platform

A self-hosted, Docker-based platform for text-to-speech synthesis, speech-to-text transcription, custom voice training, and voice cloning. German first: every default that has a language defaults to German.

Services

Only the Frontend is meant to be reached over the network: it proxies every backend call, the live microphone WebSocket included. The backend ports are published on 127.0.0.1 only (for curl, benchmarks and scripts on the host itself), because the backends have no authentication. Set BACKEND_BIND_ADDR=0.0.0.0 to reach them from other machines, or FRONTEND_BIND_ADDR=127.0.0.1 to keep the UI local behind a reverse proxy.

Quick Start

The default stack expects an NVIDIA GPU (the GPU services reserve one device); without one, use the CPU-capable services shown below.

cp .env.example .env                        # every setting is documented there; the defaults work
docker compose --profile all up -d          # default full stack: UI, Piper, Whisper, Qwen3-ASR, Qwen3-TTS (NVIDIA GPU)

# No NVIDIA GPU: CPU-capable services only
docker compose --profile piper-tts --profile frontend up -d            # TTS and the UI
ENABLE_WHISPER_CPP=true docker compose --profile whisper-cpp --profile frontend up -d  # whisper.cpp surfaced in UI

# Optional extras (heavy images, off by default; the UI also needs the matching ENABLE_* flag)
ENABLE_WHISPER_CPP=true docker compose --profile all --profile whisper-cpp up -d       # plus whisper.cpp
ENABLE_CHATTERBOX_TTS=true docker compose --profile all --profile chatterbox-tts up -d # plus Chatterbox
ENABLE_MAGPIE_TTS=true docker compose --profile all --profile magpie-tts up -d         # plus NVIDIA Magpie

# Check status
docker compose ps
docker compose logs -f
curl -fsS http://127.0.0.1:5001/ready       # is the Whisper model loaded? 503 with a reason until it is

# Stop everything, whichever profiles started it; the model caches stay. Every service belongs
# to a profile, so a bare `docker compose down` stops nothing. Never add -v to this one: with
# --profile "*" it deletes every model cache. (Or name the profiles you started, as for up.)
docker compose --profile "*" down

Open http://localhost:3000 for the web interface. The first start downloads the models in the background (several GB), so containers report healthy long before they can transcribe or speak; the UI shows each backend's state, and GET /ready on a backend says whether it can serve a request yet.

The gear in the UI opens Settings (/settings): host names, the reverse-proxy URL, API access and named API keys, which engines are offered, the default engines and the gateway limits, applied at once without a restart and kept in ./settings (SETTINGS_DIR). A saved value wins over .env, setting by setting. Saving needs an admin key: API_KEY, or on a stack without one a key created with a one-time code that only docker compose logs frontend-service shows.

whisper-cpp, Parakeet, Canary, Chatterbox and Magpie are opt-in. Starting a profile alone is not enough for the service to appear in the frontend; offer it under Settings -> Engines, or set the matching ENABLE_WHISPER_CPP, ENABLE_PARAKEET_ASR, ENABLE_CANARY_ASR, ENABLE_CHATTERBOX_TTS or ENABLE_MAGPIE_TTS=true for the frontend service as well.

Images are pulled from GHCR as ghcr.io/steffenhebestreit/tts-stt-*:${IMAGE_TAG} (latest by default, which follows the newest release or master build). Pin IMAGE_TAG=X.Y.Z where a deployment has to be reproducible, and PULL_POLICY=always together with latest to fetch a moved tag on up.

Developer Docs

• docs/api.md - the API: /v1 (OpenAI-compatible), access control, limits, readiness, errors
• docs/provider-contracts.md - provider registry schema and API contracts used by the frontend
• docs/developer-roadmap.md - planned architecture and migration steps for provider-generalization, with the current status
• docs/device-deployment-matrix.md - which services and models to run on which device, with ready-made presets in deploy/profiles/
• docs/model-and-optimisation-research-2026-08.md - model and optimisation decisions, with an implementation-status addendum
• benchmarks/README.md - measure German WER and latency of a backend on your own audio, and compare Whisper, Parakeet, Qwen3-ASR and a German fine-tune

---

GPU Setup

Different services support different GPU backends. Choose based on your hardware and host OS.

Which backend does each service use?

The CUDA images are built for Blackwell (RTX 50 series, sm_120) as well as older cards: CUDA 12.8 and PyTorch cu128 wheels, so the NVIDIA driver must report CUDA 12.8 or newer (R570+).

Which GPU works in which environment?

ROCm requires /dev/kfd which is only available on native Linux (bare metal or passthrough VM). Vulkan requires /dev/dri which Docker Desktop's internal VM does not expose. It works with Docker Engine running directly inside WSL2.

---

NVIDIA CUDA

Works on: native Linux · WSL2 + Docker Engine · WSL2 + Docker Desktop · Docker Desktop (Windows)

Prerequisites:

• NVIDIA drivers installed on the host
• NVIDIA Container Toolkit (nvidia-docker2 / nvidia-container-toolkit)

Verify:

nvidia-smi                             # host check
docker run --rm --gpus all nvidia/cuda:12-base nvidia-smi  # container check

Run (default — no overlay needed):

docker compose --profile all up -d
# or specific services:
docker compose --profile stt up -d
docker compose --profile qwen3-asr up -d
docker compose --profile training up -d
ENABLE_WHISPER_CPP=true docker compose --profile all --profile whisper-cpp up -d

The base docker-compose.yml already includes NVIDIA device reservations (deploy.resources.reservations.devices).

WSL2 + Docker Desktop: Enable GPU support in Docker Desktop → Settings → Resources → GPU. NVIDIA's WSL2 driver handles CUDA passthrough automatically.

---

AMD ROCm

Works on: native Linux only

Prerequisites (native Linux):

• ROCm 6.2+ installed
• User added to video and render groups: sudo usermod -aG video,render $USER
• Verify: rocminfo | grep -i "gfx\|gpu"

Run:

docker compose -f docker-compose.yml -f docker-compose.rocm.yml --profile all up -d

# Single service:
docker compose -f docker-compose.yml -f docker-compose.rocm.yml --profile qwen3-asr up -d

The ROCm overlay (docker-compose.rocm.yml) replaces CUDA-based images with ROCm variants, removes NVIDIA device reservations, and mounts /dev/kfd + /dev/dri.

Strix Halo (gfx1151 / RDNA 3.5 / Radeon 8060S): gfx1151 is supported by current ROCm. The overlay defaults HSA_OVERRIDE_GFX_VERSION=11.0.0 only because the Dockerfile.rocm images pin the rocm6.2 PyTorch wheel index, whose wheels carry no gfx1151 kernels; rocm7.0+ wheels do, so clear the override when those base images are bumped. PYTORCH_ALLOC_CONF=expandable_segments:True is set automatically for the unified LPDDR5X pool. See docker-compose.rocm.yml for the full gfx target table.

WSL2 limitation: /dev/kfd is not exposed by Docker Desktop's VM, nor by the WSL2 GPU-PV driver. ROCm requires bare-metal Linux or a Linux VM with direct PCIe GPU passthrough.

---

Vulkan (whisper-cpp only)

Works on: native Linux · WSL2 + Docker Engine (not Docker Desktop)

Vulkan is supported only by whisper-cpp (GGML Vulkan backend). It works on any Vulkan 1.2+ GPU — AMD RDNA, Intel Arc, NVIDIA — without needing CUDA or ROCm drivers.

Why not for Python services? PyTorch and CTranslate2 (used by stt-service, qwen3-asr, etc.) have no Vulkan backend.

Prerequisites:

• GPU with Vulkan 1.2+ support
• /dev/dri accessible from the container (see environment notes below)
• Verify: vulkaninfo --summary (install vulkan-tools if needed)

Run:

docker compose -f docker-compose.yml -f docker-compose.vulkan.yml --profile whisper-cpp up -d

The Vulkan overlay (docker-compose.vulkan.yml) replaces the CPU Dockerfile with Dockerfile.vulkan (compiled with -DGGML_VULKAN=1), mounts /dev/dri, and sets GGML_VULKAN_DEVICE=0.

Strix Halo (gfx1151 / Radeon 8060S): No GFX version override needed — Vulkan requires no per-target kernel compilation, which is what makes it the portable AMD path. Set GGML_VULKAN_DEVICE=1 if you have a discrete GPU and want device index 1 instead.

Vulkan on WSL2 with Docker Engine (no Docker Desktop)

WSL2's GPU-PV driver exposes the GPU as /dev/dri/renderD128 inside the WSL2 instance. If Docker Engine is installed directly in WSL2 (not Docker Desktop), containers can access it:

# Inside your WSL2 Ubuntu distro:
sudo apt-get install docker.io
sudo usermod -aG docker $USER
newgrp docker

# Verify GPU is visible:
ls /dev/dri/          # should show card0 and renderD128
vulkaninfo --summary  # should list your GPU

# Then run the Vulkan compose from WSL2:
docker compose -f docker-compose.yml -f docker-compose.vulkan.yml --profile whisper-cpp up -d

Docker Desktop on Windows: Docker Desktop runs containers inside its own LinuxKit VM, which does not expose /dev/dri. The Vulkan overlay will fail with "no such file or directory" in this environment. Use the CPU build instead (default), or switch to Docker Engine in WSL2.

---

CPU (no GPU)

Works on: all environments — native Linux, WSL2, Docker Desktop, Windows

No overlay needed. All services fall back to CPU automatically when no GPU is detected. Performance varies:

Use smaller models (WHISPER_MODEL_SIZE=small, WHISPER_MODEL=small) to improve CPU throughput.

---

Features

Text-to-Speech (PiperTTS)

• 40+ pre-trained voices across English, German, French, Spanish, Italian, Dutch
• Intelligent voice selection by language, quality, and gender
• Custom trained model support (ONNX format)
• Adjustable speech speed

Speech-to-Text — faster-whisper (stt-service)

• Multiple model sizes: tiny, base, small, medium, large-v3 (all multilingual)
• distil-large-v3 available for English-only use cases (fastest) — do not use for non-English
• Configurable VAD filter (vad_filter, vad_threshold) — disable if short clips are being rejected
• Streaming transcription via SSE (/transcribe-stream)
• Language detection (/detect_language)
• /health returns multilingual flag to avoid accidentally using an English-only model
• CUDA/ROCm/CPU; set FORCE_ACCELERATION=cuda|rocm|cpu

Speech-to-Text — whisper-cpp

• C++ implementation — lower memory footprint than Python/PyTorch stack
• GGUF quantized models (smaller downloads, faster inference)
• OpenAI-compatible /v1/audio/transcriptions endpoint
• CPU + Vulkan GPU backends; no CUDA or ROCm required
• Model downloaded automatically from HuggingFace on first start

Voice Training (Piper Training)

• VITS neural network architecture
• Automatic audio preprocessing and transcription via STT service
• FP32 training with automatic batch size adjustment on OOM
• Checkpoint saving and recovery after each epoch
• ONNX model export for PiperTTS deployment

Voice Cloning (Qwen3-TTS)

• Upload a short voice sample (3–10 seconds) and generate speech in that voice
• Cross-lingual cloning across 13 languages

Speech Recognition (Qwen3-ASR)

• Fast multilingual speech recognition via Qwen3-ASR-1.7B
• Long recordings are cut into pieces at the quietest point near QWEN3_ASR_CHUNK_S seconds (30 by default), so nothing is silently truncated
• Requires GPU for practical use; CPU inference is very slow (~60s per minute of audio vs ~2s on GPU)

Speech Recognition — Parakeet and Canary (NeMo, optional)

• Parakeet (parakeet-tdt-0.6b-v3, 25 European languages incl. German) and Canary (canary-180m-flash, en/de/es/fr with punctuation)
• Images default to NeMo 3.0.x on torch 2.11 (cu128), with a one-argument rollback to NeMo 2.x (NEMO_TOOLKIT_SPEC, or the prebuilt -nemo2 images via NEMO_IMAGE_SUFFIX=-nemo2). The rollback is for these two only: Magpie needs NeMo 3 and has its own MAGPIE_NEMO_TOOLKIT_SPEC
• A local .nemo checkpoint (for example a German fine-tune) can be used through PARAKEET_ASR_MODEL / CANARY_ASR_MODEL
• Opt-in: profile parakeet-asr / canary-asr plus ENABLE_PARAKEET_ASR / ENABLE_CANARY_ASR on the frontend

Text-to-Speech — Chatterbox (optional)

• Multilingual TTS and voice cloning (MIT licence; every file carries a PerTh watermark)
• Long texts are generated in chunks (up to CHATTERBOX_MAX_TEXT_CHARS, 5000, then HTTP 413) and can be streamed sentence by sentence (/tts-stream)
• The image installs the multilingual v3 checkpoint's code from a pinned GitHub commit; the PyPI release has no v3
• Opt-in: profile chatterbox-tts plus ENABLE_CHATTERBOX_TTS on the frontend

Text-to-Speech — Magpie (NVIDIA, optional)

• NVIDIA's Magpie-TTS Multilingual (magpie_tts_multilingual_357m) through NeMo 3: five built-in voices (Aria, Jason, John, Leo, Sofia) in German, English, Spanish, French, Japanese and Chinese. No voice cloning and no streaming. NVIDIA Open Model License, a custom license: read it before relying on it
• Digits, dates and times are spoken (text normalization is on; without it the model silently drops every number), though German numbers written with thousands dots ("1.200") are read digit by digit; see the Magpie README. A language it cannot speak is a 400, never English with a 200
• About 4.2 GB of VRAM, about 7 GB of host RAM and a first-start download of about 2.5 GB. The first request in a language other than German waits 18 to 48 s while its text normalizer is built (MAGPIE_WARM_LANGUAGES builds it during the first model load instead). Normalizers stay in host RAM across idle unloads (a few hundred MB per language, no VRAM), so a reload after TTS_MODEL_TTL repeats neither that wait nor the warm-up; /ready answers 503 loading during the first load only
• Opt-in: profile magpie-tts plus ENABLE_MAGPIE_TTS on the frontend. Details: magpie-tts-service/README.md

Model memory

Every GPU model is released after MODEL_TTL seconds without use (300 by default; per-service STT_MODEL_TTL, ASR_MODEL_TTL, TTS_MODEL_TTL override it, -1 keeps a model loaded) and reloads on the next request, so several multi-GB models can share one card. POST /api/providers/<id>/unload frees one immediately.

---

API Endpoints

The endpoints below are the backends' own (ports as in the service table, published on 127.0.0.1 by default). Use the gateway on port 3000 for anything beyond a local check: its OpenAI-compatible /v1 surface and the /api/* routes are described in docs/api.md, and each spec is served at /static/openapi/{gateway,stt,pipertts,training}.json.

Every model service has GET /health (liveness: 200 while a model loads or is idle-unloaded) and GET /ready (readiness: 503 while the first load runs or after it failed, with the reason; it never loads a model). Uploads and recordings beyond a service's limit answer 413.

PiperTTS (port 5000)

Adding a Piper voice: copy its .onnx and .onnx.json files into <models dir>/default. A fresh install is seeded with the voices baked into the image by the one-shot piper-voices-seed service; PIPER_VOICES (a build argument) chooses which voices a custom image ships.

STT — faster-whisper (port 5001)

Use a multilingual model (medium, large-v3) for non-English languages. distil-large-v3 is English-only and silently outputs English regardless of the language parameter.

STT — whisper-cpp (port 5003)

This backend is optional and is only shown in the frontend when ENABLE_WHISPER_CPP=true is set for frontend-service.

Piper Training (port 8080)

Qwen3-TTS (port 5004)

/tts (built-in speakers) needs a CustomVoice model variant and answers 409 on a Base model; the Base models clone voices (/clone, /voices/save). No language means German (QWEN3_DEFAULT_LANGUAGE); an unsupported one is a 400.

Qwen3-ASR (port 5002)

Parakeet-ASR (port 5005) and Canary-ASR (port 5006)

Chatterbox-TTS (port 5007)

Magpie-TTS (port 5008)

---

Configuration

Copy .env.example to .env and adjust as needed. The file documents every variable (over 130), including the per-service limits; the ones most people touch:

---

Project Structure

tts-stt-playground/
├── docker-compose.yml           # Base stack — NVIDIA CUDA or CPU
├── docker-compose.rocm.yml      # AMD ROCm overlay (native Linux only)
├── docker-compose.vulkan.yml    # Vulkan overlay for whisper-cpp (Linux / WSL2+Docker Engine)
├── docker-compose.arm64.yml     # ARM64 / no-GPU overlay (RK3588S): frontend, Piper, whisper-cpp
├── docker-compose.dev.yml       # Bind-mounts service source for local development
├── docker-compose.truenas*.yml  # TrueNAS SCALE: Custom App YAML and the build-from-source overlay
├── .env.example                 # Every setting, documented
├── deploy/profiles/             # Ready-made .env presets per device (RK3588S, 5060 Ti, 4080, Strix Halo)
├── frontend-service/            # Web UI + API gateway (FastAPI + Jinja2)
├── piper-tts-service/           # PiperTTS synthesis
├── piper-training-service/      # VITS training pipeline
├── stt-service/                 # faster-whisper STT (Python, CUDA/ROCm/CPU)
│   ├── app.py
│   ├── Dockerfile               # CUDA / CPU
│   └── Dockerfile.rocm          # AMD ROCm
├── whisper-cpp-service/         # whisper.cpp STT (C++, Vulkan/CPU)
│   ├── entrypoint.sh            # Downloads GGUF model from HuggingFace on first start
│   ├── Dockerfile               # CPU build
│   └── Dockerfile.vulkan        # Vulkan GPU build
├── qwen3-tts-service/           # Qwen3 TTS + voice cloning
├── qwen3-asr-service/           # Qwen3 ASR
├── parakeet-asr-service/        # Parakeet STT (NeMo), optional
├── canary-asr-service/          # Canary STT (NeMo), optional
├── chatterbox-tts-service/      # Chatterbox TTS + cloning, optional
├── magpie-tts-service/          # Magpie TTS (NVIDIA NeMo), optional
│                                #   each service dir: app.py, Dockerfile (CUDA), Dockerfile.rocm where one exists
├── benchmarks/                  # German ASR evaluation (WER, latency, significance test)
├── scripts/                     # sync_openapi.py, plan_publish.py, check_no_skips.py, scripts/truenas/*
├── tests/                       # pytest suite (unit, config-drift and, opt-in, live-stack tests)
├── docs/                        # API reference, provider contracts, device matrix, TrueNAS guide, research
├── truenas/                     # TrueNAS custom catalog scaffold
├── settings/                    # What the web UI's Settings page saves (gitignored)
└── models/                      # Shared model storage (gitignored)

---

Troubleshooting

Health checks

/health says the process is up; /ready says it can serve a request (503 with a reason while the first model is downloading or loading, or when the load failed). Run these on the Docker host: the backend ports are bound to 127.0.0.1 unless BACKEND_BIND_ADDR says otherwise.

curl http://localhost:5000/ready    # PiperTTS (503 if no voice is installed)
curl http://localhost:5001/ready    # STT faster-whisper
curl http://localhost:5002/ready    # Qwen3-ASR
curl http://localhost:5004/ready    # Qwen3-TTS
curl http://localhost:5005/ready    # Parakeet-ASR (optional)
curl http://localhost:5006/ready    # Canary-ASR (optional)
curl http://localhost:5007/ready    # Chatterbox-TTS (optional)
curl http://localhost:5008/ready    # Magpie-TTS (optional)
curl http://localhost:8080/ready    # Piper Training (names unwritable directories, a wrong PyTorch build, low disk)
curl http://localhost:5003/         # whisper-cpp (any HTTP response = up)
curl http://localhost:3000/health   # Frontend
curl http://localhost:3000/api/health   # every backend's state, as the UI shows it

Replace /ready with /health for the liveness view (what the Docker healthchecks use).

Common issues

CUDA not available in container

# Verify NVIDIA Container Toolkit is installed
docker run --rm --gpus all nvidia/cuda:12-base nvidia-smi
# If this fails: install nvidia-container-toolkit and restart Docker

ROCm: "No such file or directory: /dev/kfd" ROCm requires native Linux. Docker Desktop (WSL2 or Windows) does not expose /dev/kfd. Run on bare-metal Linux.

Vulkan: "no such file or directory: /dev/dri" Docker Desktop's VM does not expose /dev/dri. Options:

• Use the CPU build (default, no overlay)
• Install Docker Engine directly inside WSL2 (not Docker Desktop)
• Run on native Linux

Vulkan: "no suitable Vulkan device found"

# Check inside the container:
docker exec <container> vulkaninfo --summary
# If empty: the GPU driver is not visible. Verify /dev/dri is mounted and
# the container user has access (render/video groups).

STT translating instead of transcribing The distil-large-v3 model is English-only — it outputs English regardless of the language parameter. Switch to large-v3 or medium for multilingual transcription:

WHISPER_MODEL_SIZE=large-v3   # for stt-service
WHISPER_MODEL=large-v3        # for whisper-cpp

No speech detected / audio rejected by VAD The Silero VAD filter can be too aggressive on short clips or compressed audio (webm/opus). Disable or tune it:

POST /transcribe
  vad_filter=false          # disable completely
  vad_threshold=0.3         # lower threshold (default: 0.5)
  no_speech_threshold=0.95  # raise silence tolerance (default: 0.6)

VAD parameter defaults (applied to all /transcribe and /transcribe-stream requests):

For browser recordings (webm/opus), use no_speech_threshold=0.95 to avoid false silence detection.

Slow first start Models are downloaded on first launch, in the background: containers are healthy before they can serve, and /ready (or the UI's status indicators) shows when a backend is done. Sizes:

• Whisper large-v3-turbo: ~1.6 GB (large-v3: ~3 GB)
• Qwen3-ASR-1.7B: ~3.5 GB
• Qwen3-TTS-0.6B: ~2.5 GB (the 1.7B variant: ~4.5 GB)

"Backend unreachable" from another machine Backend ports are published on 127.0.0.1 only. Use the UI or the API on port 3000, which proxies every backend; set BACKEND_BIND_ADDR=0.0.0.0 only on a network you trust.

Every request answers 403 Host ... is not allowed (host_not_allowed) The gateway only answers to names that cannot be re-pointed by a third party: IP addresses, localhost, *.local, *.lan, *.internal, *.home.arpa and single-label names. A real domain in front of a reverse proxy, or a Tailscale MagicDNS name, has to be listed: open http://<ip>:3000/settings by IP address and add it under Access (it works on the next request), or set TRUSTED_HOSTS=voice.example.com.

Every button in the UI answers 403 behind a reverse proxy The proxy rewrites the Host header, so the UI's requests look cross-origin. Add the public URL under Settings -> Access, or to TRUSTED_ORIGINS (and believe X-Forwarded-Host, TRUST_PROXY_HEADERS=true, only if the proxy overwrites it). A proxy that passes Host through only needs its name as a host name.

The UI asks for an API key, or answers 401 API_KEY is set on the gateway, or the Settings page requires a key. The UI prompts once per browser tab and keeps the key in sessionStorage; scripts send Authorization: Bearer <key>. There is no exemption for the UI.

The Settings page saves nothing, or the admin key is lost Without API_KEY and before the first claim nobody may change settings: ask the page for a one-time code and read it in docker compose logs frontend-service (the same recovers a lost admin key). A saved value that broke something: Restore, or Back to the app YAML, under History & recovery, or delete settings/gateway.json (.env applies from the next request). ENABLE_SETTINGS_UI=false makes the gateway ignore what the page saved.

503 with Retry-After The service is at its limit rather than broken: too many uploads or ffmpeg conversions at the gateway (MAX_CONCURRENT_UPLOADS, MAX_CONCURRENT_FFMPEG), or a full or timed-out queue at a model backend (ASR_MAX_QUEUE, TTS_MAX_QUEUE, *_QUEUE_TIMEOUT_S). Retry after the delay; raise the limit only if the hardware has room.

Out of memory

• Use a smaller model (small, medium)
• Lower the idle TTL (MODEL_TTL) so idle models give their memory back, or free one now with POST /api/providers/<id>/unload
• For Qwen3 on ROCm/APU: PYTORCH_HIP_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512 (set automatically in ROCm overlay)
• For training: reduce batch size

Port conflicts Default ports: 3000, 5000-5008, 8080. Override with env vars: FRONTEND_PORT, PIPER_TTS_PORT, STT_PORT, QWEN3_ASR_PORT, WHISPER_CPP_PORT, QWEN3_TTS_PORT, PARAKEET_ASR_PORT, CANARY_ASR_PORT, CHATTERBOX_TTS_PORT, MAGPIE_TTS_PORT, TRAINING_PORT.

---

Local Development

The base docker-compose.yml runs from the built images (deployment-ready). For development, add the docker-compose.dev.yml overlay, which bind-mounts service source files so code changes apply on container restart — no rebuild needed:

docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile all up -d
docker compose restart stt-service   # pick up an edited file

All processing is local. No data is sent to external services.

Tests, lint and generated files

pip install -r tests/requirements.txt
pytest tests/                       # ~2 minutes; tests that need a running stack skip themselves
ruff check .                        # syntax errors and undefined names only (ruff.toml)
python scripts/sync_openapi.py      # regenerate the OpenAPI specs after changing a route or model

The suite is offline: the heavy dependencies (torch, NeMo, faster-whisper, librosa) are stubbed, so it needs only the packages in tests/requirements.txt. Besides the unit tests it holds config-drift suites that fail when compose, .env.example, the device presets, the Dockerfiles and the OpenAPI specs disagree with each other (REQUIRE_DRIFT_SUITES=1, set in CI, turns a missing dependency into a failure instead of a skip). CI (.github/workflows/ci.yml) runs them on Python 3.11 and 3.12 together with lint, docker compose config for every overlay, actionlint and a JavaScript syntax check; shellcheck, hadolint, a dependency audit and the training and browser suites report without failing the run. Images are published by .github/workflows/publish-images.yml only after those checks pass: a vX.Y.Z tag publishes X.Y.Z, X.Y and latest, a push to master publishes latest, and a manual run never touches latest or a version tag.

Storage location

Persistent data (models, output, caches, training data) defaults to ./ under the repo. Set APP_DATA_DIR=/path/to/data (e.g. a TrueNAS dataset) to relocate everything with one variable; see docs/truenas-deployment.md.

---

TrueNAS Deployment

Runs on TrueNAS SCALE 24.10 (Electric Eel) or newer as a Custom App from prebuilt images: no clone and no build. All data lives in one dataset and only the web UI port (3000) is published; the frontend proxies every backend call, live microphone transcription included. docs/truenas-installation-guide.md is the complete guide (install, update, roll back, VRAM planning, troubleshooting).

Install in 5 minutes

1. Create a dataset, for example /mnt/tank/apps/tts-stt.
2. Optional, in the TrueNAS shell: scripts/truenas/preflight.sh --data-dir /mnt/tank/apps/tts-stt checks Docker, the NVIDIA driver, VRAM, disk space and the port (the guide shows how to fetch the scripts without cloning the repository).
3. Apps → Discover Apps → Custom App → Install via YAML: paste docker-compose.truenas-app.yml, replace ${APP_DATA_DIR:?set dataset path} (every occurrence) with your dataset path, install.
4. Open http://<truenas-ip>:3000. Models download in the background on the first start.

The images (ghcr.io/steffenhebestreit/tts-stt-*) are about 25 GB and TrueNAS aborts an Apps job after 20 minutes: run scripts/truenas/pull-images.sh first on a slow link.

Most settings are then changed in the web UI, not in the YAML: Settings (the gear, or http://<truenas-ip>:3000/settings) holds host names, API access and keys, which optional engines are offered, the default engines and the gateway limits. They apply at once and are kept in the dataset, so Edit, Update and Stop/Start keep them. The YAML keeps what needs a container to be recreated: the release, the GPU, the port and the profiles: lines that install optional services.

Updates

The compose file pins a release (IMAGE_TAG), so nothing changes until you change it. scripts/truenas/update-check.sh is a read-only check that prints UPDATE AVAILABLE when a newer release (or a newer image for a floating tag) is published. To update: snapshot the dataset, change the release in the app's YAML, Save; to roll back, put the previous release back (data folders are only added to). IMAGE_TAG:-latest with PULL_POLICY:-always follows the newest build instead, and an optional diun service sends notifications only. Details are in the guide.

Other routes

• Settings file: the same compose file with a settings.env on the dataset, so an update is one changed line (truenas/custom-app-include.yml). Not needed on TrueNAS now that most settings live in the web UI; it stays for running the file from a shell.
• Custom catalog (a form instead of YAML): truenas/tts-stt/ is a scaffold that TrueNAS cannot load as it stands, see truenas/README.md.
• Build from source on the NAS with docker-compose.truenas.yml: docs/truenas-deployment.md.

Requirements and sizing

• An NVIDIA driver that supports CUDA 12.8 for the GPU services (R570 or newer; RTX 50 series cards require it), installed from Apps → Settings → Install NVIDIA Drivers.
• The default set (Whisper large-v3-turbo, Qwen3-ASR, Qwen3-TTS 0.6B) is about 8 GB of VRAM resident; optional services add more (Canary, Parakeet, Chatterbox, Magpie, training). Every GPU model is freed after MODEL_TTL seconds idle (300), so the sum need not fit at once. Budget against what nvidia-smi reports for your card; the per-service table is in the guide.
• Remote browser access needs nothing beyond port 3000 (ALLOWED_ORIGINS stays empty) as long as you open it by IP address, *.local or a plain name such as truenas. A real domain or a reverse proxy needs its name under Settings -> Access or in TRUSTED_HOSTS (and the proxy's URL, TRUSTED_ORIGINS, if it rewrites Host). The microphone from another machine needs HTTPS.

Cleanup (optional)

Remove leftover XTTS cache from an earlier version, in the repository checkout (requires root):

sudo rm -rf cache/tts cache/xtts-cache   # ~3.6 GB
sudo rm -f models/luna.json models/luna.onnx  # old root-level export

API

The stack is usable as a drop-in OpenAI-compatible speech API — point the official SDK at the gateway and it works unchanged across every device:

from openai import OpenAI
client = OpenAI(base_url="http://your-host:3000/v1", api_key="unused")  # a key if the gateway requires one
client.audio.transcriptions.create(model="whisper-1", file=open("audio.wav", "rb")).text

POST /v1/audio/transcriptions (json, text, verbose_json, srt, vtt) · POST /v1/audio/speech (mp3, wav, pcm) · GET /v1/models

The same request returns the same response shape whether the backend is whisper-cpp on an ARM SBC or faster-whisper on a workstation GPU. Full reference, including the language-handling rules that matter for German: docs/api.md.

The richer project-native /api/* surface (segment timestamps, streaming TTS, voice training) is unchanged and documented in docs/provider-contracts.md.