GM Voice Studio (Pocket TTS)

AI voice engine: use built-in voices or clone a voice from a short recording, then generate speech with Pocket TTS (Kyutai). English only; CPU-optimized, no GPU required.

How to start all the servers

Option A — Docker (one service, recommended)
Backend serves the main UI and the built React app at /preview. No separate frontend process.

# Optional: include React app in image (from project root)
cd preview-react && npm install && npm run build && cd ..
docker build -t kani-tts .
docker run -p 7862:7862 -v kani-voice_storage:/app/voice_storage --env-file .env kani-tts

Use --env-file .env so the container gets ANTHROPIC_API_KEY, DEEPGRAM_API_KEY, HF_TOKEN, etc. from your project root .env.

• Main UI: http://localhost:7862
• DM Preview (React): http://localhost:7862/preview (only if you ran npm run build in preview-react before docker build; otherwise the legacy preview HTML is used).

Option B — Docker Compose (API + optional Redis)
Same as above, with optional Redis for async clone/narrate:

docker compose up -d app
# Optional: start Redis + Celery worker (set CELERY_BROKER_URL=redis://redis:6379/0 in app env):
# docker compose --profile celery up -d redis
# docker compose run --rm -e CELERY_BROKER_URL=redis://redis:6379/0 app celery -A celery_app worker --loglevel=info

Option C — Local: backend only (single process)
One Python process serves everything (including /preview if you’ve built the React app).

# From project root, with venv activated
pip install -r requirements-core.txt && pip install -r requirements-server.txt && pip install -r requirements-rag.txt
python server.py

• Main UI: http://localhost:7862
• DM Preview: http://localhost:7862/preview (after cd preview-react && npm run build)

Option D — Local: backend + React dev server (two processes)
Backend for API; Vite dev server for React with hot reload. Use when actively developing the frontend.

1. Terminal 1 — Backend

   pip install -r requirements-core.txt && pip install -r requirements-server.txt && pip install -r requirements-rag.txt
   python server.py
   

2. Terminal 2 — React (Vite)

   cd preview-react && npm install && npm run dev
   

• Main UI: http://localhost:7862
• DM Preview (dev): http://localhost:5173 (Vite proxies /api, /adventure, /tts, /voices, /ws, etc. to http://localhost:7862)

If the frontend is on a different origin, set CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173 in the backend env.

---

Run the server (detailed)

# From project root, with venv activated
pip install -r requirements-core.txt && pip install -r requirements-server.txt
# PDF parsing and RAG/Co-DM: pip install -r requirements-rag.txt
python server.py

Python 3.10–3.14, PyTorch 2.5+. Pocket TTS runs on CPU by default and does not require a GPU.

If pip reports "resolution-too-deep": Install in order one package at a time:

pip install torch>=2.5.0
pip install "soundfile>=0.13.0"
pip install pocket-tts
pip install fastapi uvicorn slowapi "gradio>=6.6.0"
python server.py

• Web UI: http://localhost:7862 (default port; override with PORT env var).
• Interactive API docs: http://localhost:7862/docs.

React Preview UI (/preview)

/preview now serves a React app build when available (fallback: legacy static/index.preview.html).

# One-time install
cd preview-react
npm install

# Build into static/preview-react (served by FastAPI at /preview)
npm run build

# Optional local dev server for frontend iteration
npm run dev

After npm run build, open http://localhost:7862/preview.

Config (env)

Optional features (see config.py for full list):

API overview

• GET / – Web UI (TTS, voice clone from upload or mic, script narration, Export WAV).
• GET /health – Liveness: {"status":"ok","service":"kani-tts"}.
• GET /ready – Readiness: 503 until TTS model has been loaded (use for load balancer probe).
• GET /config – Client config, e.g. {"require_api_key": true}.
• GET /voices – Returns language_tags (e.g. ["en"]) and preset_voices (e.g. ["alba", "marius", ...]).
• GET /limits – Narrate limits: max_narrate_chars, max_narrate_chunks.
• POST /tts – Generate speech: form fields text, language_tag (ignored; English only), voice_id (preset name or cloned voice ID), optional temperature, top_p, repetition_penalty; optional file reference_audio for one-off clone. Returns WAV.
• POST /voices/clone – Create persistent voice: form fields audio (file), optional name, consent_scope, faction; returns voice_id or (when Celery enabled) job_id.
• GET /jobs/{job_id} – When Celery enabled: poll clone (or async) job status; when completed, includes voice_id.
• POST /tts/narrate – Long-form: JSON text, voice_id (preset or cloned), optional language_tag, chunk_by, max_chars; returns WAV.
• GET /voices/list, GET /voices/{id}, PATCH /voices/{id}, DELETE /voices/{id} – List and manage cloned voices.
• DELETE /admin/voices/{voice_id} – Take-down (requires X-Admin-Key when ADMIN_API_KEY is set).

Full request/response schemas: http://localhost:7862/docs (or your host/port).

Built-in and cloned voices

• Preset voices: Pocket TTS includes built-in voices (alba, marius, javert, jean, fantine, cosette, eponine, azelma). Use voice_id set to the preset name (e.g. alba) for TTS or narrate without cloning.
• Cloned voices: Upload a short clean sample (WAV/MP3) to create a persistent voice; it is stored as a .safetensors file and appears in the voice list.

Testing

pip install -r requirements-dev.txt
pytest tests/ -v

By default, slow tests (POST /tts, which loads the model) are skipped. To run them: pytest tests/ -v -m slow.

Deploy

Docker: Build and run the API (default port 7862). Pass your .env so the container has API keys (e.g. ANTHROPIC_API_KEY, DEEPGRAM_API_KEY, HF_TOKEN):

docker build -t kani-tts .
docker run -p 7862:7862 -v kani-voice_storage:/app/voice_storage --env-file .env kani-tts

Docker Compose: API + Redis (for optional async clone/narrate):

docker compose up -d app
# With Redis and Celery worker (set CELERY_BROKER_URL=redis://redis:6379/0 in app env):
# docker compose --profile celery up -d redis && docker compose run -e CELERY_BROKER_URL=redis://redis:6379/0 app celery -A celery_app worker --loglevel=info

Env: Set PORT, VOICE_STORAGE_PATH (or use a volume), and optionally API_KEYS, REQUIRE_API_KEY, DATABASE_URL, CELERY_BROKER_URL, CORS_ORIGINS (see Config table). For production, back up voice_storage and your database (SQLite file or PostgreSQL).

Use from a TTRPG app (GM Voice Studio API)

The web UI is a GM-focused voice studio; the same API can be called from your TTRPG app (VTT, companion app, or bot).

Auth: If the server has REQUIRE_API_KEY=1, send X-API-Key: <your-key> (or Authorization: Bearer <key>) on every request. Voices are scoped per key when API_KEYS and a DB are configured.

Endpoints and example payloads:

• Create a character voice: POST /voices/clone — form: audio (file), optional name, consent_scope (e.g. tts or commercial), faction. Returns voice_id or (with Celery) job_id; poll GET /jobs/{job_id} until done.
• List voices: GET /voices/list — returns cloned voices. Use GET /voices for preset voice names.
• Speak a line: POST /tts — form: text, optional voice_id (preset name or cloned ID), temperature, top_p, repetition_penalty. Returns WAV bytes.
• Narrate a scene: POST /tts/narrate — JSON: { "text": "...", "voice_id": "alba", "language_tag": "en", "chunk_by": "sentence", "max_chars": 500 }. Returns WAV. For long scripts, use "async": true when Celery is configured; then poll GET /jobs/{job_id} and fetch WAV from GET /jobs/{job_id}/result.

Example (preset voice then clone):

# Speak with built-in voice
curl -X POST http://localhost:7862/tts -F "text=You approach the gates." -F "voice_id=alba" --output out.wav
# Clone a voice (after uploading audio)
curl -X POST http://localhost:7862/voices/clone -F "audio=@sample.wav" -F "name=Dragon Queen" -H "X-API-Key: YOUR_KEY"
# Then speak as that voice
curl -X POST http://localhost:7862/tts -F "text=You approach the gates." -F "voice_id=VOICE_ID" -H "X-API-Key: YOUR_KEY" --output out.wav

Voice cloning

• Upload: WAV/MP3, 3–120 s (Pocket TTS). Clean speech works best.
• Mic: Record in the UI; recording can be played back before creating the voice. Use "Re-record" to clear and try again.
• Cloned voices are stored as .safetensors and appear in the voice dropdown with built-in presets.
• When Celery is configured, clone returns a job_id; the UI polls until the job completes and then shows the new voice_id.

Troubleshooting

PDF parsing requires 'pymupdf4llm'
Install RAG dependencies: pip install -r requirements-rag.txt. With Docker, the image already includes them; if you see this in a container, rebuild the image (docker build -t kani-tts .) and run again.

404 on GET /api/campaigns or POST /adventure/ai-parse
The server build you’re running doesn’t include these routes. Rebuild and restart so the process uses the latest code (e.g. docker build -t kani-tts . then docker run ... again, or restart python server.py after pulling). Ensure DATABASE_URL is set (e.g. sqlite:///./voice_metadata.db) so /api/campaigns can store data.

403 on WebSocket /ws/audio
Usually the same cause: an older server build that rejects the WebSocket. Rebuild/restart the server. If the frontend is on a different origin (e.g. Vite at http://localhost:5173 and API at http://localhost:7862), set CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173 in the server env and restart.

Bind for 0.0.0.0:7862 failed: port is already allocated
Something is already using port 7862. Stop it, then run the container again. Examples:

• Stop the existing container: docker stop kani-tts-app (or the name from docker ps).
• If you run the backend locally: stop python server.py in that terminal.
• Or run the container on another port: docker run -p 7863:7862 -v kani-voice_storage:/app/voice_storage --env-file .env kani-tts and use http://localhost:7863.